A spatial registration method for a mobile markerless surgical navigation system

Abstract

The present invention discloses a movable markerless surgical navigation system spatial registration method, comprising: extracting a CT image of a target position and obtaining a spatial point cloud of the target outer surface image; installing a positioning tool directly above a designed structured light camera housing; calibrating the structured light camera and the positioning tool using a camera board and an optical positioning system; moving a handheld structured light camera to collect the spatial point cloud of the target outer surface; converting the spatial point cloud to the surgical space of the optical positioning system based on the calibration result and the coordinate transformation relationship of the positioning tool obtained in real time by the optical positioning system; then performing point cloud merging and segmentation to obtain a surgical space point cloud of the target outer surface; using the surgical space point cloud as a target point set, first coarsely registering the image space point cloud to the surgical space point cloud, and then using fine registration to improve the registration accuracy, thereby completing registration. The present invention can greatly improve the flexibility of surgical registration, reduce the complexity of surgical registration, and at the same time ensure surgical registration accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of optical surgical navigation space registration, and in particular to a movable markerless surgical navigation system space registration method. Background Art

[0002] With the recent development of optical positioning systems in the medical field, there is a need to match the CT image space with the surgical navigation space to enable robot navigation and control within optical positioning systems. Currently, most clinically applied methods use marker-based spatial registration or probe-based surface registration. Marker-based spatial registration methods require preoperative markers to be attached to the medical image for scanning and then their coordinates extracted. During surgery, the markers are identified by the optical positioning system and then used to match the medical image space with the surgical navigation space. Probe-based surface registration methods are not applicable to soft tissue and require manual operation. Structured light cameras can achieve markerless surgical space registration, but currently, most methods fix the optical positioning system and structured light camera positions, then calibrate the spatial relationship between them on-site. The structured light camera then acquires a surface point cloud, which is converted to a point cloud of the surgical space and then matched to the medical image space. A problem with existing technologies is that structured light cameras cannot be moved during use, capturing only a partial point cloud, not the full picture. Calibration is required for each use, making the process complex. Summary of the Invention

[0003] The purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology and propose a movable markerless surgical navigation system spatial registration method, which fixes the positioning tool just above the structured light camera so that it can be captured by the optical positioning system in real time, and calibrates the fixed coordinate transformation relationship between the positioning tool and the structured light camera. Through this relationship, the point cloud obtained by the structured light camera can be converted to the surgical space in real time, which greatly improves the flexibility of surgical registration, reduces the complexity of surgical registration, and ensures sufficient surgical registration accuracy.

[0004] To achieve the above objectives, the present invention provides a technical solution: a method for spatial registration of a mobile markerless surgical navigation system, comprising the following steps:

[0005] S1: Extract the CT image of the target position, process the CT image and obtain the target outer surface image space point cloud V, and at the same time install the positioning tool A just above the housing of the designed structured light camera C;

[0006] S2: Calculate the coordinate transformation relationship between the structured light camera C and the positioning tool A in step S1 through the calibration plate B and the optical positioning system O [R AC ,T AC ], where R ACT represents the rotation matrix of the coordinate system centered on the structured light camera C in the coordinate system centered on the positioning tool A. AC Represents the translation matrix of the coordinate system centered on the structured light camera C in the coordinate system centered on the positioning tool A;

[0007] S3: The handheld structured light camera C moves to collect the outer surface space point cloud P of the target position, according to the [R AC ,T AC ] and the optical positioning system O to obtain the coordinate transformation relationship of the positioning tool A in real time [R' OA ,T' OA ]Convert the spatial point cloud P to the surgical space of the optical positioning system, and then merge and segment the point cloud to obtain the target outer surface surgical space point cloud H, where R' OA It represents the real-time rotation matrix of the coordinate system centered on the positioning tool A in the coordinate system centered on the optical positioning system O, T' OA represents the real-time translation matrix of the coordinate system centered on the positioning tool A in the coordinate system centered on the optical positioning system O;

[0008] S4: Use the surgical space point cloud H obtained in step S3 as the target point set, first coarsely align the image space point cloud V obtained in step S1 to the surgical space point cloud H, and then use fine registration to improve the registration accuracy, thereby completing the surgical space registration of the markerless surgical navigation system.

[0009] Furthermore, in step S1, a CT image space coordinate system is established based on the image sequence obtained by the CT scan, the image space coordinates are obtained according to the starting position of the scan, the voxel spacing and the slice dimension, the model point cloud is obtained by volume mapping, and the target outer surface image space point cloud V is obtained by traversing the model point cloud; a hole is opened directly above the housing of the structured light camera C, and the positioning tool A is fixed flatly directly above the housing of the structured light camera C by using screws; a hole is opened at the rear of the housing of the structured light camera C, and the handle is fixed directly below the housing of the structured light camera C by using screws.

[0010] Furthermore, the specific operation steps of step S2 are as follows:

[0011] S21: Fix the calibration plate B, the optical positioning system O, and the structured light camera C with the positioning tool A respectively, so that the calibration plate B can be recognized by the optical positioning system O and the structured light camera C, and the positioning tool A can be recognized by the optical positioning system O;

[0012] S22: Use the optical positioning system O to obtain the coordinate transformation relationship between the calibration plate B and the positioning tool A [R OB ,T OB ] and [R OA ,T OA], use the structured light camera C to obtain the coordinate transformation relationship with the calibration plate B [R CB ,T CB ], where R OB It represents the rotation matrix of the coordinate system centered on the calibration plate B in the coordinate system centered on the optical positioning system O, T OB It represents the translation matrix of the coordinate system centered on the calibration plate B in the coordinate system centered on the optical positioning system O, R OA It represents the rotation matrix of the coordinate system centered on the positioning tool A in the coordinate system centered on the optical positioning system O during the calibration process, T OA It represents the translation matrix of the coordinate system centered on the positioning tool A in the coordinate system centered on the optical positioning system O during the calibration process, R CB It represents the rotation matrix of the coordinate system centered on the calibration plate B in the coordinate system centered on the structured light camera C, T CB Represents the translation matrix of the coordinate system centered on the calibration plate B in the coordinate system centered on the structured light camera C;

[0013] S23: According to the following coordinate conversion formula:

[0014]

[0015] T OA =-R OA *T AO

[0016]

[0017]

[0018] The coordinate transformation relationship between the positioning tool A and the structured light camera C can be obtained [R AC ,T AC ], where R AO It represents the rotation matrix of the coordinate system centered on the optical positioning system O in the coordinate system centered on the positioning tool A during the calibration process, Represents the rotation matrix R AO The inverse, Represents the rotation matrix R AO The transpose of T AO It represents the translation matrix of the coordinate system centered on the optical positioning system O in the coordinate system centered on the positioning tool A during the calibration process, Represents the rotation matrix R CB The inverse of.

[0019] Furthermore, in step S3, the handheld structured light camera moves to collect the outer surface space point cloud P of the target position, and the optical positioning system O obtains the coordinate transformation relationship [R' of the positioning tool A in real time OA ,T' OA ], according to the following formula:

[0020] R OC =R′ OA *R AC

[0021] T OC =R′ OA *T AC +T′ OA

[0022] The coordinate transformation relationship between the optical positioning system O and the structured light camera C can be obtained in real time [R OC ,T OC ], where R OC It represents the rotation matrix of the coordinate system centered on the structured light camera C in the coordinate system centered on the optical positioning system O, T OC Represents the translation matrix of the coordinate system centered on the structured light camera C in the coordinate system centered on the optical positioning system O. According to the relationship [R OC ,T OC ]The outer surface space point cloud P is converted to the optical positioning system surgical space, and then the point cloud is merged and segmented to obtain the target outer surface surgical space point cloud H.

[0023] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0024] 1. The method of the present invention uses a handheld structured light camera as a surface point cloud acquisition device, which is aligned with preoperative CT image data to achieve mobile fully automatic surgical registration.

[0025] 2. The method of the present invention can realize multi-angle surface point cloud acquisition through a mobile structured light camera, and process the point cloud data to obtain more comprehensive and holistic point cloud data. At the same time, it is easy to carry and has a wider range of applicable scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Schematic diagram of the process of the present invention.

[0027] Figure 2 Schematic diagram of CT image processing in an embodiment of the present invention.

[0028] Figure 3 Schematic diagram of the structured light camera housing and positioning tool in an embodiment of the present invention.

[0029] Figure 4Schematic diagram of calibration of structured light camera and positioning tool in an embodiment of the present invention.

[0030] Figure 5 Schematic diagram of point cloud registration in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The present invention will be described in further detail below with reference to the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0032] like Figure 1 As shown, this embodiment provides a method for spatial registration of a movable markerless surgical navigation system, which includes the following steps:

[0033] S1: Extract the CT image of the target position, process the CT image and obtain the target outer surface image space point cloud V, and at the same time install the positioning tool A directly above the housing of the designed structured light camera C.

[0034] like Figure 2 As shown in FIG, a CT image space coordinate system is established based on the image sequence obtained by CT scanning, and the image space coordinates are obtained according to the scanning information such as the starting position of the scan, the voxel spacing and the slice dimension. The model point cloud is obtained by volume mapping, and the target outer surface image space point cloud V is obtained by traversing the model point cloud; as shown in FIG. Figure 3 As shown, a hole is opened on the upper part of the structured light camera C housing, and the positioning tool A is fixed flatly on the upper part of the structured light camera C housing by screws. A hole is opened on the rear part of the structured light camera C housing, and the handle is fixed on the lower part of the structured light camera C housing by screws.

[0035] S2: Calculate the coordinate transformation relationship between the structured light camera C and the positioning tool A in step S1 through the calibration plate B and the optical positioning system O [R AC ,T AC ], where R AC T represents the rotation matrix of the coordinate system centered on the structured light camera C in the coordinate system centered on the positioning tool A. AC Represents the translation matrix of the coordinate system centered on the structured light camera C in the coordinate system centered on the positioning tool A; Figure 4 The specific steps are as follows:

[0036] S21: Fix the calibration plate B, the optical positioning system O, and the structured light camera C with the positioning tool A respectively, so that the calibration plate B can be recognized by the optical positioning system O and the structured light camera C, and the positioning tool A can be recognized by the optical positioning system O;

[0037] S22: Use the optical positioning system O to obtain the coordinate transformation relationship between the calibration plate B and the positioning tool A [R OB ,T OB] and [R OA ,T OA ], use the structured light camera C to obtain the coordinate transformation relationship with the calibration plate B [R CB ,T CB ], where R OB It represents the rotation matrix of the coordinate system centered on the calibration plate B in the coordinate system centered on the optical positioning system O, T OB It represents the translation matrix of the coordinate system centered on the calibration plate B in the coordinate system centered on the optical positioning system O, R OA It represents the rotation matrix of the coordinate system centered on the positioning tool A in the coordinate system centered on the optical positioning system O during the calibration process, T OA It represents the translation matrix of the coordinate system centered on the positioning tool A in the coordinate system centered on the optical positioning system O during the calibration process, R CB It represents the rotation matrix of the coordinate system centered on the calibration plate B in the coordinate system centered on the structured light camera C, T CB Represents the translation matrix of the coordinate system centered on the calibration plate B in the coordinate system centered on the structured light camera C;

[0038] S23: According to the following coordinate conversion formula:

[0039]

[0040] T OA =-R OA *T AO

[0041]

[0042]

[0043] The coordinate transformation relationship between the positioning tool A and the structured light camera C can be obtained [R AC ,T AC ], where R AO It represents the rotation matrix of the coordinate system centered on the optical positioning system O in the coordinate system centered on the positioning tool A during the calibration process, Represents the rotation matrix R AO The inverse, Represents the rotation matrix R AO The transpose of T AO It represents the translation matrix of the coordinate system centered on the optical positioning system O in the coordinate system centered on the positioning tool A during the calibration process, Represents the rotation matrix R CB The inverse of.

[0044] S3: The handheld structured light camera moves to collect the outer surface space point cloud P of the target position, and the optical positioning system O obtains the coordinate transformation relationship of the positioning tool A in real time [R' OA ,T' OA ], where R' OA It represents the real-time rotation matrix of the coordinate system centered on the positioning tool A in the coordinate system centered on the optical positioning system O, T' OA It represents the real-time translation matrix of the coordinate system centered on the positioning tool A in the coordinate system centered on the optical positioning system O; according to the following formula:

[0045] R OC =R′ OA *R AC

[0046] T OC =R′ OA *T AC +T′ OA

[0047] The coordinate transformation relationship between the optical positioning system O and the structured light camera C can be obtained in real time [R OC ,T OC ], where R OC It represents the rotation matrix of the coordinate system centered on the structured light camera C in the coordinate system centered on the optical positioning system O, T OC Represents the translation matrix of the coordinate system centered on the structured light camera C in the coordinate system centered on the optical positioning system O. According to the relationship [R OC ,T OC ]The outer surface space point cloud P is converted to the optical positioning system surgical space, and then the point cloud is merged and segmented to obtain the target outer surface surgical space point cloud H.

[0048] S4: As Figure 5 As shown, the surgical space point cloud H obtained in step S3 is used as the target point set, the image space point cloud V obtained in step S1 is first coarsely registered to the surgical space point cloud H, and then fine registration is used to improve the registration accuracy, thereby completing the surgical space registration of the markerless surgical navigation system.

[0049] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A spatial registration method for a mobile markerless surgical navigation system, characterized in that: The following steps are involved: S1: Extract the CT image of the target position, process the CT image and obtain the target outer surface image space point cloud V, and at the same time install the positioning tool A just above the housing of the designed structured light camera C; S2: Calculate the coordinate transformation relationship between the structured light camera C and the positioning tool A in step S1 through the calibration plate B and the optical positioning system O [R AC ,T AC ], where R AC T represents the rotation matrix of the coordinate system centered on the structured light camera C in the coordinate system centered on the positioning tool A. AC Represents the translation matrix of the coordinate system centered on the structured light camera C in the coordinate system centered on the positioning tool A; S3: The handheld structured light camera C moves to collect the outer surface space point cloud P of the target position, according to the [R AC ,T AC ] and the optical positioning system O to obtain the coordinate transformation relationship of the positioning tool A in real time [R' OA ,T' OA ]Convert the spatial point cloud P to the surgical space of the optical positioning system, and then merge and segment the point cloud to obtain the target outer surface surgical space point cloud H, where R' OA It represents the real-time rotation matrix of the coordinate system centered on the positioning tool A in the coordinate system centered on the optical positioning system O, T' OA represents the real-time translation matrix of the coordinate system centered on the positioning tool A in the coordinate system centered on the optical positioning system O; S4: Use the surgical space point cloud H obtained in step S3 as the target point set, first coarsely align the image space point cloud V obtained in step S1 to the surgical space point cloud H, and then use fine registration to improve the registration accuracy, thereby completing the surgical space registration of the markerless surgical navigation system.

2. A method for spatial registration of a mobile markerless surgical navigation system according to claim 1, characterized in that: In step S1, a CT image space coordinate system is established based on the image sequence obtained by the CT scan, the image space coordinates are obtained according to the starting position of the scan, the voxel spacing and the slice dimension, the model point cloud is obtained by volume mapping, and the target outer surface image space point cloud V is obtained by traversing the model point cloud; a hole is opened directly above the housing of the structured light camera C, and the positioning tool A is fixed flatly directly above the housing of the structured light camera C by using screws; a hole is opened at the rear of the housing of the structured light camera C, and the handle is fixed directly below the housing of the structured light camera C by using screws.

3. A method for spatial registration of a mobile markerless surgical navigation system according to claim 2, characterized in that: The specific operation steps of step S2 are as follows: S21: Fix the calibration plate B, the optical positioning system O, and the structured light camera C with the positioning tool A respectively, so that the calibration plate B can be recognized by the optical positioning system O and the structured light camera C, and the positioning tool A can be recognized by the optical positioning system O; S22: Use the optical positioning system O to obtain the coordinate transformation relationship between the calibration plate B and the positioning tool A [R OB ,T OB ] and [R OA ,T OA ], use the structured light camera C to obtain the coordinate transformation relationship with the calibration plate B [R CB ,T CB ], where R OB It represents the rotation matrix of the coordinate system centered on the calibration plate B in the coordinate system centered on the optical positioning system O, T OB It represents the translation matrix of the coordinate system centered on the calibration plate B in the coordinate system centered on the optical positioning system O, R OA It represents the rotation matrix of the coordinate system centered on the positioning tool A in the coordinate system centered on the optical positioning system O during the calibration process, T OA It represents the translation matrix of the coordinate system centered on the positioning tool A in the coordinate system centered on the optical positioning system O during the calibration process, R CB It represents the rotation matrix of the coordinate system centered on the calibration plate B in the coordinate system centered on the structured light camera C, T CB Represents the translation matrix of the coordinate system centered on the calibration plate B in the coordinate system centered on the structured light camera C; S23: According to the following coordinate conversion formula: T OA =-R OA *T AO The coordinate transformation relationship between the positioning tool A and the structured light camera C can be obtained [R AC ,T AC ], where R AO It represents the rotation matrix of the coordinate system centered on the optical positioning system O in the coordinate system centered on the positioning tool A during the calibration process, Represents the rotation matrix R AO The inverse, Represents the rotation matrix R AO The transpose of T AO It represents the translation matrix of the coordinate system centered on the optical positioning system O in the coordinate system centered on the positioning tool A during the calibration process, Represents the rotation matrix R CB The inverse of.

4. A method for spatial registration of a mobile markerless surgical navigation system according to claim 3, characterized in that: In step S3, the handheld structured light camera moves to collect the outer surface space point cloud P of the target position, and the optical positioning system O obtains the coordinate transformation relationship of the positioning tool A in real time [R' OA ,T' OA ], according to the following formula: R OC =R′ OA *R AC T OC =R′ OA *T AC +T′ OA The coordinate transformation relationship between the optical positioning system O and the structured light camera C can be obtained in real time [R OC ,T OC ], where R OC It represents the rotation matrix of the coordinate system centered on the structured light camera C in the coordinate system centered on the optical positioning system O, T OC Represents the translation matrix of the coordinate system centered on the structured light camera C in the coordinate system centered on the optical positioning system O. According to the relationship [R OC ,T OC ]The outer surface space point cloud P is converted to the optical positioning system surgical space, and then the point cloud is merged and segmented to obtain the target outer surface surgical space point cloud H.

Images