A real-time surgical registration method for approximating respiratory states

Abstract

The present invention relates to a real-time surgical registration method for approximating the respiratory state, comprising: importing a CT sequence acquired during surgery, extracting the skin contour of the patient's upper surface and reconstructing it into a three-dimensional point set H. Using a depth camera to acquire and segment the three-dimensional point set V of the patient's skin surface in real time. Aligning the source point set H to the target point set V in real time and feeding back the error of each alignment. According to the error data, an error curve of the alignment error and the progress time is drawn in real time. The execution is continued until the error curve of at least one respiratory cycle is acquired and drawn. Observe the trend of the alignment curve, and when the error approaches the minimum value, control the patient to slow down the breathing, and control the patient to hold the breath near the minimum value of the alignment error. Apply the spatial transformation relationship obtained by the point set alignment at this time to the source point set H to realize surgical registration. The present invention effectively approximates the respiratory state when the CT scan is taken before surgery, does not need to stick or tie the detection object on the patient's body surface, and is a contactless and more accurate surgical registration method.

Description

Technical Field

[0001] The present invention relates to the technical field of surgical navigation and point cloud registration, and in particular to a real-time surgical registration method for approximating a respiratory state. Background Art

[0002] Surgical registration is a key technology in surgical navigation. Through surgical registration, the mapping between image space and surgical space can be achieved, so as to reflect the real surgical situation in the virtual environment, accurately position the surgical instruments, and assist doctors to complete puncture and ablation surgery quickly and accurately.

[0003] However, during interventional surgery on the corresponding organs of the chest and abdomen, the positions of the internal organs and lesions in the body also change with breathing due to the influence of respiratory movement. Therefore, the CT scan taken before surgery cannot actually reflect the real-time physiological and anatomical structure of the body and the distribution of lesions during surgery. Some scholars have pointed out that under uniform breathing, the breathing of the human body shows periodic changes, and the actual physiological and anatomical structure of the body also shows periodic changes. Moreover, when the same breathing state is in different respiratory cycles, it can be considered that the physiological and anatomical structures at these two moments are highly similar.

[0004] At present, academia and clinical medicine have proposed a variety of methods for approximating breathing status, such as tying a pressure belt, holding a gas flow meter in the mouth, and sticking markers on the body surface. Some scholars have used CT sequences at multiple times to prove that the accuracy of using the skin surface to map the anatomical structure in the body is higher than low-dimensional detection methods such as pressure belts, meters, and markers. In addition, while approximating breathing through point cloud registration, it can also output coordinate transformation relationships to achieve surgical registration functions.

[0005] The disclosure of the above background technology content is only used to assist in understanding the inventive concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application. In the absence of clear evidence that the above content has been disclosed on the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of the present application. Summary of the invention

[0006] The main purpose of the present invention is to overcome the shortcomings of the prior art and provide a real-time surgical registration method for approximating the respiratory state. The method can effectively approximate the respiratory state during preoperative CT scanning without the need to stick or tie detection objects on the patient's body surface, and can provide a contactless and more accurate surgical registration method.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A real-time surgical registration method for approximating a respiratory state comprises the following steps:

[0009] S1: Import the CT sequence acquired during the operation, process the CT image, extract the skin contour of the patient's upper surface and reconstruct it into a three-dimensional point set H;

[0010] S2: Let the patient breathe normally, use the depth camera to collect the three-dimensional coordinate point set of depth data in real time, and segment the three-dimensional point set V of the patient's skin surface from the three-dimensional point set. The point set V will be updated every time the data is collected;

[0011] S3: align the source point set H to the target point set V in real time and feedback the error of each alignment;

[0012] S4: Based on the error data, the software draws the error curve of registration error and progress time in real time and displays it visually for observation;

[0013] S5: Continue to execute steps S2, S3, and S4 until an error curve of at least one respiratory cycle (expiratory and inspiratory cycle) is collected and drawn;

[0014] S6: Observe the trend of the registration curve. When the error is close to the minimum value, control the patient to slow down breathing, and control the patient to hold his breath near the minimum value of the registration error.

[0015] S7: Apply the spatial transformation relationship obtained by the point set registration at this time to the source point set H to achieve surgical registration.

[0016] "Real time" means "greater than or equal to 5 times per second".

[0017] In step S1, the CT image is subjected to windowing processing, binarization processing and contour extraction in succession to obtain a two-dimensional skin contour, and then the upper quarter of the contour is intercepted as the upper surface skin contour.

[0018] In step S2, depth data is collected in real time, a height allowable range is set, and the three-dimensional point set of non-related objects is filtered out according to the height of the measured object from the camera, thereby segmenting the three-dimensional point set of the skin area.

[0019] In step S3, the point set H is registered with the point set V in real time using a point cloud registration method, and the similarity between the skin surfaces represented by the point set H and the point set V is evaluated by the output registration error.

[0020] In step S4, a graph of the registration error and the processing time is drawn using computer software, with the output error as the vertical axis and the processing time as the horizontal axis. The graph is displayed on a corresponding screen to facilitate observation of the error trend.

[0021] In step S6, the error curve is always followed, and the patient is controlled to hold his breath near the minimum value. The minimum error value indicates that the breathing state at this moment is closest to the breathing state when the CT scan is taken, and the physiological and anatomical structure is closest.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] 1. The present invention provides a real-time surgical registration method for approximating the breathing state. The method uses a depth camera to collect surface point clouds and CT segmented surface point clouds for real-time registration and feedback of errors. The approximated breathing situation is judged according to the error value. While approximating breathing, the anatomical structure is transformed from the CT image space to the surgical space, thereby realizing surgical registration.

[0024] 2. The present invention can effectively approximate the respiratory state during preoperative CT scans, without the need to stick or tie detection objects on the patient's body surface, and can provide a contactless and more accurate surgical registration method. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the process of the present invention.

[0026] Figure 2 Schematic diagram of the CT processing process in an embodiment of the present invention.

[0027] Figure 3 It is a schematic diagram of drawing a breathing error curve in an embodiment of the present invention.

[0028] Figure 4 is the registration result of the point set in this embodiment. DETAILED DESCRIPTION

[0029] In order to make the technical problems, technical solutions and beneficial effects to be solved by the embodiments of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be emphasized that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0030] Embodiment 1:

[0031] See also Figure 1 , a real-time surgical registration method for approximating a respiratory state, comprising the following steps:

[0032] S1: Import the CT sequence acquired during the operation, process the CT image, extract the skin contour of the patient's upper surface and reconstruct it into a three-dimensional point set H;

[0033] S2: Let the patient breathe normally, use the depth camera to collect the three-dimensional coordinate point set of depth data in real time, and segment the three-dimensional point set V of the patient's skin surface from the three-dimensional point set. The point set V will be updated every time the data is collected;

[0034] S3: align the source point set H to the target point set V in real time and feedback the error of each alignment;

[0035] S4: Based on the error data, the software draws the error curve of registration error and progress time in real time and displays it visually for observation;

[0036] S5: Continue to execute steps S2, S3, and S4 until an error curve of at least one respiratory cycle (expiratory and inspiratory cycle) is collected and drawn;

[0037] S6: Observe the trend of the registration curve. When the error is close to the minimum value, control the patient to slow down breathing, and control the patient to hold his breath near the minimum value of the registration error.

[0038] S7: Apply the spatial transformation relationship obtained by the point set registration at this time to the source point set H to achieve surgical registration.

[0039] This example method can achieve contactless, high-precision breathing approximation and surgery registration.

[0040] Embodiment 2:

[0041] In this embodiment, the “real time” means “greater than or equal to 5 times per second”.

[0042] In this embodiment, in step S1, the CT image is subjected to windowing processing, binarization processing and contour extraction in sequence to obtain a two-dimensional skin contour, and then the upper quarter of the contour is intercepted as the upper surface skin contour.

[0043] In this embodiment, in step S2, depth data is collected in real time, a height allowable range is set, and three-dimensional point sets of irrelevant objects are filtered out according to the height of the measured object from the camera, thereby segmenting the three-dimensional point set of the skin area.

[0044] In this embodiment, in step S3, the point set H and the point set V are registered in real time using a point cloud registration method, and the similarity between the skin surfaces represented by the point set H and the point set V is evaluated by the output registration error.

[0045] In this embodiment, in step S4, a curve graph of the registration error and the processing time is drawn using computer software with the output error as the vertical axis and the processing time as the horizontal axis, and is displayed on a corresponding screen to facilitate observation of the error trend.

[0046] In this embodiment, in step S6, the trend of the error curve is always paid attention to, and the patient is controlled to hold his breath near the minimum value. The minimum error value represents that the breathing state at this moment is closest to the breathing state when the CT scan is taken, and the physiological and anatomical structure is closest.

[0047] This embodiment is a real-time surgical registration method for approximating the respiratory state. This method can effectively approximate the respiratory state during preoperative CT scans without the need to stick or tie detection objects on the patient's body surface, and can provide contactless and more accurate surgical registration.

[0048] Embodiment three:

[0049] In this embodiment, see Figures 1 to 4 , a real-time surgical registration method for approximating a respiratory state, specifically comprising the following steps:

[0050] S1: Import CT and extract skin contour. Figure 2 As shown, the CT sequence of the lesion site is imported, and the CT image is successively subjected to windowing processing, binarization processing and contour extraction to obtain a two-dimensional skin contour, and then the upper quarter of the contour is intercepted as the upper surface skin contour.

[0051] S2: Skin surface point set collection and segmentation. The depth camera collects the depth data from the lesion to the camera, thereby obtaining the 3D coordinate point set of the lesion, setting the height allowable range, and filtering out the 3D point set of non-related objects according to the height of the measured object from the camera, thereby segmenting the 3D point set of the skin area from the 3D point set. The frequency of surface point set collection and segmentation is greater than or equal to 5Hz.

[0052] S3: Real-time registration and error output. The downsampling algorithm is used to reduce the data size of the point set H and the point set V, and the two point sets are registered using the point cloud registration method based on local features. The registration process is as follows: the local features of the point set are described using a local descriptor, the local features are matched using a feature matching method and mismatches are eliminated, and the coordinate transformation between the matched point sets is solved using the SVD algorithm and the registration error is solved. The registration error output by this process is used to evaluate the similarity between the skin surfaces represented by the point set H and the point set V. The frequency of real-time registration and error output is greater than or equal to 5Hz.

[0053] S4: Draw the error curve. Figure 3 As shown, according to the error data output in step S3, with the output error as the vertical axis and the alignment time as the horizontal axis, a curve chart of the alignment error and the alignment time is drawn using computer software and displayed on the corresponding screen to facilitate observation of the error trend.

[0054] S5: Continue to execute steps S2, S3, and S4 until an error curve of at least one respiratory cycle (expiratory and inspiratory cycle) is collected and drawn.

[0055] S6: Control breathing according to the error curve. Observe the trend of the registration curve. When the error is close to the minimum value, control the patient to slow down breathing, and control the patient to hold his breath near the minimum value of the registration error. The minimum error value means that the breathing state at this moment is closest to the breathing state when the CT scan was taken, and the physiological and anatomical structure is closest.

[0056] S7: Complete the surgery registration. Figure 4As shown, the spatial transformation relationship obtained by the point set registration at this time is applied to the source point set H to achieve surgical registration.

[0057] In summary, the above-mentioned embodiment is a real-time surgical registration method for approximating the respiratory state. First, the CT sequence collected during the operation is imported, and the skin contour of the patient's upper surface is extracted and reconstructed into a three-dimensional point set H. A depth camera is used to collect and segment the three-dimensional point set V on the patient's skin surface in real time. The source point set H is aligned to the target point set V in real time and the error of each alignment is fed back. According to the error data, an error curve of the alignment error and the progress time is drawn in real time. Continue to execute until the error curve of at least one respiratory cycle is collected and drawn. Observe the direction of the alignment curve. When the error is close to the minimum value, control the patient to slow down his breathing, and control the patient to hold his breath near the minimum value of the alignment error. Apply the spatial transformation relationship obtained by the point set alignment at this time to the source point set H to achieve surgical registration. The above-mentioned embodiment of the present invention effectively approximates the respiratory state when the CT scan is taken before the operation, without the need to stick or tie the detection object on the patient's body surface. It is a contactless and more accurate surgical registration method.

[0058] The above contents are further detailed descriptions of the present invention in combination with specific / preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, they can also make several substitutions or modifications to these described embodiments without departing from the concept of the present invention, and these substitutions or modifications should be regarded as belonging to the protection scope of the present invention.

Claims

1. A real-time surgical registration method for approximating a respiratory state, characterized in that: The following steps are involved: S1: Import the CT sequence acquired during the operation, process the CT image, extract the skin contour of the patient's upper surface and reconstruct it into a three-dimensional point set H; S2: Let the patient breathe normally, use the depth camera to collect the three-dimensional coordinate point set of depth data in real time, and segment the three-dimensional point set V of the patient's skin surface from the three-dimensional point set. The point set V will be updated every time the data is collected; S3: align the source point set H to the target point set V in real time and feedback the error of each alignment; S4: Based on the error data, the software draws the error curve of registration error and progress time in real time and displays it visually for observation; S5: Continuously perform steps S2, S3, and S4 until at least one respiratory cycle, i.e., an error curve of an exhalation cycle and an inhalation cycle, is collected and drawn; S6: Observe the trend of the registration curve. When the error is close to the minimum value, control the patient to slow down breathing, and control the patient to hold his breath near the minimum value of the registration error. S7: Apply the spatial transformation relationship obtained by the point set registration at this time to the source point set H to achieve surgical registration.

2. The real-time surgery registration method for approximating the respiratory state according to claim 1, characterized in that: "Real time" means "greater than or equal to 5 times per second".

3. The real-time surgery registration method for approximating the respiratory state according to claim 1, characterized in that: In step S1, the CT image is subjected to windowing processing, binarization processing and contour extraction in succession to obtain a two-dimensional skin contour, and then the upper quarter of the contour is intercepted as the upper surface skin contour.

4. The real-time surgery registration method for approximating the respiratory state according to claim 1, characterized in that: In step S2, depth data is collected in real time, a height allowable range is set, and the three-dimensional point set of non-related objects is filtered out according to the height of the measured object from the camera, thereby segmenting the three-dimensional point set of the skin area.

5. The real-time surgery registration method for approximating the respiratory state according to claim 1, characterized in that: In step S3, the point set H is registered with the point set V in real time using a point cloud registration method, and the similarity between the skin surfaces represented by the point set H and the point set V is evaluated by the output registration error.

6. The real-time surgery registration method for approximating the respiratory state according to claim 1, characterized in that: In step S4, a graph of the registration error and the processing time is drawn using computer software, with the output error as the vertical axis and the processing time as the horizontal axis. The graph is displayed on a corresponding screen to facilitate observation of the error trend.

7. The real-time surgery registration method for approximating the respiratory state according to claim 1, characterized in that: In step S6, the error curve is always followed, and the patient is controlled to hold his breath near the minimum value. The minimum error value indicates that the breathing state at this moment is closest to the breathing state when the CT scan is taken, and the physiological and anatomical structure is closest.

Images