CT and ultrasound cross-modal image registration method and device
By acquiring the real-time position information of the ultrasound probe and image preprocessing, combined with the contour registration method, the precise registration of CT and ultrasound images is achieved, solving the problem of difficult image positioning in surgical navigation, and improving the accuracy and safety of the operation .
Patent Information
- Application Number
- CN202411214115.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-31
- Publication Date
- 2024-12-06
AI Technical Summary
The prior art is difficult to effectively combine the advantages of CT and ultrasound images to achieve cross-modal image registration and overcome their respective limitations, especially in the accurate identification and judgment of lesions in surgical navigation.
By obtaining the real-time position and orientation information of the ultrasound probe in three-dimensional space, real-time ultrasound image preprocessing and extracting the two-dimensional spinal boundary. Combining the spinal boundary in the preoperative three-dimensional CT image, three-dimensional ultrasound and CT are achieved using contour registration method Accurate registration of images and image fusion processing.
The precise registration of two-dimensional ultrasound images and three-dimensional CT images is achieved, which improves the image positioning accuracy and safety during surgery, reduces the risk of damage to important organs, and enhances the accuracy and efficiency of the surgery.
Smart Images

Figure CN119074222A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical image processing, and in particular to a method and device for cross-modality registration of CT and ultrasound images. Background Art
[0002] With the continuous development of medical imaging technology, image-guided surgical navigation has become an indispensable and important tool in modern surgery. This technology relies on advanced medical imaging computing and computer image processing technology. It helps clinicians to plan surgical paths and simulate surgeries by accurately segmenting, identifying, reconstructing and visualizing medical images obtained before surgery. This process not only improves the safety and effectiveness of surgery, but also improves the accuracy of surgery, especially in the positioning and treatment of complex lesions.
[0003] During surgery, it is crucial to accurately align the medical images acquired before surgery with the patient's real physical space. This alignment process enables the preoperative medical images, the actual position of the patient during surgery, and the surgical instruments to be effectively integrated in the same three-dimensional coordinate system. Through real-time positioning equipment, the relative position of the surgical instruments in the patient's body can be accurately determined, and clinicians can observe the relationship between the surgical instruments and tissues and lesions in real time. This real-time fusion display not only enables doctors to monitor the posture and related parameters (such as angle, depth, etc.) of surgical instruments from multiple angles, but also minimizes damage to the patient's important organs (such as large blood vessels), thereby reaching the lesion in the shortest time and implementing precise surgical treatment.
[0004] Among many medical imaging technologies, ultrasound imaging has become a commonly used intraoperative imaging device in clinical practice due to its real-time, non-invasive and non-ionizing radiation characteristics. With the rapid development of ultrasound-guided surgical navigation technology, the role of ultrasound images in surgery has become increasingly important. However, due to the inherent limitations of ultrasound imaging mechanisms, ultrasound two-dimensional images often face problems such as low resolution, narrow imaging field of view, and unique speckle noise, which makes it difficult to accurately identify and determine the location of lesions in the human body through ultrasound images. Compared with ultrasound two-dimensional images, electronic computed tomography (CT) images have higher imaging resolution and wider imaging field of view, and can accurately identify and segment human tissues and most lesions. However, CT imaging causes radiation damage to patients during the acquisition process, and it is impossible to perform real-time imaging during surgery, so it cannot completely replace the role of ultrasound images in surgical navigation. Therefore, how to effectively combine the advantages of CT and ultrasound images, overcome their respective limitations, and achieve cross-modal image registration has become an important topic in the current research of medical imaging and surgical navigation technology.
[0005] The disclosure of the above background technology content is only used to assist in understanding the concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application. In the absence of clear evidence that the above content has been disclosed on the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of the present application. Summary of the invention
[0006] The present application provides a CT and ultrasound cross-modality image registration method and device, which can achieve accurate registration of two-dimensional ultrasound images and three-dimensional CT images.
[0007] To achieve the above objectives, the present application discloses the following technical solutions:
[0008] In a first aspect, an embodiment of the present application provides a method for cross-modality image registration between CT and ultrasound, comprising the following steps:
[0009] Acquire real-time position information and real-time orientation information of the ultrasound probe in three-dimensional space;
[0010] Obtain real-time ultrasound images collected during surgery;
[0011] Preprocessing real-time ultrasound images;
[0012] A first preset algorithm is used to extract a real-time two-dimensional spine boundary from a real-time ultrasound image;
[0013] constructing a three-dimensional ultrasound image and a first three-dimensional spine boundary according to the real-time position information and real-time orientation information of the ultrasound probe, the real-time two-dimensional spine boundary and the ultrasound image;
[0014] Obtain three-dimensional CT images acquired before surgery;
[0015] extracting a second three-dimensional spine boundary from the three-dimensional CT image using a second preset algorithm;
[0016] The first three-dimensional spine boundary is registered with the second three-dimensional spine boundary by a contour registration method to obtain an aligned and registered three-dimensional CT image and a three-dimensional ultrasound image;
[0017] The three-dimensional CT image is set as the main image, and the three-dimensional ultrasound image is set as the auxiliary image. The three-dimensional CT image and the three-dimensional ultrasound image are fused to obtain a registered image.
[0018] In an embodiment of the present application, by obtaining the real-time position information and orientation information of the ultrasound probe in three-dimensional space, it is ensured that the acquisition of the ultrasound image is accurate and real-time. This is the basis of registration, which solves the problem of spatial positioning in the process of ultrasound image acquisition. The real-time ultrasound image is preprocessed and the two-dimensional spine boundary is extracted to improve the image quality and the accuracy of feature extraction. Extracting key anatomical structures (spinal boundaries) provides reliable basic data for subsequent registration. By constructing a three-dimensional ultrasound image and a first three-dimensional spine boundary, combining the preoperative three-dimensional CT image and the second three-dimensional spine boundary, and aligning them using a contour registration method, accurate registration of images of different modalities is achieved. Image fusion processing integrates the information of the two imaging modalities into a unified registration image. In this way, accurate registration of two-dimensional ultrasound images and three-dimensional CT images is achieved, which is conducive to spinal surgery planning and navigation, helping surgeons to perform surgical operations more accurately and reduce surgical risks.
[0019] In some possible implementations of the first aspect, the step of acquiring real-time position information and real-time orientation information of the ultrasound probe in three-dimensional space includes:
[0020] Receiving ultrasound probe video acquired by a binocular vision system;
[0021] Obtaining first calibration parameters of the binocular vision system;
[0022] Based on the first calibration parameter and the ultrasound probe video, the binocular vision algorithm is applied to calculate the position information and orientation information of the ultrasound probe in three-dimensional space. In this way, the binocular vision system can use the parallax principle to collect images of the ultrasound probe from different angles, and combine the calibration parameters to more accurately calculate the position and orientation information of the probe in three-dimensional space. This method significantly improves the accuracy of positioning, ensures the accurate registration of ultrasound images and CT images, and reduces the positioning errors that may occur during surgery. Secondly, the binocular vision system can collect videos in real time at a high frame rate, and output the position and orientation information of the ultrasound probe in real time through a fast calculation algorithm. This real-time performance ensures that the position information and orientation information of the ultrasound probe can be quickly updated during surgery, so that real-time registration and image fusion can be smoothly achieved, improving the efficiency and safety of surgery. In addition, the setting of the binocular vision system is relatively simple. It only needs to arrange two cameras around the ultrasound probe and calibrate them according to the calibration parameters, without complex hardware changes. This not only reduces the complexity and cost of equipment configuration, but also improves the applicability and maintainability of the system.
[0023] In some possible implementations of the first aspect, an optical positioning ball is installed on the ultrasound probe; and the step of applying a binocular vision algorithm to calculate the position information and orientation information of the ultrasound probe in three-dimensional space based on the first calibration parameter and the ultrasound probe video includes:
[0024] Obtaining a second calibration parameter of the optical positioning sphere;
[0025] Based on the second calibration parameter, the first calibration parameter and the ultrasound probe video, the binocular vision algorithm is applied to calculate the position information and orientation information of the ultrasound probe in three-dimensional space. Among them, the optical positioning ball can provide obvious spatial landmarks, and combined with the calibration parameters, the position information and orientation information of the probe can be calculated more accurately.
[0026] In some possible implementations of the first aspect, the number of optical positioning balls is at least four, so that more reference points can be provided, calculation errors can be effectively reduced, positioning accuracy can be improved, and spatial positioning can be made more accurate and stable.
[0027] In some possible implementations of the first aspect, the step of registering the first three-dimensional spine boundary with the second three-dimensional spine boundary by a contour registration method includes:
[0028] Based on the anatomical structure feature points, the first three-dimensional spine boundary is preliminarily aligned with the second three-dimensional spine boundary;
[0029] The first three-dimensional spine boundary and the second three-dimensional spine boundary after preliminary alignment are finely registered by cross-correlation or point cloud registration method. Among them, by using anatomical fixed points as references for preliminary alignment, it is ensured that the starting point of the registration is accurate, and the cross-correlation or point cloud registration method is used to further optimize the registration results and reduce the registration error, thereby significantly improving the accuracy of the registration. In addition, the use of the two steps of preliminary alignment and fine registration can also increase the robustness of the registration process. Even in complex or changing surgical environments, the stability and reliability of the registration can be guaranteed.
[0030] In some possible implementations of the first aspect, the first preset algorithm is a boundary operator convolution algorithm or a deep learning positioning algorithm.
[0031] In a second aspect, an embodiment of the present application provides a CT and ultrasound cross-modality image registration device, comprising:
[0032] A first acquisition module is used to acquire real-time position information and real-time orientation information of the ultrasound probe in three-dimensional space;
[0033] A second acquisition module is used to acquire real-time ultrasound images collected during surgery;
[0034] A preprocessing module, used for preprocessing real-time ultrasound images;
[0035] A first extraction module, configured to extract a real-time two-dimensional spine boundary from a real-time ultrasound image using a first preset algorithm;
[0036] A first construction module is used to construct a three-dimensional ultrasound image and a first three-dimensional spine boundary according to the real-time position information and real-time orientation information of the ultrasound probe, the real-time two-dimensional spine boundary and the ultrasound image;
[0037] A third acquisition module is used to acquire the three-dimensional CT image collected before surgery;
[0038] A second extraction module is used to extract a second three-dimensional spine boundary from the three-dimensional CT image using a second preset algorithm;
[0039] A first registration module is used to register the first three-dimensional spine boundary with the second three-dimensional spine boundary by a contour registration method to obtain an aligned and registered three-dimensional CT image and a three-dimensional ultrasound image;
[0040] The first fusion module is used to set the 3D CT image as the main image and the 3D ultrasound image as the auxiliary image, and perform image fusion processing on the 3D CT image and the 3D ultrasound image to obtain a registered image.
[0041] In some possible implementations of the second aspect, the first acquisition module is specifically used to: receive an ultrasound probe video captured by a binocular vision system; obtain a first calibration parameter of the binocular vision system; and based on the first calibration parameter and the ultrasound probe video, apply a binocular vision algorithm to calculate the position information and orientation information of the ultrasound probe in three-dimensional space.
[0042] In some possible implementations of the second aspect, an optical positioning ball is installed on the ultrasound probe; the first acquisition module is specifically used to: obtain a second calibration parameter of the optical positioning ball; based on the second calibration parameter, the first calibration parameter and the ultrasound probe video, a binocular vision algorithm is used to calculate the position information and orientation information of the ultrasound probe in three-dimensional space.
[0043] In some possible implementations of the second aspect, the number of optical positioning balls is at least four.
[0044] In some possible implementations of the second aspect, the first registration module is specifically used to: preliminarily align the first three-dimensional spine boundary with the second three-dimensional spine boundary based on anatomical structure feature points; and perform fine alignment of the preliminarily aligned first three-dimensional spine boundary and the second three-dimensional spine boundary through cross-correlation or point cloud registration methods.
[0045] In some possible implementations of the second aspect, the first preset algorithm is a boundary operator convolution algorithm or a deep learning positioning algorithm.
[0046] In a third aspect, an embodiment of the present application provides an electronic device, comprising one or more processors; a storage device on which one or more programs are stored; when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any technical solution of the first aspect.
[0047] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in any technical solution of the first aspect is implemented.
[0048] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method described in any technical solution of the first aspect.
[0049] Among them, the technical effects brought about by any design method in the second to fifth aspects can refer to the technical effects brought about by different design methods in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.
[0051] Figure 1 A schematic diagram of a flow chart of a CT and ultrasound cross-modality image registration method provided in some embodiments of the present application;
[0052] Figure 2 Based on Figure 1 A schematic diagram of ultrasound and CT image registration of a CT and ultrasound cross-modality image registration method is shown;
[0053] Figure 3 A schematic diagram of the structure of a CT and ultrasound cross-modality image registration device provided in some embodiments of the present application;
[0054] Figure 4 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present application. DETAILED DESCRIPTION
[0055] Specific embodiments of the present invention will now be mentioned in detail. Although the present invention is described in conjunction with these specific embodiments, it should be appreciated that it is not intended to limit the present invention to these specific embodiments. On the contrary, these embodiments are intended to cover substitutions, changes or equivalent embodiments that may be included in the spirit and scope of the invention defined by the claims. In the following description, a large number of specific details are set forth in order to provide a comprehensive understanding of the present invention. The present invention may be implemented without some or all of these specific details.
[0056] When used in conjunction with "including," "methods comprising," or similar language in this specification and the appended claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0057] With the continuous development of medical imaging technology, image-guided surgical navigation has become an indispensable and important tool in modern surgery. This technology relies on advanced medical imaging computing and computer image processing technology. It helps clinicians to plan surgical paths and simulate surgeries by accurately segmenting, identifying, reconstructing and visualizing medical images obtained before surgery. This process not only improves the safety and effectiveness of surgery, but also improves the accuracy of surgery, especially in the positioning and treatment of complex lesions.
[0058] During surgery, it is crucial to accurately align the medical images acquired before surgery with the patient's real physical space. This alignment process enables the preoperative medical images, the actual position of the patient during surgery, and the surgical instruments to be effectively integrated in the same three-dimensional coordinate system. Through real-time positioning equipment, the relative position of the surgical instruments in the patient's body can be accurately determined, and clinicians can observe the relationship between the surgical instruments and tissues and lesions in real time. This real-time fusion display not only enables doctors to monitor the posture and related parameters (such as angle, depth, etc.) of surgical instruments from multiple angles, but also minimizes damage to the patient's important organs (such as large blood vessels), thereby reaching the lesion in the shortest time and implementing precise surgical treatment.
[0059] Among many medical imaging technologies, ultrasound imaging has become a commonly used intraoperative imaging device in clinical practice due to its real-time, non-invasive and non-ionizing radiation characteristics. With the rapid development of ultrasound-guided surgical navigation technology, ultrasound images play an increasingly important role in surgery. However, due to the inherent limitations of ultrasound imaging mechanisms, ultrasound two-dimensional images often face problems such as low resolution, narrow imaging field of view, and unique speckle noise, which makes it difficult to accurately identify and determine the location of lesions in the human body through ultrasound images.
[0060] Compared with ultrasound 2D images, computed tomography (CT) images have higher imaging resolution and wider imaging field of view, and can accurately identify and segment human tissues and most lesions. However, CT imaging causes radiation damage to patients during the acquisition process, and it is impossible to perform real-time imaging during surgery, so it cannot completely replace the role of ultrasound images in surgical navigation.
[0061] Therefore, how to effectively combine the advantages of CT and ultrasound images, overcome their respective limitations, and achieve cross-modal image registration has become an important topic in current research on medical imaging and surgical navigation technology.
[0062] In response to the above technical problems, an embodiment of the present application provides a CT and ultrasound cross-modal image registration method, including the following steps: obtaining real-time position information and real-time orientation information of an ultrasound probe in three-dimensional space; obtaining a real-time ultrasound image acquired during surgery; preprocessing the real-time ultrasound image; using a first preset algorithm to extract a real-time two-dimensional spine boundary from the real-time ultrasound image; constructing a three-dimensional ultrasound image and a first three-dimensional spine boundary based on the real-time position information and real-time orientation information of the ultrasound probe, the real-time two-dimensional spine boundary and the ultrasound image; obtaining a three-dimensional CT image acquired before surgery; using a second preset algorithm to extract a second three-dimensional spine boundary from the three-dimensional CT image; registering the first three-dimensional spine boundary with the second three-dimensional spine boundary through a contour registration method to obtain an aligned and registered three-dimensional CT image and a three-dimensional ultrasound image; setting the three-dimensional CT image as the main image and the three-dimensional ultrasound image as the auxiliary image, performing image fusion processing on the three-dimensional CT image and the three-dimensional ultrasound image to obtain a registered image.
[0063] Among them, by obtaining the real-time position information and orientation information of the ultrasound probe in three-dimensional space, it is ensured that the acquisition of ultrasound images is accurate and real-time. This is the basis of registration and solves the problem of spatial positioning in the process of ultrasound image acquisition. The real-time ultrasound image is preprocessed and the two-dimensional spine boundary is extracted to improve the image quality and the accuracy of feature extraction. The extraction of key anatomical structures (spinal boundaries) provides reliable basic data for subsequent registration. By constructing a three-dimensional ultrasound image and the first three-dimensional spine boundary, combining the preoperative three-dimensional CT image and the second three-dimensional spine boundary, and aligning them using the contour registration method, accurate registration of images of different modalities is achieved. Image fusion processing integrates the information of the two imaging modalities into a unified registration image. In this way, accurate registration of two-dimensional ultrasound images and three-dimensional CT images is achieved, which is conducive to spinal surgery planning and navigation, helping surgeons to perform surgical operations more accurately and reduce surgical risks.
[0064] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application will be specifically described below in conjunction with the drawings in the specification.
[0065] See also Figure 1 and Figure 2 The present application embodiment provides a method for cross-modality image registration between CT and ultrasound, comprising the following steps:
[0066] S101: Acquire real-time position information and real-time orientation information of the ultrasound probe in three-dimensional space;
[0067] Specifically, in some embodiments, the execution subject (for example, a computer device) of the CT and ultrasound cross-modality image registration method can be connected to a binocular vision system (i.e., two cameras) by wired connection or wireless connection to obtain real-time position information and real-time orientation information of the ultrasound probe in three-dimensional space. The specific steps are as follows:
[0068] The first step is to receive the ultrasound probe video collected by the binocular vision system;
[0069] The second step is to obtain the first calibration parameters of the binocular vision system;
[0070] The third step is to calculate the position information and orientation information of the ultrasound probe in three-dimensional space using a binocular vision algorithm based on the first calibration parameters and the ultrasound probe video.
[0071] Specifically, in some embodiments, an optical positioning ball is installed on the ultrasound probe, and the above-mentioned execution subject can calculate the position information and orientation information of the ultrasound probe in three-dimensional space by applying a binocular vision algorithm based on the first calibration parameter and the ultrasound probe video through the following steps:
[0072] The first sub-step is to obtain the second calibration parameter of the optical positioning sphere;
[0073] In the second sub-step, based on the second calibration parameter, the first calibration parameter and the ultrasound probe video, the binocular vision algorithm is used to calculate the position information and orientation information of the ultrasound probe in three-dimensional space. The optical positioning ball can provide obvious spatial landmarks, and combined with the calibration parameters, the position information and orientation information of the probe can be calculated more accurately.
[0074] In this embodiment, the binocular vision system uses the parallax principle to collect images of the ultrasound probe from different angles, and combines the calibration parameters to more accurately calculate the position and orientation information of the probe in three-dimensional space. This method significantly improves the accuracy of positioning, ensures the accurate registration of ultrasound images and CT images, and reduces the positioning errors that may occur during surgery. Secondly, the binocular vision system can collect video in real time at a high frame rate, and output the position and orientation information of the ultrasound probe in real time through a fast calculation algorithm. This real-time performance ensures that the position information and orientation information of the ultrasound probe can be quickly updated during surgery, so that real-time registration and image fusion can be smoothly achieved, improving surgical efficiency and safety. In addition, the setting of the binocular vision system is relatively simple. It only needs to arrange two cameras around the ultrasound probe and calibrate according to the calibration parameters, without the need for complex hardware changes. This not only reduces the complexity and cost of equipment configuration, but also improves the applicability and maintainability of the system.
[0075] Furthermore, the number of the optical positioning balls is at least four, so that more reference points can be provided, calculation errors can be effectively reduced, positioning accuracy can be improved, and spatial positioning can be made more accurate and stable.
[0076] S102: Acquire real-time ultrasound images collected during surgery;
[0077] Specifically, the execution subject may be connected to the ultrasonic imaging device by wired connection or wireless connection to obtain the real-time ultrasonic image; the real-time ultrasonic image corresponds to the ultrasonic probe video;
[0078] S103: Preprocessing the real-time ultrasound image;
[0079] Specifically, preprocessing includes but is not limited to filtering and clipping.
[0080] S104: Using a first preset algorithm, extracting a real-time two-dimensional spine boundary from the real-time ultrasound image.
[0081] Specifically, the above-mentioned execution subject can estimate the precise position of the spine boundary in the ultrasound image through boundary operator convolution or deep learning positioning algorithm. That is, the first preset algorithm can be boundary operator convolution or deep learning positioning algorithm. The boundary operator convolution method includes log-gabor operator or canny operator. The deep learning algorithm is mainly a supervised deep learning model based on labeled data, such as a U-net structure including an encoder-decoder structure and a segmentation algorithm based on a transformer architecture. In the deep learning method, the labeled data can be collected and recorded by an optical tracking system in a controllable environment (such as a body membrane, a large animal model). Through this step, the two-dimensional boundary of the spine can be extracted from the ultrasound image and presented in the form of pixels or coordinate positions.
[0082] S105: Construct a three-dimensional ultrasound image and a first three-dimensional spine boundary according to the real-time position information and real-time orientation information of the ultrasound probe, the real-time two-dimensional spine boundary and the ultrasound image.
[0083] Specifically, the reconstruction of a 2D ultrasound image into a 3D ultrasound image is based on the real-time position and orientation of the ultrasound probe provided by the optical tracking system. The specific solution is to embed the 2D image into the 3D space according to the position and orientation of the probe. The 3D voxels can be filled by directly assigning values to the 3D voxels occupied by pixels in the 2D image; the 3D voxels not occupied by the 2D image can be estimated by interpolation according to the nearby filled 3D voxels.
[0084] The construction of the two-dimensional spine boundary to the three-dimensional spine boundary can adopt a similar boundary voxel filling method.
[0085] S106: Obtain the three-dimensional CT image collected before the operation.
[0086] Specifically, the above-mentioned execution entity can connect to PACS (Picture Archiving and Communication System), Hospital Information System (HIS) or Radiology Information System (RIS) through wired or wireless connection to obtain the three-dimensional CT images collected before surgery.
[0087] S107: Using a second preset algorithm, extract a second three-dimensional spine boundary from the three-dimensional CT image.
[0088] Specifically, the above-mentioned execution subject can estimate the precise position of the spine bone boundary on the three-dimensional CT image through a traditional algorithm or a deep learning algorithm. That is, the second preset algorithm is a deep learning algorithm or a traditional algorithm. The traditional algorithm steps include first identifying the core area of the spine through threshold segmentation, and then extending outward from the core area, using a canny operator-like operation to identify voxels with faster grayscale value changes. The deep learning algorithm includes a U-net structure and a segmentation algorithm based on a transformer architecture. The algorithm requires annotated data of the spine boundary during the training process. The processed data obtained in this step is a three-dimensional spine boundary contour based on the CT image.
[0089] S108: The first three-dimensional spine boundary and the second three-dimensional spine boundary are registered by a contour registration method to obtain an aligned and registered three-dimensional CT image and a three-dimensional ultrasound image.
[0090] Specifically, in some embodiments, the execution subject may register the first three-dimensional spine boundary with the second three-dimensional spine boundary by a contour registration method through the following steps to obtain an aligned and registered three-dimensional CT image and a three-dimensional ultrasound image:
[0091] In the first step, based on the anatomical structure feature points, the first three-dimensional spine boundary is preliminarily aligned with the second three-dimensional spine boundary;
[0092] In the second step, the first 3D spine boundary and the second 3D spine boundary after preliminary alignment are finely aligned by cross-correlation or point cloud registration method. Specifically, the cross-correlation algorithm calculates the coordinate rotation and displacement combination with the highest correlation of the two boundary overlaps, and the point cloud registration algorithm constructs the covariance matrix by finding corresponding points, and then estimates the rotation and translation matrices to align the two point clouds.
[0093] Among them, by using anatomical fixed points as references for preliminary alignment, the starting point of the registration is ensured to be accurate, and the cross-correlation or point cloud registration method is used to further optimize the registration results and reduce the registration error, thereby significantly improving the accuracy of the registration. In addition, the use of the two steps of preliminary alignment and fine registration can also increase the robustness of the registration process. Even in complex or changing surgical environments, the stability and reliability of the registration can be guaranteed.
[0094] S109: The three-dimensional CT image is set as the main image, and the three-dimensional ultrasound image is set as the auxiliary image, and image fusion processing is performed on the three-dimensional CT image and the three-dimensional ultrasound image to obtain a registered image.
[0095] Specifically, due to the presence of artifacts in ultrasound spinal images and challenges in image interpretation, CT images are used as the main images, supplemented by ultrasound images to present the spinal structure.
[0096] In some embodiments, the execution subject can use libraries such as VTK and OpenCV to achieve image fusion processing of three-dimensional CT images and three-dimensional ultrasound images by setting different colors and transparencies to obtain a registered image.
[0097] See also Figure 3 Based on the inventive concept of a CT and ultrasound cross-modality image registration method in the aforementioned embodiment, the embodiment of the present application provides a CT and ultrasound cross-modality image registration device, comprising:
[0098] A first acquisition module 201 is used to acquire real-time position information and real-time orientation information of the ultrasound probe in three-dimensional space;
[0099] A second acquisition module 202 is used to acquire real-time ultrasound images collected during surgery;
[0100] A preprocessing module 203 is used to preprocess the real-time ultrasound image;
[0101] A first extraction module 204 is used to extract a real-time two-dimensional spine boundary from a real-time ultrasound image using a first preset algorithm;
[0102] A first construction module 205 is used to construct a three-dimensional ultrasound image and a first three-dimensional spine boundary according to the real-time position information and real-time orientation information of the ultrasound probe, the real-time two-dimensional spine boundary and the ultrasound image;
[0103] The third acquisition module 206 is used to acquire the three-dimensional CT image collected before the operation;
[0104] A second extraction module 207, configured to extract a second three-dimensional spine boundary from the three-dimensional CT image using a second preset algorithm;
[0105] A first registration module 208, configured to register the first three-dimensional spine boundary with the second three-dimensional spine boundary by a contour registration method, to obtain an aligned and registered three-dimensional CT image and a three-dimensional ultrasound image;
[0106] The first fusion module 206 is used to set the 3D CT image as the main image and the 3D ultrasound image as the auxiliary image, and perform image fusion processing on the 3D CT image and the 3D ultrasound image to obtain a registered image.
[0107] In some embodiments, the first acquisition module 201 is specifically used to: receive an ultrasound probe video captured by a binocular vision system; obtain a first calibration parameter of the binocular vision system; and based on the first calibration parameter and the ultrasound probe video, apply a binocular vision algorithm to calculate the position information and orientation information of the ultrasound probe in three-dimensional space.
[0108] In some embodiments, an optical positioning ball is installed on the ultrasound probe; the first acquisition module 201 is specifically used to: obtain the second calibration parameter of the optical positioning ball; based on the second calibration parameter, the first calibration parameter and the ultrasound probe video, apply the binocular vision algorithm to calculate the position information and orientation information of the ultrasound probe in three-dimensional space.
[0109] In some embodiments, the number of optical positioning spheres is at least four.
[0110] In some embodiments, the first registration module 208 is specifically used to: preliminarily align the first three-dimensional spine boundary with the second three-dimensional spine boundary based on anatomical structure feature points; and finely align the preliminarily aligned first three-dimensional spine boundary with the second three-dimensional spine boundary through cross-correlation or point cloud registration methods.
[0111] In some embodiments, the first preset algorithm is a boundary operator convolution algorithm or a deep learning positioning algorithm.
[0112] It is understandable that the modules recorded in the CT and ultrasound cross-modality image registration device are similar to those in the reference Figure 1Therefore, the operations, features and beneficial effects described above for the method are also applicable to the CT and ultrasound cross-modality image registration device and the modules contained therein, and will not be described in detail here.
[0113] See also Figure 4 Based on the inventive concept of a CT and ultrasound cross-modal image registration method in the aforementioned embodiment, an embodiment of the present application provides an electronic device. The electronic device may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device includes a processing device 301 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a ROM 302 (read-only memory) or a program loaded from a storage device 308 to a RAM 303 (random access memory). In RAM 303, various programs and data required for the operation of the electronic device are also stored. The processing device 301, ROM 302, and RAM 303 are connected to each other via a bus 304. The input / output interface (i.e., an I / O interface 305) is also connected to the bus 304.
[0114] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data.
[0115] In particular, according to some embodiments of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present application are executed.
[0116] It should be noted that the computer-readable medium recorded in some embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In some embodiments of the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0117] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0118] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the method steps of any of the above technical solutions may be implemented.
[0119] The computer program code for performing the operation of some embodiments of the present application can be written in one or more programming languages or a combination thereof, and the above-mentioned programming languages include object-oriented programming languages, such as Java, Smalltalk, C++, and also include conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).
[0120] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0121] The modules described in some embodiments of the present application may be implemented in software or hardware, and the modules described may also be arranged in a processor.
[0122] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0123] Some embodiments of the present application further provide a computer program product, including a computer program, which implements any of the above-mentioned CT and ultrasound cross-modality image registration methods when executed by a processor.
[0124] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the present invention can be implemented using a combination of hardware and software. When software is used, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by general or special-purpose computers.
[0125] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.
Claims
1. A method for cross-modality image registration between CT and ultrasound, characterized in that: The following steps are involved: Acquire real-time position information and real-time orientation information of the ultrasound probe in three-dimensional space; Obtain real-time ultrasound images collected during surgery; Preprocessing the real-time ultrasound image; Using a first preset algorithm, extracting a real-time two-dimensional spine boundary from the real-time ultrasound image; constructing a three-dimensional ultrasound image and a first three-dimensional spine boundary according to the real-time position information and real-time orientation information of the ultrasound probe, the real-time two-dimensional spine boundary and the ultrasound image; Obtain three-dimensional CT images acquired before surgery; Using a second preset algorithm, extracting a second three-dimensional spine boundary from the three-dimensional CT image; Registering the first three-dimensional spine boundary with the second three-dimensional spine boundary by a contour registration method to obtain the aligned and registered three-dimensional CT image and the three-dimensional ultrasound image; The three-dimensional CT image is set as the main image, and the three-dimensional ultrasound image is set as the auxiliary image. The three-dimensional CT image and the three-dimensional ultrasound image are subjected to image fusion processing to obtain a registered image.
2. The CT and ultrasound cross-modality image registration method according to claim 1, characterized in that: The steps of obtaining the real-time position information and real-time orientation information of the ultrasound probe in three-dimensional space include: Receiving ultrasound probe video acquired by a binocular vision system; Obtaining first calibration parameters of the binocular vision system; Based on the first calibration parameters and the ultrasound probe video, a binocular vision algorithm is applied to calculate the position information and orientation information of the ultrasound probe in three-dimensional space.
3. The CT and ultrasound cross-modality image registration method according to claim 2, characterized in that: The ultrasonic probe is mounted with an optical positioning ball; based on the first calibration parameter and the ultrasonic probe video, the step of applying a binocular vision algorithm to calculate the position information and orientation information of the ultrasonic probe in three-dimensional space comprises: Obtaining a second calibration parameter of the optical positioning sphere; Based on the second calibration parameters, the first calibration parameters and the ultrasound probe video, a binocular vision algorithm is applied to calculate the position information and orientation information of the ultrasound probe in three-dimensional space.
4. The CT and ultrasound cross-modality image registration method according to claim 3, characterized in that: The number of the optical positioning balls is at least four.
5. The CT and ultrasound cross-modality image registration method according to claim 1, characterized in that: The step of registering the first three-dimensional spine boundary with the second three-dimensional spine boundary by a contour registration method comprises: Based on the anatomical structure feature points, preliminarily aligning the first three-dimensional spine boundary with the second three-dimensional spine boundary; The first three-dimensional spine boundary and the second three-dimensional spine boundary after preliminary alignment are finely registered by cross-correlation or point cloud registration method.
6. The CT and ultrasound cross-modality image registration method according to any one of claims 1 to 5, characterized in that: The first preset algorithm is a boundary operator convolution algorithm or a deep learning positioning algorithm.
7. A CT and ultrasound cross-modality image registration device, characterized in that: include: A first acquisition module is used to acquire real-time position information and real-time orientation information of the ultrasound probe in three-dimensional space; A second acquisition module is used to acquire real-time ultrasound images collected during surgery; A preprocessing module, used for preprocessing the real-time ultrasound image; A first extraction module, configured to extract a real-time two-dimensional spine boundary from the real-time ultrasound image using a first preset algorithm; A first construction module is used to construct a three-dimensional ultrasound image and a first three-dimensional spine boundary according to the real-time position information and real-time orientation information of the ultrasound probe, the real-time two-dimensional spine boundary and the ultrasound image; A third acquisition module is used to acquire the three-dimensional CT image collected before surgery; A second extraction module, configured to extract a second three-dimensional spine boundary from the three-dimensional CT image using a second preset algorithm; A first registration module is used to register the first three-dimensional spine boundary with the second three-dimensional spine boundary by a contour registration method to obtain the aligned and registered three-dimensional CT image and the three-dimensional ultrasound image; The first fusion module is used to set the three-dimensional CT image as the main image and the three-dimensional ultrasound image as the auxiliary image, and perform image fusion processing on the three-dimensional CT image and the three-dimensional ultrasound image to obtain a registered image.
8. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processing device, the method according to any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processing device, the method according to any one of claims 1 to 6 is implemented.
Citation Information
Cited By
Reference frame system for orthopedic surgery and orthopedic surgery navigation method
CN119908841A
Spine displacement positioning method and system for orthopedic surgery navigation
CN120918798A
