A method for fusing intracavitary ultrasound images with CT images

Abstract

This invention discloses a method for fusing intracavitary ultrasound images and CT images based on electromagnetic positioning, comprising the following steps: scanning CT images of the lesion site and extracting a set of CT marker points S1; installing an electromagnetic positioning sensor on the intracavitary ultrasound probe; scanning multiple intracavitary ultrasound images of the lesion site and extracting a set of electromagnetic marker points S2; registering the CT image coordinate system with the electromagnetic positioning system coordinate system based on the CT marker point set S1 and the electromagnetic marker point set S2; and achieving real-time fusion of the CT image and the intracavitary ultrasound image based on the registration result. Using this invention, real-time fusion of intracavitary ultrasound images and CT images is achieved based on an electromagnetic positioning system, fully utilizing the advantages of multiple medical images to provide doctors with rich imaging information about the patient's lesion.

Description

Technical Field

[0001] This invention relates to the field of medical imaging, and more particularly to a method for fusing intracavitary ultrasound images with CT images. Background Technology

[0002] Image-guided minimally invasive interventional surgery utilizes medical images to guide surgical instruments, such as puncture needles, directly into the tumor to target specific lesions, playing an increasingly important role in many fields, including liver tumor treatment. However, as a relatively new minimally invasive surgical technique, image-guided minimally invasive interventional surgery also has some limitations and shortcomings.

[0003] First, from the perspective of the premise of the surgery, minimally invasive interventional treatment is performed under the guidance of medical imaging. Currently, most interventional treatments rely on single-modality imaging such as CT, MRI, or ultrasound. CT images have a significant advantage in displaying the contours of bones, lungs, and organs; however, CT imaging is slow and lacks real-time capability during the procedure. MRI images have good imaging effects on soft tissues, but the image slices are thick, the resolution is low, and the imaging speed is even slower during the procedure. Two-dimensional ultrasound images have good real-time capability, are easy to operate, and have no ionizing radiation; however, ultrasound images are easily interfered with by bones and gases, resulting in poor overall observation and an inability to clearly show the spatial relationship between the lesion and surrounding tissues and organs. Each single-modality medical imaging has its advantages and disadvantages, and its individual application cannot currently meet the requirements of precise interventional treatment.

[0004] Secondly, both the puncture path planning and the actual operation require experienced physicians. Young doctors without relevant experience need a long training period to grow into skilled and experienced physicians, and they also need a wealth of clinical surgical cases and hands-on experience. However, even excellent physicians cannot completely avoid hand tremors during interventional punctures. Therefore, under real-time monitoring with image monitoring equipment, the physician must repeatedly advance and check the instrument tip to the target site. This surgical procedure is not only time-consuming but also causes unnecessary damage to the healthy tissue surrounding the puncture target, significantly impacting the effectiveness of minimally invasive interventional treatment.

[0005] Finally, even experienced clinicians find it difficult to accurately deliver interventional surgical instruments directly to target sites such as tumor centers according to a predetermined planned path. With the current single-modal medical image guidance technology, doctors have difficulty forming a clear understanding of the spatial relative position between the surgical instruments and the puncture target.

[0006] The above-mentioned issues have greatly affected the treatment efficacy and applicability of image-guided minimally invasive interventional surgery.

[0007] In recent years, multimodal image fusion technology has shown great advantages in the treatment of patients' diseases compared with single-modal images. However, there is currently little research on the specific implementation methods of real-time ultrasound image and CT image fusion, both domestically and internationally. In addition, there are very few research reports on the emerging intracavitary ultrasound image and CT image fusion technology. Summary of the Invention

[0008] Therefore, in order to fully utilize the complementarity and redundancy of multimodal medical images in describing lesions, and to enable doctors to better understand the patient's lesion status during image-guided minimally invasive interventional surgery, thereby improving the surgical effect and quality of interventional treatment, this application proposes a method for fusing intracavitary ultrasound images and CT images based on electromagnetic positioning.

[0009] A method for fusing intracavitary ultrasound images and CT images includes the following steps: scanning CT images of the lesion site and extracting a set of CT marker points S1; installing an electromagnetic positioning sensor on the intracavitary ultrasound probe; scanning multiple intracavitary ultrasound images of the lesion site and extracting a set of electromagnetic marker points S2; registering the CT image coordinate system with the electromagnetic positioning system coordinate system based on the CT marker point set S1 and the electromagnetic marker point set S2; and achieving real-time fusion of CT images and intracavitary ultrasound images based on the registration results.

[0010] Using this invention, real-time fusion of intracavitary ultrasound images and CT images is achieved based on an electromagnetic positioning system, making full use of the advantages of multiple medical images to provide doctors with rich imaging information of patient lesions. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of a patient's abdominal CT scan.

[0012] Figure 2 This is a schematic diagram of an intra-abdominal ultrasound probe.

[0013] Figure 3 This is a schematic diagram of an electromagnetic positioning system.

[0014] Figure 4 This is a schematic diagram of installing an electromagnetic positioning sensor on an intracavitary ultrasound probe.

[0015] Figure 5 This is a flowchart of the method for fusing intracavitary ultrasound images and CT images according to the present invention. Detailed Implementation

[0016] This application proposes a method for fusing intracavitary ultrasound images and CT images based on electromagnetic positioning. The method includes the following steps: scanning CT images of the lesion site and extracting a set of CT marker points S1; installing an electromagnetic positioning sensor on the intracavitary ultrasound probe; scanning multiple intracavitary ultrasound images of the lesion site and extracting a set of electromagnetic marker points S2; registering the CT image coordinate system with the electromagnetic positioning system coordinate system based on the CT marker point set S1 and the electromagnetic marker point set S2; and achieving real-time fusion of the CT image and the intracavitary ultrasound image based on the registration result. Thus, based on the electromagnetic positioning system, real-time fusion of intracavitary ultrasound images and CT images is achieved by installing an electromagnetic positioning sensor on the intracavitary ultrasound probe and acquiring two sets of point data.

[0017] Since the procedures and methods of minimally invasive interventional therapy for different parts of the body are largely similar, the following uses interventional therapy for liver tumors as an example to illustrate the scheme of this application.

[0018] (1) Scan the CT images of the lesion site and extract the CT marker set S1.

[0019] For patients with liver tumors requiring minimally invasive interventional treatment, an abdominal plain CT scan or enhanced CT scan is performed the day before or the day of surgery to obtain abdominal CT images, such as... Figure 1 As shown. The coordinate system in which the CT image is located is called the CT image coordinate system, denoted as Lct. The three-dimensional position information of each pixel in the image in the CT image coordinate system is known.

[0020] For affected areas in CT images—such as the liver—traditional algorithms like pattern recognition or intelligent AI algorithms like deep learning can be used to extract anatomical landmarks, such as vascular bifurcation points. This application does not limit the specific algorithms used. The set of extracted anatomical landmarks is called the CT landmark set S1. This set records the position information of each key anatomical landmark in the CT image coordinate system, i.e.:

[0021] S1={s1 i |s1 i =(x fi ,y fi ,z fi ,1) T}, i = 1, 2, 3, ...

[0022] (2) Install an electromagnetic positioning sensor on the intracavitary ultrasound probe.

[0023] Commonly used intraperitoneal ultrasound probes such as Figure 2 As shown in the image, the left side of the image is the imaging end of the intracavitary ultrasound probe, which can be used for intracavitary ultrasound imaging examinations. The right side of the image is the operating end of the intracavitary ultrasound probe, which is used by doctors to hold it.

[0024] The electromagnetic positioning system includes components such as a magnetic field generator 11 and an electromagnetic positioning sensor 12, etc. Figure 3 As shown, after all components are properly connected, the magnetic field generator 11 can generate a stable electromagnetic field within a certain range. The real-time three-dimensional positional relationship of each electromagnetic positioning sensor 12 located in this electromagnetic field relative to the magnetic field generator 11 is known, and the spatial transformation relationship between the two can be represented by a fourth-order matrix. It means, that is

[0025]

[0026] Fourth-order matrix The 3D matrix (denoted by the letter 'r') consisting of 9 elements in the upper left corner is called the rotation matrix, representing the attitude transformation between the two coordinate systems (the sensor coordinate system and the electromagnetic positioning system coordinate system). The 3D vector (denoted by the letter 't') consisting of 3 elements in the upper right corner is called the translation vector, representing the translation transformation between the two coordinate systems. According to the principle of spatial positioning, the coordinate values ​​of each 3D point located in the electromagnetic positioning sensor coordinate system are multiplied by the matrix 'r'. The coordinates of the spatial point in the electromagnetic positioning system coordinate system can then be obtained.

[0027] The electromagnetic positioning sensor 12 is mounted on the intracavitary ultrasound probe 10 using a mechanical bracket or direct adhesive bonding, ensuring that the installation position does not interfere with the surgeon's normal surgical procedures. Figure 4 As shown in the figure, the electromagnetic positioning sensor 12, mounted on the intracavitary ultrasound probe 10, is schematically illustrated by a box. The electromagnetic positioning sensor has a sensor coordinate system Ls, and the coordinate system of the ultrasound image scanned by the intracavitary ultrasound probe is called the ultrasound image coordinate system L. 超 Since the electromagnetic positioning sensor 12 is fixedly connected to the intracavitary ultrasound probe 10, and the two do not move relative to each other during use, the relative spatial relationship between the ultrasound image currently scanned by the intracavitary ultrasound probe 10 and the electromagnetic positioning sensor can be calculated through the mechanical dimensions of the intracavitary ultrasound probe and the mechanical support used to install the electromagnetic positioning sensor, i.e., the ultrasound image coordinate system L. 超 Spatial transformation relationship with sensor coordinate system Ls It is known.

[0028] (3) Scan multiple intracavitary ultrasound images of the lesion site and extract the electromagnetic marker set S2.

[0029] An intracavitary ultrasound probe equipped with an electromagnetic positioning sensor is used to scan the patient's liver in real time, acquiring a set of two-dimensional ultrasound images of the liver. Similar to the processing of CT images, traditional algorithms such as pattern recognition algorithms or intelligent AI algorithms such as deep learning can be used to extract multiple anatomical landmarks of the liver from multiple two-dimensional ultrasound images, such as vascular bifurcation points. This application does not limit the specific algorithms used.

[0030] The set of extracted anatomical markers is called the ultrasound image marker set S0. This set records the position information of each key anatomical marker in the CT marker set S1 within its respective ultrasound image coordinate system in multiple two-dimensional ultrasound images, i.e.:

[0031] S0={s0 i |s0 i =(x ui ,y ui ,z ui ,1) T}, i = 1, 2, 3, ...

[0032] Based on the spatial transformation relationship between the ultrasound image coordinate system and the sensor coordinate system Spatial transformation relationship between the sensor coordinate system and the electromagnetic positioning system coordinate system The coordinates of the ultrasound image marker set S0 in the electromagnetic positioning system coordinate system, i.e., the electromagnetic marker set S2, can be obtained by the following method:

[0033]

[0034] (4) Based on the two point sets S1 and S2, complete the registration between the CT image coordinate system and the electromagnetic positioning system coordinate system.

[0035] Based on the two sets of point cloud coordinate data S1 and S2, traditional algorithms such as the ICP algorithm or Landmark algorithm, or intelligent AI algorithms such as deep learning, can be used to determine the spatial transformation relationship between the two sets of point cloud data. This spatial transformation relationship is the spatial transformation relationship between the coordinate system of the electromagnetic positioning system and the coordinate system of the CT image, thereby completing the registration between the two. Since the magnetic field generator of the electromagnetic positioning system is placed in a fixed position in the operating room during surgery, a fourth-order matrix... The value remains constant.

[0036] (5) Real-time fusion of CT images and intracavitary ultrasound images based on registration results

[0037] After completing the registration between the CT image coordinate system and the electromagnetic positioning system coordinate system, for any real-time intracavitary ultrasound image during surgery, the spatial transformation relationship between the ultrasound image coordinate system and the CT image coordinate system is determined. The calculation method is as follows:

[0038]

[0039] For all pixels in the current real-time intracavitary two-dimensional ultrasound image, multiply by the fourth-order matrix on the left. The position information of each pixel in the CT image coordinate system can be obtained, thereby finding the CT image section corresponding to the current intracavitary ultrasound image, and thus realizing the real-time fusion display of CT image and intracavitary ultrasound image.

[0040] Thus, by utilizing this invention, real-time fusion of intracavitary ultrasound images and CT images can be achieved based on an electromagnetic positioning system, making full use of the advantages of multiple medical images to provide doctors with rich imaging information on patient lesions.

[0041] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and do not constitute a limitation thereof. Those skilled in the art should understand that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for fusing intracavitary ultrasound images and CT images, characterized in that, Includes the following steps: Scan the CT images of the lesion site and extract the CT marker set S1; An electromagnetic positioning sensor is installed on the intracavitary ultrasound probe; Multiple intracavitary ultrasound images of the lesion site were scanned, and the electromagnetic marker point set S2 was extracted. Based on the CT marker set S1 and the electromagnetic marker set S2, the registration of the CT image coordinate system and the electromagnetic positioning system coordinate system is completed. Real-time fusion of CT images and intracavitary ultrasound images is achieved based on the registration results.

2. The method for fusing intracavitary ultrasound images and CT images according to claim 1, characterized in that, The electromagnetic positioning system includes a magnetic field generator and the electromagnetic positioning sensor. The spatial transformation relationship between the real-time three-dimensional position of the electromagnetic positioning sensor relative to the magnetic field generator is expressed by a fourth-order matrix. Indicates, that is Fourth-order matrix The 3rd-order matrix represented by the letter 'r', consisting of 9 elements in the upper left corner, is called the rotation matrix. It represents the attitude transformation between the sensor coordinate system and the electromagnetic positioning system coordinate system. The 3-dimensional vector represented by the letter 't', consisting of 3 elements in the upper right corner, is called the translation vector. It represents the translation transformation between the sensor coordinate system and the electromagnetic positioning system coordinate system.

3. The method for fusing intracavitary ultrasound images and CT images according to claim 1, characterized in that, Based on the spatial transformation relationship between the ultrasound image coordinate system and the sensor coordinate system Spatial transformation relationship between the sensor coordinate system and the electromagnetic positioning system coordinate system The coordinates of the ultrasound image marker set S0 in the electromagnetic positioning system coordinate system, i.e., the electromagnetic marker set S2, can be obtained by the following method: Among them, the ultrasound image marker set S0 records the position information of each key anatomical marker in the CT marker set S1 in the respective ultrasound image coordinate system in multiple two-dimensional ultrasound images.

4. The method for fusing intracavitary ultrasound images and CT images according to claim 1, characterized in that, Based on the CT marker set S1 and the electromagnetic marker set S2, determine the spatial transformation relationship between the CT marker set S1 and the electromagnetic marker set S2. This spatial transformation relationship is the spatial transformation relationship between the electromagnetic positioning system coordinate system and the CT image coordinate system, thereby completing the registration between the electromagnetic positioning system coordinate system and the CT image coordinate system.

5. The method for fusing intracavitary ultrasound images and CT images according to claim 1, characterized in that, In the step of real-time fusion of CT images and intracavitary ultrasound images based on registration results, after the registration of the CT image coordinate system and the electromagnetic positioning system coordinate system is completed, the spatial transformation relationship between the ultrasound image coordinate system and the CT image coordinate system is determined for a single real-time intracavitary ultrasound image during surgery. The calculation method is as follows:

Images