Method and device for registering a ct image with a transesophageal echocardiogram and storage medium

Abstract

The application provides a CT image and transesophageal echocardiogram registration method and device and storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring a CT image specific section of a heart; acquiring a first real-time transesophageal echocardiogram collected by a transesophageal echocardiogram device; acquiring key points of the CT image specific section to obtain a first key point set; acquiring key points of the first real-time transesophageal echocardiogram to obtain a second key point set; performing rigid body registration on the CT image specific section and the first real-time transesophageal echocardiogram based on the first key point set and the second key point set to obtain a preliminary fusion image; and performing non-rigid body registration on the preliminary fusion image to obtain a target fusion image. The application can realize accurate and rapid registration of the CT image and the transesophageal echocardiogram.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a CT image and transesophageal echocardiogram registration method and device and storage medium. BACKGROUND

[0002] TEE (Transesophageal Echocardiography, transesophageal echocardiogram) is an echocardiogram examination technology that inserts an ultrasonic probe through the mouth and esophagus to probe the deep structure of the heart from the back to the front at a close distance. CT (Computed Tomography, computed tomography) is a visual tomographic image generated based on computed tomography technology.

[0003] TEE is a widely used real-time image tool that can dynamically provide real-time high-resolution images of the heart and adjacent anatomical structures, but TEE only provides two-dimensional information, and its field of view is limited and lacks intuitive presentation of three-dimensional anatomical relationships. CT images can provide high-precision three-dimensional anatomical information, but CT images are usually static data and lack real-time performance. Therefore, TEE and CT images can be registered to obtain a super image that has the advantages of both TEE and CT images.

[0004] Therefore, how to accurately and quickly register CT images and transesophageal echocardiograms has become a technical problem to be solved. SUMMARY

[0005] Therefore, in order to solve the above technical problems, the present application provides a CT image and transesophageal echocardiogram registration method, device and storage medium.

[0006] The present application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a CT image and transesophageal echocardiogram registration method, comprising:

[0008] Obtaining a CT image specific section of the heart;

[0009] Obtaining a first real-time transesophageal echocardiogram collected by a transesophageal echocardiogram device;

[0010] Obtaining a first key point set by performing key point acquisition on the CT image specific section;

[0011] Obtaining a second key point set by performing key point acquisition on the first real-time transesophageal echocardiogram;

[0012] perform rigid registration on the CT image specific section and the first real-time transesophageal echocardiogram based on the first key point set and the second key point set to obtain a preliminary fusion image;

[0013] perform non-rigid registration on the preliminary fusion image to obtain a target fusion image.

[0014] Optionally, before the key point acquisition on the first real-time transesophageal echocardiogram to obtain the second key point set, the CT image and transesophageal echocardiogram registration method further comprises:

[0015] perform image preprocessing on the first real-time transesophageal echocardiogram.

[0016] Optionally, the image preprocessing on the first real-time transesophageal echocardiogram specifically comprises:

[0017] perform filtering processing and image enhancement processing on the first real-time transesophageal echocardiogram.

[0018] Optionally, the CT image specific section is marked with a target region boundary.

[0019] After the non-rigid registration on the preliminary fusion image to obtain the target fusion image, the CT image and transesophageal echocardiogram registration method further comprises:

[0020] project the target region boundary onto the first real-time transesophageal echocardiogram based on the target fusion image.

[0021] Optionally, after the non-rigid registration on the preliminary fusion image to obtain the target fusion image, the CT image and transesophageal echocardiogram registration method further comprises:

[0022] acquire a second real-time transesophageal echocardiogram collected by the transesophageal echocardiogram device;

[0023] perform comparative analysis on the second real-time transesophageal echocardiogram and the target fusion image to obtain a comparative analysis result;

[0024] when the comparative analysis result shows that the second real-time transesophageal echocardiogram does not match the target fusion image, perform registration on the second real-time transesophageal echocardiogram and the CT image specific section, and replace the previous target fusion image with the newly obtained target fusion image;

[0025] project the target region boundary onto the second real-time transesophageal echocardiogram based on the newly obtained target fusion image.

[0026] Optionally, the target region boundary is projected onto the first real-time transesophageal echocardiogram, specifically comprising:

[0027] The target region boundary is projected onto the first real-time transesophageal echocardiogram through augmented reality technology or a preset navigation device.

[0028] Optionally, key points are acquired from the specific slice of the CT image to obtain a first key point set, specifically comprising:

[0029] The key points are acquired from the specific slice of the CT image using a scale-invariant feature transform algorithm to obtain the first key point set.

[0030] Optionally, key points are acquired from the first real-time transesophageal echocardiogram to obtain a second key point set, specifically comprising:

[0031] The key points are acquired from the first real-time transesophageal echocardiogram using a Harris corner detection algorithm to obtain the second key point set.

[0032] In a second aspect, the present application further provides a CT image and transesophageal echocardiogram registration device, comprising:

[0033] A first acquisition module is configured to acquire a specific slice of a CT image of a heart;

[0034] A second acquisition module is configured to acquire a first real-time transesophageal echocardiogram collected by a transesophageal echocardiogram device;

[0035] A third acquisition module is configured to acquire key points from the specific slice of the CT image to obtain a first key point set;

[0036] A fourth acquisition module is configured to acquire key points from the first real-time transesophageal echocardiogram to obtain a second key point set;

[0037] A preliminary registration module is configured to perform rigid body registration on the specific slice of the CT image and the first real-time transesophageal echocardiogram based on the first key point set and the second key point set to obtain a preliminary fusion image;

[0038] A fine registration module is configured to perform non-rigid body registration on the preliminary fusion image to obtain a target fusion image.

[0039] In a third aspect, the present application further provides a computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements each step of the CT image and transesophageal echocardiogram registration method described above.

[0040] The application adopts the above technical scheme, and through rigid body registration of a specific section of a CT image and a first real-time transesophageal echocardiogram, a preliminary fusion image is obtained, then, non-rigid body registration is performed on the preliminary fusion image, and further refined registration is realized, so that the application can realize accurate and rapid registration of the CT image and the transesophageal echocardiogram. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0042] Figure 1 is a flow diagram of a CT image and transesophageal echocardiogram registration method provided by an embodiment of the present application;

[0043] Figure 2 is a structural diagram of a CT image and transesophageal echocardiogram registration device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described in detail below. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0045] TEE is an ultrasonic echocardiogram examination technology that inserts an ultrasonic probe through the oral cavity and esophagus to probe the deep structure of the heart from the posterior to the front at a close distance. CT image is a visualized tomographic image generated based on computer tomography technology.

[0046] TEE is a widely used real-time image tool that can dynamically provide real-time high-resolution images of the heart and adjacent anatomical structures, but TEE only provides two-dimensional information, and its field of view is limited and lacks intuitive presentation of three-dimensional anatomical relationships. CT image can provide high-precision three-dimensional anatomical information, but CT image is usually static data and lacks real-time performance. Therefore, TEE and CT image can be registered to obtain a super image that has both the advantages of TEE and the advantages of CT image.

[0047] Based on this, in order to realize accurate and rapid registration of CT image and transesophageal echocardiogram, the present application provides a CT image and transesophageal echocardiogram registration method, device and storage medium.

[0048] The technical solutions of the present application will be described in detail below with reference to the drawings.

[0049] Figure 1 is a flowchart of a CT image and transesophageal echocardiogram registration method provided by an embodiment of the present application. As shown in the figure, the flowchart includes the following steps: Figure 1

[0050] Step 101: Obtain a CT image specific section of a heart.

[0051] Specifically, the CT image specific section is extracted from a pre-shot CT image of the heart. The CT image specific section can be a long-axis section or a mitral valve horizontal section, etc.

[0052] Step 102: Obtain a first real-time transesophageal echocardiogram collected by a transesophageal echocardiogram device.

[0053] Specifically, the ultrasonic irradiation position and angle of the transesophageal echocardiogram device are pre-planned.

[0054] Step 103: Obtain key points of the CT image specific section to obtain a first key point set.

[0055] Specifically, the SIFT (Scale-Invariant Feature Transform) algorithm can be used to obtain key points of the CT image specific section to obtain a first key point set. The SIFT algorithm can detect key points in the image, such as corner points and edges, and generate feature descriptors with high discrimination.

[0056] Step 104: Obtain key points of the first real-time transesophageal echocardiogram to obtain a second key point set.

[0057] Specifically, the Harris corner detection algorithm can be used to obtain key points of the first real-time transesophageal echocardiogram to obtain a second key point set. The Harris corner detection algorithm can be used to identify corner regions with obvious gray scale changes in the image, such as object corners and edge intersection points, etc.

[0058] Step 105: Based on the first key point set and the second key point set, perform rigid registration on the CT image specific section and the first real-time transesophageal echocardiogram to obtain a preliminary fusion image.

[0059] Specifically, rigid registration aims to align the space between different images through translation, rotation, etc.

[0060] ​Here, the rigid registration takes the key points in the first key point set as a reference benchmark, calculates the translation vector and rotation matrix required for the key points in the second key point set, so that the first real-time transesophageal echocardiogram is preliminarily matched with the CT image specific section in space position without changing the shape, thereby narrowing the overall spatial difference between the images and laying a foundation for subsequent more detailed non-rigid registration.

[0061] Step 106: performing non-rigid registration on the preliminary fusion image to obtain a target fusion image.

[0062] Specifically, the non-rigid registration is a key technology for further solving the local deformation problem in the image on the basis of the preliminary fusion image obtained by the rigid registration. The non-rigid registration sets a control point grid in the image, constantly adjusts the positions of the control points, and performs local stretching, compression, bending and other deformation operations on the image, so that the CT image specific section and the first real-time transesophageal echocardiogram can accurately match the myocardial fiber, valve leaflet and other details prone to deformation, thereby improving the image fusion accuracy.

[0063] The embodiment of the present application adopts the above technical solution, performs rigid registration on the CT image specific section and the first real-time transesophageal echocardiogram first to obtain a preliminary fusion image, and then performs non-rigid registration on the preliminary fusion image to realize further fine registration, so that the present application can realize accurate and rapid registration of the CT image and the transesophageal echocardiogram.

[0064] In the embodiment of the present application, before the key points are obtained from the first real-time transesophageal echocardiogram to obtain the second key point set, the CT image and the transesophageal echocardiogram registration method of the present application can further include:

[0065] performing image preprocessing on the first real-time transesophageal echocardiogram.

[0066] In the embodiment of the present application, the image preprocessing on the first real-time transesophageal echocardiogram can specifically include:

[0067] performing filtering processing on the first real-time transesophageal echocardiogram to remove noise in the first real-time transesophageal echocardiogram, and performing image enhancement processing on the first real-time transesophageal echocardiogram to highlight the structure boundary and improve the visibility and feature recognition of the image.

[0068] In the embodiment of the present application, the CT image specific section is marked with a target region boundary. The target region boundary is often artificially pre-marked.

[0069] After the non-rigid registration is performed on the preliminary fusion image to obtain the target fusion image, the CT image and the transesophageal echocardiogram registration method of the present application can further include:

[0070] Based on the target fusion image, the target region boundary is projected onto the first real-time transesophageal echocardiogram, so that by observing the first real-time transesophageal echocardiogram, the specific position of the target region boundary can also be directly observed.

[0071] In the embodiment of the present application, after the preliminary fusion image is subjected to non-rigid registration to obtain the target fusion image, the registration method of the CT image and the transesophageal echocardiogram of the present application can further include:

[0072] (1) obtaining a second real-time transesophageal echocardiogram collected by a transesophageal echocardiogram device.

[0073] (2) comparing and analyzing the second real-time transesophageal echocardiogram and the target fusion image to obtain a comparison and analysis result.

[0074] (3) when the comparison and analysis result shows that the second real-time transesophageal echocardiogram does not match the target fusion image, registering the second real-time transesophageal echocardiogram and the CT image specific section based on the foregoing registration method, and using the newly obtained target fusion image to replace the previous target fusion image.

[0075] (4) based on the newly obtained target fusion image, projecting the target region boundary onto the second real-time transesophageal echocardiogram to ensure the consistency of the target region boundary and the corresponding actual region boundary.

[0076] In the embodiment of the present application, the target region boundary is projected onto the first real-time transesophageal echocardiogram, which can specifically include:

[0077] The target region boundary is projected onto the first real-time transesophageal echocardiogram through an augmented reality technology or a preset navigation device. The preset navigation device can be an optical tracking navigation system or an electromagnetic tracking navigation system in the prior art.

[0078] Based on the same inventive concept, the present application also provides a registration device of a CT image and a transesophageal echocardiogram. Figure 2 is a structural schematic diagram of a registration device of a CT image and a transesophageal echocardiogram provided by the embodiment of the present application. As shown in Figure 2 the device includes:

[0079] The first acquisition module 21 is configured to acquire a CT image specific section of a heart.

[0080] The second acquisition module 22 is configured to acquire a first real-time transesophageal echocardiogram collected by a transesophageal echocardiogram device.

[0081] The third acquisition module 23 is configured to acquire key points of the CT image specific section to obtain a first key point set.

[0082] The fourth acquisition module 24 is configured to acquire key points in the first real-time transesophageal echocardiogram to obtain a second key point set.

[0083] The preliminary registration module 25 is configured to perform rigid registration on the CT image specific section and the first real-time transesophageal echocardiogram based on the first key point set and the second key point set to obtain a preliminary fusion image.

[0084] The fine registration module 26 is configured to perform non-rigid registration on the preliminary fusion image to obtain a target fusion image.

[0085] Optionally, the CT image and transesophageal echocardiogram registration device can further comprise an image preprocessing module configured to:

[0086] perform image preprocessing on the first real-time transesophageal echocardiogram.

[0087] Optionally, the image preprocessing module can be specifically configured to:

[0088] perform filtering processing and image enhancement processing on the first real-time transesophageal echocardiogram.

[0089] Optionally, the target region boundary is marked on the CT image specific section.

[0090] The CT image and transesophageal echocardiogram registration device can further comprise a first projection module configured to:

[0091] project the target region boundary onto the first real-time transesophageal echocardiogram based on the target fusion image.

[0092] Optionally, the CT image and transesophageal echocardiogram registration device can further comprise:

[0093] A fifth acquisition module is configured to acquire a second real-time transesophageal echocardiogram collected by the transesophageal echocardiogram device.

[0094] An analysis module is configured to perform comparative analysis on the second real-time transesophageal echocardiogram and the target fusion image to obtain a comparative analysis result.

[0095] A re-registration module is configured to perform registration on the second real-time transesophageal echocardiogram and the CT image specific section when the comparative analysis result shows that the second real-time transesophageal echocardiogram and the target fusion image do not match, and replace the previous target fusion image with the newly obtained target fusion image.

[0096] A second projection module is configured to project the target region boundary onto the second real-time transesophageal echocardiogram based on the newly obtained target fusion image.

[0097] Optionally, the first projection module is specifically used for:

[0098] The target region boundary is projected onto the first real-time transesophageal echocardiogram by using an augmented reality technology or a preset navigation device.

[0099] Optionally, the third acquisition module 23 can be specifically used for:

[0100] The scale-invariant feature transform algorithm is used to acquire key points on a specific section of the CT image, and a first key point set is obtained.

[0101] Optionally, the fourth acquisition module 24 can be specifically used for:

[0102] The Harris corner detection algorithm is used to acquire key points on the first real-time transesophageal echocardiogram, and a second key point set is obtained.

[0103] Based on a general inventive concept, the application further provides a computer readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement each step of the CT image and transesophageal echocardiogram registration method.

[0104] It can be understood that the same or similar parts in the above embodiments can be mutually referenced, and the content not described in detail in some embodiments can be referred to the same or similar content in other embodiments.

[0105] It should be noted that, in the description of the application, the terms "first", "second", etc. are only used for the purpose of description, and cannot be understood as indicating or implying relative importance. In addition, in the description of the application, unless otherwise specified, the meaning of "a plurality of" is at least two.

[0106] Any process or method descriptions in flowchart diagrams or described herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for implementing specific logic functions or steps in the processes. The scope of the preferred embodiments of the application includes additional implementation in which the functions are performed in a different order, in substantially simultaneous fashion, or in reverse order, according to the functions involved, and this should be understood by those skilled in the art to which the embodiments of the application belong.

[0107] It should be understood that each part of the present application can be realized by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if realized by hardware, and as in another embodiment, it can be realized by any one or a combination of the following technologies known in the art: discrete logic circuit with logic gate circuit for implementing logic function on data signal, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA) and the like.

[0108] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment method can be completed by a program instructing the relevant hardware, and the program can be stored in a computer readable storage medium, and when executed, includes one or a combination of steps of the embodiment method.

[0109] In addition, each functional unit in each embodiment of the present application can be integrated in one processing module, or each unit can exist physically, or two or more units can be integrated in one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of software functional module. The integrated module, if realized in the form of software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium.

[0110] The above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.

[0111] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above-mentioned terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0112] Although the embodiments of the present application have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the present application.

Claims

1. A method of registration of a CT image with a transesophageal echocardiogram, the method comprising: The method comprises the following steps: acquiring a CT image of a specific section of a heart; acquiring a first real-time transesophageal echocardiogram collected by a transesophageal echocardiography device; acquiring key points of the CT image of the specific section to obtain a first key point set; acquiring key points of the first real-time transesophageal echocardiogram to obtain a second key point set; performing rigid registration on the CT image of the specific section and the first real-time transesophageal echocardiogram based on the first key point set and the second key point set to obtain a preliminary fusion image; performing non-rigid registration on the preliminary fusion image to obtain a target fusion image; the CT image of the specific section is marked with a target region boundary; after the non-rigid registration on the preliminary fusion image to obtain the target fusion image, the CT image and the transesophageal echocardiogram registration method further comprises: projecting the target region boundary onto the first real-time transesophageal echocardiogram based on the target fusion image; after the non-rigid registration on the preliminary fusion image to obtain the target fusion image, the CT image and the transesophageal echocardiogram registration method further comprises: acquiring a second real-time transesophageal echocardiogram collected by the transesophageal echocardiography device; performing comparative analysis on the second real-time transesophageal echocardiogram and the target fusion image to obtain a comparative analysis result; when the comparative analysis result shows that the second real-time transesophageal echocardiogram does not match the target fusion image, performing registration on the second real-time transesophageal echocardiogram and the CT image of the specific section, and replacing the previous target fusion image with the newly obtained target fusion image; projecting the target region boundary onto the second real-time transesophageal echocardiogram based on the newly obtained target fusion image.

2. The method of registration of CT images with transesophageal echocardiograms of claim 1, wherein, Before the step of acquiring key points of the first real-time transesophageal echocardiogram to obtain a second key point set, the method further comprises: performing image preprocessing on the first real-time transesophageal echocardiogram.

3. The method of registration of CT images with transesophageal echocardiograms of claim 2, wherein, The image preprocessing on the first real-time transesophageal echocardiogram specifically comprises: performing filtering processing and image enhancement processing on the first real-time transesophageal echocardiogram.

4. The method of registering a CT image with an esophageal echocardiogram of claim 1, wherein, The step of projecting the target region boundary onto the first real-time transesophageal echocardiogram specifically comprises: projecting the target region boundary onto the first real-time transesophageal echocardiogram through augmented reality technology or a preset navigation device.

5. The method of registering a CT image with an esophageal echocardiogram of claim 1, wherein, The step of acquiring key points of the CT image of the specific section to obtain a first key point set specifically comprises: acquiring key points of the CT image of the specific section to obtain a first key point set by using a scale-invariant feature transform algorithm.

6. The method of registering a CT image with an esophageal echocardiogram of claim 1, wherein, The step of acquiring key points of the first real-time transesophageal echocardiogram to obtain a second key point set specifically comprises: acquiring key points of the first real-time transesophageal echocardiogram to obtain a second key point set by using a Harris corner point detection algorithm.

7. An apparatus for registering a CT image with a transesophageal echocardiogram, the apparatus comprising: The device is applied to the CT image and the transesophageal echocardiogram registration method in any one of claims 1 to 6. A first obtaining module is configured to obtain a CT image of a specific section of a heart; A second obtaining module is configured to obtain a first real-time transesophageal echocardiogram collected by a transesophageal echocardiography device; A third obtaining module is configured to obtain key points of the CT image of the specific section of the heart to obtain a first key point set; A fourth obtaining module is configured to obtain key points of the first real-time transesophageal echocardiogram to obtain a second key point set; A preliminary registration module is configured to perform rigid registration on the CT image of the specific section and the first real-time transesophageal echocardiogram based on the first key point set and the second key point set to obtain a preliminary fusion image; A fine registration module is configured to perform non-rigid registration on the preliminary fusion image to obtain a target fusion image.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements each step in the registration method of a CT image and a transesophageal echocardiogram according to any one of claims 1 to 6.

Images