CT image and transesophageal echocardiogram registration method and device and storage medium

Abstract

The invention provides a CT image and transesophageal echocardiogram registration method and device and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining a CT image specific section of a heart; obtaining a first real-time transesophageal echocardiogram collected by a transesophageal echocardiogram device; key point acquisition is carried out on the specific section of the CT image to obtain a first key point set; key point acquisition is conducted on the first real-time transesophageal echocardiogram, and a second key point set is obtained; based on the first key point set and the second key point set, rigid body registration is carried out on the CT image specific section and the first real-time transesophageal echocardiogram, and a preliminary fusion image is obtained; and performing non-rigid registration on the preliminary fusion image to obtain a target fusion image. According to the invention, accurate and rapid registration of the CT image and the transesophageal echocardiogram can be realized.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method, apparatus, and storage medium for registering CT images with transesophageal echocardiography. Background Technology

[0002] TEE (Transesophageal Echocardiography) is an echocardiographic technique that involves inserting an ultrasound probe through the mouth and esophagus to examine the deep structures of the heart from the back to the front at close range. CT (Computed Tomography) images are visual tomographic images generated based on computed tomography technology.

[0003] TEE (Transmitted Image Processing) is a widely used real-time imaging tool that can dynamically provide high-resolution images of the heart and adjacent anatomical structures in real time. However, TEE only provides two-dimensional information, with a limited field of view and a lack of intuitive representation of three-dimensional anatomical relationships. CT images can provide high-precision three-dimensional anatomical information; however, CT images are usually static data and lack real-time capability. Therefore, TEE can be registered with CT images to obtain a super-image that combines the advantages of both TEE and CT images.

[0004] Therefore, how to achieve accurate and rapid registration of CT images with transesophageal echocardiography has become an urgent technical problem to be solved. Summary of the Invention

[0005] In view of this, in order to solve the above-mentioned technical problems, the present invention provides a method, apparatus and storage medium for registering CT images with transesophageal echocardiography.

[0006] The present invention adopts the following technical solution: In a first aspect, the present invention provides a method for registering CT images with transesophageal echocardiography, comprising: Obtain specific sections of CT images of the heart; Acquire the first real-time transesophageal echocardiogram (TEE) image from the transesophageal echocardiogram device; Key points are acquired from specific sections of the CT image to obtain a first set of key points; Key points are acquired from the first real-time transesophageal echocardiogram to obtain a second set of key points; Based on the first set of key points and the second set of key points, rigid body registration is performed on a specific section of the CT image and the first real-time transesophageal echocardiogram to obtain a preliminary fused image. Non-rigid registration is performed on the preliminary fused image to obtain the target fused image.

[0007] Optionally, before acquiring key points from the first real-time transesophageal echocardiogram to obtain a second set of key points, the registration method of the CT image and transesophageal echocardiogram of the present invention further includes: Image preprocessing is performed on the first real-time transesophageal echocardiogram.

[0008] Optionally, image preprocessing is performed on the first real-time transesophageal echocardiogram, specifically including: The first real-time transesophageal echocardiogram is subjected to filtering and image enhancement processing.

[0009] Optionally, the target region boundary is marked on a specific section of the CT image; After performing non-rigid registration on the preliminary fused image to obtain the target fused image, the registration method of CT image and transesophageal echocardiography of the present invention further includes: Based on the target fusion image, the boundary of the target region is projected onto the first real-time transesophageal echocardiogram.

[0010] Optionally, after performing non-rigid registration on the preliminary fused image to obtain the target fused image, the registration method of CT image and transesophageal echocardiography of the present invention further includes: Acquire a second real-time transesophageal echocardiogram from the transesophageal echocardiography device; The second real-time transesophageal echocardiogram and the target fused image are compared and analyzed to obtain the comparison and analysis results. When the comparative analysis results show that there is a mismatch between the second real-time transesophageal echocardiogram and the target fusion image, the second real-time transesophageal echocardiogram and the CT image are registered in a specific section, and the newly obtained target fusion image is used to replace the previous target fusion image. Based on the newly obtained target fusion image, the boundary of the target region is projected onto the second real-time transesophageal echocardiogram.

[0011] Optionally, projecting the boundary of the target region onto the first real-time transesophageal echocardiogram specifically includes: The boundary of the target area is projected onto the first real-time transesophageal echocardiogram using augmented reality technology or a pre-set navigation device.

[0012] Optionally, key points are acquired from specific sections of the CT image to obtain a first set of key points, specifically including: Using the scale-invariant feature transformation algorithm, key points are obtained from a specific section of the CT image to obtain a first set of key points.

[0013] Optionally, key points are acquired from the first real-time transesophageal echocardiogram to obtain a second set of key points, specifically including: Using the Harris corner detection algorithm, key points are acquired from the first real-time transesophageal echocardiogram to obtain a second set of key points.

[0014] Secondly, the present invention also provides a registration device for CT images and transesophageal echocardiography, comprising: The first acquisition module is used to acquire specific sections of CT images of the heart; The second acquisition module is used to acquire the first real-time transesophageal echocardiogram collected by the transesophageal echocardiography device; The third acquisition module is used to acquire key points from a specific section of the CT image to obtain a first set of key points. The fourth acquisition module is used to acquire key points from the first real-time transesophageal echocardiogram to obtain a second set of key points; The preliminary registration module is used to perform rigid body registration of a specific section of the CT image and the first real-time transesophageal echocardiogram based on the first set of key points and the second set of key points to obtain a preliminary fused image. The fine registration module is used to perform non-rigid registration on the preliminary fused image to obtain the target fused image.

[0015] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the various steps in the registration method of CT images and transesophageal echocardiography as described above.

[0016] The present invention adopts the above technical solution, which first performs rigid body registration on a specific section of a CT image and a first real-time transesophageal echocardiogram to obtain a preliminary fused image, and then performs non-rigid body registration on the preliminary fused image to achieve further refined registration, so that the present invention can achieve accurate and rapid registration between CT images and transesophageal echocardiograms. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart of a method for registering CT images with transesophageal echocardiography provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a CT image and transesophageal echocardiography registration device provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0020] TEE (Transcranial Echocardiography) is an echocardiographic technique that involves inserting an ultrasound probe through the mouth and esophagus to examine the deep structures of the heart from the back to the front. CT images are visual tomographic images generated based on computed tomography (CT) technology.

[0021] TEE (Transmitted Image Processing) is a widely used real-time imaging tool that can dynamically provide high-resolution images of the heart and adjacent anatomical structures in real time. However, TEE only provides two-dimensional information, with a limited field of view and a lack of intuitive representation of three-dimensional anatomical relationships. CT images can provide high-precision three-dimensional anatomical information; however, CT images are usually static data and lack real-time capability. Therefore, TEE can be registered with CT images to obtain a super-image that combines the advantages of both TEE and CT images.

[0022] Based on this, in order to achieve accurate and rapid registration of CT images and transesophageal echocardiography, the present invention provides a method, apparatus and storage medium for registering CT images and transesophageal echocardiography.

[0023] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0024] Figure 1 This is a schematic flowchart of a method for registering CT images with transesophageal echocardiography provided in an embodiment of the present invention. Figure 1 As shown, this process includes: Step 101: Obtain a specific section of a CT image of the heart.

[0025] Specifically, specific sections of the CT images are extracted from pre-taken cardiac CT images. These specific sections can be long-axis sections or mitral valve level sections, etc.

[0026] Step 102: Acquire the first real-time transesophageal echocardiogram from the transesophageal echocardiography device.

[0027] Specifically, the position and angle of ultrasound irradiation in transesophageal echocardiography are pre-planned.

[0028] Step 103: Obtain key points from specific sections of the CT image to obtain the first set of key points.

[0029] Specifically, the SIFT (Scale-Invariant Feature Transform) algorithm can be used to extract key points from specific sections of a CT image, resulting in a first set of key points. The SIFT algorithm can detect key points in an image, such as corners and edges, and generate highly discriminative feature descriptors.

[0030] Step 104: Obtain key points from the first real-time transesophageal echocardiogram to obtain the second set of key points.

[0031] Specifically, the Harris corner detection algorithm can be used to acquire key points from the first real-time transesophageal echocardiogram, resulting in a second set of key points. The Harris corner detection algorithm can be used to identify corner regions in the image with significant grayscale changes, such as the corners of objects and edge intersections.

[0032] Step 105: Based on the first set of key points and the second set of key points, perform rigid body registration on specific sections of the CT image and the first real-time transesophageal echocardiogram to obtain a preliminary fused image.

[0033] Specifically, rigid body registration aims to achieve spatial alignment between different images through operations such as translation and rotation.

[0034] Here, rigid body registration uses the key points in the first set of key points as reference benchmarks, calculates the translation vectors and rotation matrices required for the key points in the second set of key points, so that the first real-time transesophageal echocardiogram can achieve preliminary spatial position matching with a specific section of the CT image without changing its shape, reducing the overall spatial difference between the images and laying the foundation for subsequent more refined non-rigid body registration.

[0035] Step 106: Perform non-rigid registration on the preliminary fused image to obtain the target fused image.

[0036] Specifically, non-rigid registration is a key technology that further addresses local deformation issues in images based on the preliminary fused images obtained from rigid registration. Non-rigid registration involves setting a control point grid in the image and continuously adjusting the positions of these control points to perform localized stretching, compression, and bending deformation operations on the image. This allows specific sections of the CT image and the first real-time transesophageal echocardiogram to accurately match easily deformable details such as myocardial fibers and valve leaflets, thereby improving image fusion accuracy.

[0037] The embodiments of the present invention adopt the above technical solution. By first performing rigid body registration on a specific section of the CT image and the first real-time transesophageal echocardiogram to obtain a preliminary fused image, and then performing non-rigid body registration on the preliminary fused image to achieve further refined registration, the present invention can achieve accurate and rapid registration between CT images and transesophageal echocardiograms.

[0038] In this embodiment of the invention, before acquiring key points from the first real-time transesophageal echocardiogram to obtain the second set of key points, the registration method of CT images and transesophageal echocardiograms of the present invention may further include: Image preprocessing was performed on the first real-time transesophageal echocardiogram.

[0039] In this embodiment of the invention, image preprocessing of the first real-time transesophageal echocardiogram may specifically include: The first real-time transesophageal echocardiogram is filtered to remove noise; and the first real-time transesophageal echocardiogram is enhanced to highlight structural boundaries and improve image visibility and feature recognition.

[0040] In this embodiment of the invention, the boundary of the target region is marked on a specific section of the CT image. The boundary of the target region is often pre-marked manually.

[0041] After performing non-rigid registration on the preliminary fused image to obtain the target fused image, the registration method of CT image and transesophageal echocardiography of the present invention may further include: Based on the target fusion image, the boundary of the target region is projected onto the first real-time transesophageal echocardiogram, so that the specific location of the target region boundary can be seen intuitively by observing the first real-time transesophageal echocardiogram.

[0042] In this embodiment of the invention, after performing non-rigid registration on the preliminary fused image to obtain the target fused image, the registration method of CT image and transesophageal echocardiography of the present invention may further include: (1) Acquire a second real-time transesophageal echocardiogram from a transesophageal echocardiogram device.

[0043] (2) The second real-time transesophageal echocardiogram and the target fusion image were compared and analyzed to obtain the comparison analysis results.

[0044] (3) When the comparative analysis results show that there is a mismatch between the second real-time transesophageal echocardiogram and the target fusion image, the second real-time transesophageal echocardiogram and the CT image are registered according to the aforementioned registration method, and the newly obtained target fusion image is used to replace the previous target fusion image.

[0045] (4) Based on the latest target fusion image, the boundary of the target region is projected onto the second real-time transesophageal echocardiogram to ensure the consistency between the boundary of the target region and the corresponding actual region boundary.

[0046] In this embodiment of the invention, projecting the boundary of the target region onto a first real-time transesophageal echocardiogram may specifically include: The boundary of the target area is projected onto a first real-time transesophageal echocardiogram using augmented reality technology or a pre-set navigation device. The pre-set navigation device can be an existing optical tracking navigation system or an electromagnetic tracking navigation system.

[0047] Based on a general inventive concept, the present invention also provides a registration device for CT images and transesophageal echocardiography. Figure 2 This is a schematic diagram of a CT image and transesophageal echocardiography registration device provided in an embodiment of the present invention. Figure 2 As shown, this device includes: The first acquisition module 21 is used to acquire specific sections of CT images of the heart.

[0048] The second acquisition module 22 is used to acquire the first real-time transesophageal echocardiogram collected by the transesophageal echocardiography device.

[0049] The third acquisition module 23 is used to acquire key points of a specific section of a CT image to obtain a first set of key points.

[0050] The fourth acquisition module 24 is used to acquire key points from the first real-time transesophageal echocardiogram to obtain a second set of key points.

[0051] The preliminary registration module 25 is used to perform rigid body registration of a specific section of a CT image and a first real-time transesophageal echocardiogram based on a first set of key points and a second set of key points to obtain a preliminary fused image.

[0052] The fine registration module 26 is used to perform non-rigid registration on the preliminary fused image to obtain the target fused image.

[0053] Optionally, the CT image and transesophageal echocardiography registration device of the present invention may further include an image preprocessing module for: Image preprocessing was performed on the first real-time transesophageal echocardiogram.

[0054] Optional, the image preprocessing module can be used for: The first real-time transesophageal echocardiogram was filtered and enhanced.

[0055] Optionally, the boundaries of the target region are marked on specific sections of the CT image.

[0056] The CT image and transesophageal echocardiography registration device of the present invention may further include a first projection module for: Based on the target fusion image, the boundary of the target region is projected onto the first real-time transesophageal echocardiogram.

[0057] Optionally, the CT image and transesophageal echocardiography registration device of the present invention may further include: The fifth acquisition module is used to acquire the second real-time transesophageal echocardiogram obtained by the transesophageal echocardiography device.

[0058] The analysis module is used to compare and analyze the second real-time transesophageal echocardiogram and the target fused image to obtain the comparison and analysis results.

[0059] The re-registration module is used to register specific sections of the second real-time transesophageal echocardiogram and the CT image when the comparative analysis results show a mismatch between the second real-time transesophageal echocardiogram and the target fusion image, and to replace the previous target fusion image with the latest target fusion image.

[0060] The second projection module is used to project the boundary of the target region onto the second real-time transesophageal echocardiogram based on the latest obtained target fusion image.

[0061] Optional, the first projection module is specifically used for: Using augmented reality technology or pre-set navigation equipment, the boundary of the target area is projected onto the first real-time transesophageal echocardiogram.

[0062] Optionally, the third acquisition module 23 can be used for: By using the scale-invariant feature transform algorithm, key points are obtained from specific cross sections of CT images to obtain the first set of key points.

[0063] Optionally, the fourth acquisition module 24 can be used for: Using the Harris corner detection algorithm, key points are acquired from the first real-time transesophageal echocardiogram to obtain a second set of key points.

[0064] Based on a general inventive concept, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the various steps in the registration method of CT images and transesophageal echocardiography as described above.

[0065] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0066] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0067] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0068] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0069] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0070] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0071] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0072] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0073] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for registering CT images with transesophageal echocardiography, characterized in that, include: Obtain specific sections of CT images of the heart; Acquire the first real-time transesophageal echocardiogram (TEE) image acquired by the transesophageal echocardiography device; Key points are acquired from specific sections of the CT image to obtain a first set of key points; Key points are acquired from the first real-time transesophageal echocardiogram to obtain a second set of key points; Based on the first set of key points and the second set of key points, rigid body registration is performed on a specific section of the CT image and the first real-time transesophageal echocardiogram to obtain a preliminary fused image. Non-rigid registration is performed on the preliminary fused image to obtain the target fused image.

2. The registration method for CT images and transesophageal echocardiography according to claim 1, characterized in that, Before acquiring key points from the first real-time transesophageal echocardiogram to obtain the second set of key points, the process also includes: Image preprocessing is performed on the first real-time transesophageal echocardiogram.

3. The registration method for CT images and transesophageal echocardiography according to claim 2, characterized in that, Image preprocessing of the first real-time transesophageal echocardiogram specifically includes: The first real-time transesophageal echocardiogram is subjected to filtering and image enhancement processing.

4. The registration method for CT images and transesophageal echocardiography according to claim 1, characterized in that, The target region boundary is marked on a specific section of the CT image; After performing non-rigid registration on the preliminary fused image to obtain the target fused image, the process further includes: Based on the target fusion image, the boundary of the target region is projected onto the first real-time transesophageal echocardiogram.

5. The registration method for CT images and transesophageal echocardiography according to claim 4, characterized in that, After performing non-rigid registration on the preliminary fused image to obtain the target fused image, the process further includes: Acquire a second real-time transesophageal echocardiogram from the transesophageal echocardiography device; The second real-time transesophageal echocardiogram and the target fused image are compared and analyzed to obtain the comparison and analysis results. When the comparative analysis results show that there is a mismatch between the second real-time transesophageal echocardiogram and the target fusion image, the second real-time transesophageal echocardiogram and the CT image are registered in a specific section, and the newly obtained target fusion image is used to replace the previous target fusion image. Based on the newly obtained target fusion image, the boundary of the target region is projected onto the second real-time transesophageal echocardiogram.

6. The registration method for CT images and transesophageal echocardiography according to claim 4, characterized in that, Projecting the boundary of the target region onto the first real-time transesophageal echocardiogram specifically includes: The boundary of the target area is projected onto the first real-time transesophageal echocardiogram using augmented reality technology or a pre-set navigation device.

7. The method for registering CT images and transesophageal echocardiography according to claim 1, characterized in that, Key points are acquired from specific sections of the CT image to obtain a first set of key points, which specifically includes: Using the scale-invariant feature transformation algorithm, key points are obtained from a specific section of the CT image to obtain a first set of key points.

8. The method for registering CT images and transesophageal echocardiography according to claim 1, characterized in that, Key points were acquired from the first real-time transesophageal echocardiogram to obtain a second set of key points, specifically including: Using the Harris corner detection algorithm, key points are acquired from the first real-time transesophageal echocardiogram to obtain a second set of key points.

9. A registration device for CT images and transesophageal echocardiography, characterized in that, include: The first acquisition module is used to acquire specific sections of CT images of the heart; The second acquisition module is used to acquire the first real-time transesophageal echocardiogram collected by the transesophageal echocardiography device; The third acquisition module is used to acquire key points from a specific section of the CT image to obtain a first set of key points. The fourth acquisition module is used to acquire key points from the first real-time transesophageal echocardiogram to obtain a second set of key points; The preliminary registration module is used to perform rigid body registration of a specific section of the CT image and the first real-time transesophageal echocardiogram based on the first set of key points and the second set of key points to obtain a preliminary fused image. The fine registration module is used to perform non-rigid registration on the preliminary fused image to obtain the target fused image.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the registration method for CT images and transesophageal echocardiography as described in any one of claims 1 to 8.

Images