Balloon type heart cavity endoscope imaging system and method based on three-dimensional mapping system

Abstract

The invention discloses a balloon type heart cavity endoscope imaging system and method based on a three-dimensional mapping system, and belongs to the technical field of medical equipment. The problems that in an existing imaging method, imaging in the heart cavity is interfered by blood, the image definition is insufficient, and intuitive visualization of an inner wall image cannot be achieved are solved. A filling opening is formed in the handle end of the balloon type heart cavity endoscope, the filling opening is used for injecting liquid into a balloon so that the balloon can expand to be attached to the inner wall of a heart cavity to drain blood at the attached position of the inner wall of the heart cavity, and an image sensor is used for conducting image collection on the tissue, attached to the outside of the balloon, of the inner wall of the heart cavity; the positioning assembly is used for acquiring the position and posture of the image sensor in real time; the image processing unit is used for preprocessing the image acquired by the image sensor and then transmitting the image to the three-dimensional mapping system; the three-dimensional mapping system establishes a mapping heart cavity model, and the collected image and the mapping heart cavity model are matched, fused and displayed. The method is mainly used for intracardiac imaging.

Description

Technical Field

[0001] This invention belongs to the field of medical device technology. Background Technology

[0002] In interventional cardiac procedures (such as arrhythmia ablation), three-dimensional mapping systems (e.g., the StarTrek system) are widely used to construct three-dimensional electroanatomical models of the heart chambers to guide catheter navigation. However, existing three-dimensional mapping systems mainly rely on electrical signals and geometric point cloud data, lacking intuitive image information of the solid chamber walls. Traditional intracardiac imaging techniques (such as intracardiac ultrasound or optical coherence tomography) are often affected by blood interference, resulting in blurred images and low contrast, affecting the physician's accurate judgment of tissue structures.

[0003] In the prior art, patent document CN1124721A (publication date: June 1996) describes an intracardiac ultrasound imaging device, but the imaging quality of this device is limited in a blood environment, and it does not achieve image fusion with a three-dimensional mapping system. Non-patent literature, *Heart Rhythm Journal* (2019, Vol. 16, No. 5), points out that when cardiac endoscopes image in blood, the blood needs to be isolated by flushing or a balloon, but existing balloon designs are not optimized for integration with a three-dimensional mapping system, resulting in the inability to map image data into a three-dimensional model in real time.

[0004] However, existing imaging methods suffer from several drawbacks: intracardiac imaging is affected by blood flow, resulting in insufficient image clarity; imaging data is separated from the three-dimensional mapping system, making it impossible to achieve intuitive visualization of the inner wall image; and there is a lack of efficient image-model fusion schemes, which affects surgical accuracy and efficiency. Summary of the Invention

[0005] This invention addresses the problems of existing imaging methods, such as intracardiac imaging being affected by blood flow, insufficient image clarity, and inability to achieve intuitive visualization of the inner wall image. It provides a balloon-type intracardiac endoscopic imaging system and method based on a three-dimensional mapping system.

[0006] The balloon-type cardiac endoscopy imaging system based on a three-dimensional mapping system of the present invention includes: a balloon-type cardiac endoscope, an image sensor, a positioning sensor, an image processing unit, and a three-dimensional mapping system;

[0007] A balloon-type cardiac endoscope includes a balloon, a handpiece, and a tube connecting the balloon and the handpiece;

[0008] The outer end of the handle is provided with an infusion port, which is used to inject liquid into the balloon through the infusion tube, so that the balloon expands and adheres to the inner wall of the heart chamber to drain the blood at the inner wall of the heart chamber. A positioning component is provided at the root of the balloon, which is used to acquire the position and orientation of the image sensor in real time.

[0009] An image sensor is located inside the balloon and at the front end of the balloon. The image sensor is used to acquire images of the intracardiac wall tissue that is attached to the outside of the balloon.

[0010] The image processing unit is used to preprocess the images acquired by the image sensor and transmit the preprocessed images to the three-dimensional mapping system.

[0011] The three-dimensional mapping system is used to establish a mapping model of the heart cavity. At the same time, it performs image segmentation feature extraction on the preprocessed image, and uses the extracted features combined with the position and orientation of the image sensor to match and fuse the segmented image with the mapping model of the heart cavity, and displays the mapping model of the heart cavity after matching and fusing the image.

[0012] Furthermore, in this invention, the handle end of the balloon-type cardiac endoscope is also provided with an image acquisition cable interface and a positioning component cable interface. The image acquisition cable interface and the positioning component cable interface are respectively connected to the signal output end of the image sensor and the positioning component through corresponding cables, and the cables are set inside the tube.

[0013] Furthermore, in this invention, the positioning component is a positioning electrode or a magnetic positioning sensor.

[0014] Furthermore, in this invention, the balloon is made of a transparent material.

[0015] Furthermore, in this invention, the image processing unit preprocesses the images acquired by the image sensor by enhancing and encoding the images.

[0016] Furthermore, in this invention, the three-dimensional mapping system uses an improved SIFT algorithm to extract feature points from the acquired images and the mapped cardiac cavity model. Based on the position and orientation of the cardiac endoscope, affine transformation is used to register the segmented image feature points with the feature points of the mapped cardiac cavity model, thus mapping the segmented image onto the mapped cardiac cavity model and achieving matching and fusion between the segmented image and the mapped cardiac cavity model.

[0017] Furthermore, in this invention, the three-dimensional mapping system adopts the StarTrek cardiac electrophysiology three-dimensional mapping system.

[0018] Furthermore, in this invention, the liquid injected into the balloon of the balloon-type cardiac endoscope is physiological saline or a transparent contrast agent.

[0019] Furthermore, in this invention, a cold light source is also provided inside the balloon.

[0020] A balloon-guided endoscopic imaging method for cardiac chambers using a three-dimensional mapping system, the method comprising:

[0021] Step 1: Insert the balloon-type endoscope into the target location, inject liquid into the liquid infusion port of the balloon-type endoscope, and when the balloon inflates, the outer wall of the balloon will adhere to the inner wall of the heart chamber, displacing the blood at the adhering point, and control the image sensor to acquire images of the tissue surface in contact with the outer wall of the balloon.

[0022] Simultaneously, the position and orientation of the image sensor are acquired through the positioning component of the balloon-type endoscope;

[0023] Step 2: The image processing unit is used to preprocess the image of the tissue surface in contact with the outer wall of the balloon, and the three-dimensional mapping system 300 is used to perform image segmentation and feature extraction on the preprocessed image in sequence.

[0024] Step 3: Based on the extracted features and the position and orientation of the image sensor, the segmented image is matched and fused with the mapped cardiac cavity model, and the matched and fused mapped cardiac cavity model is displayed in real time.

[0025] This invention uses a balloon to drain blood and employs a CMOS sensor to directly acquire tissue images, improving image resolution by approximately 30% (based on simulation experiments and comparisons in a blood environment); the image-model fusion algorithm has a processing latency of less than 100ms, and the average registration error between the image and the mapping model is less than 1mm; the system has high integration, allowing direct observation of inner wall details in a 3D interface, reducing the time required for repeated catheter positioning. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of a balloon-type cardiac endoscope.

[0027] Figure 2 This is a schematic diagram of the balloon-type cardiac endoscopy imaging system of the three-dimensional mapping system described in this invention.

[0028] Figure 3 To compare the three-dimensional models before and after fusing the cardiac endoscopy images, in the figure, RSPV represents the right superior pulmonary vein, RIPV represents the right inferior pulmonary vein, LAA represents the left atrial appendage, LIPV represents the left inferior pulmonary vein, and LSPV represents the left superior pulmonary vein;

[0029] Figure 3 (a) is a traditional 3D model. Figure 3 (b) is a 3D model unfolded by projection after fusing the inner wall image. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0031] Specific implementation method one: Refer to Figure 1 and Figure 2 This embodiment describes a balloon-type cardiac endoscopy imaging system based on a three-dimensional mapping system, which includes: a balloon-type cardiac endoscope 100, an image sensor 102, a positioning sensor 103, an image processing unit 200, and a three-dimensional mapping system 300.

[0032] The balloon-type cardiac endoscope 100 includes a balloon 101, a handle, and a tube 104 connecting the balloon 101 and the handle;

[0033] The outer end of the handle is provided with an infusion port, which is used to inject liquid into the balloon 101 through the infusion tube, so that the balloon 101 expands and adheres to the inner wall of the heart chamber to drain the blood at the inner wall of the heart chamber. The base of the balloon 101 is provided with a positioning component 103, which is used to acquire the position and orientation of the image sensor 102 in real time.

[0034] An image sensor 102 is disposed inside the balloon 101 and located at the front end of the balloon 101. The image sensor 102 is used to acquire images of the intracardiac wall tissue that is attached to the outside of the balloon 101.

[0035] The image processing unit 200 is used to preprocess the image acquired by the image sensor 102 and transmit the preprocessed image to the three-dimensional mapping system 300.

[0036] The three-dimensional mapping system 300 is used to establish a mapping cardiac cavity model. At the same time, it performs image segmentation feature extraction on the preprocessed image, and uses the extracted features combined with the position and orientation of the image sensor 102 to match and fuse the segmented image with the mapping cardiac cavity model, and displays the mapping cardiac cavity model after matching and fusing the image.

[0037] Furthermore, in this embodiment, the handle end of the balloon-type cardiac endoscope 100 is also provided with an image acquisition cable interface 105 and a positioning component cable interface 106. The image acquisition cable interface 105 and the positioning component cable interface 106 are respectively connected to the signal output ends of the image sensor and the positioning component through corresponding cables, and the cables are arranged inside the tube 104.

[0038] Furthermore, in this embodiment, the positioning component is a positioning electrode or a magnetic positioning sensor 103.

[0039] Furthermore, in this embodiment, the balloon is made of a transparent material.

[0040] Furthermore, in this embodiment, the process of preprocessing the image acquired by the image sensor 102 by the image processing unit 200 includes image enhancement and encoding.

[0041] Furthermore, in this invention, the three-dimensional mapping system 300 uses an improved SIFT algorithm to extract feature points in the acquired image and the mapped cardiac cavity model. Based on the position and orientation of the cardiac endoscope, it uses affine transformation to register the segmented image feature points with the feature points of the mapped cardiac cavity model, and maps the segmented image to the mapped cardiac cavity model, thereby achieving matching and fusion of the segmented image and the mapped cardiac cavity model.

[0042] Furthermore, in this embodiment, the three-dimensional mapping system 300 adopts the StarTrek cardiac electrophysiology three-dimensional mapping system 300.

[0043] Furthermore, in this embodiment, the liquid injected into the balloon of the balloon-type cardiac endoscope 100 is physiological saline or a transparent contrast agent.

[0044] Furthermore, in this embodiment, a cold light source is also provided inside the balloon.

[0045] The cold light source described in this embodiment is an LED light source set at the outer end. The handle is also provided with an optical fiber port 107, through which light signals are transmitted to the balloon. The image processing unit adjusts the brightness of the LED light source according to the brightness of the image collected by the image sensor, thus ensuring the clarity of the image.

[0046] Specific Implementation Method Two: The balloon-type endoscopic cardiac imaging method of the three-dimensional mapping system described in this implementation method is based on the system described in Specific Implementation Method One. The method includes:

[0047] Step 1: Insert the balloon-type endoscope 100 into the target position and inject liquid into the liquid infusion port 107 of the balloon-type endoscope 100. When the balloon inflates, the outer wall of the balloon comes into contact with the inner wall of the heart chamber, displacing the blood at the contact point. Control the image sensor 102 to acquire the image of the tissue surface in contact with the outer wall of the balloon.

[0048] Simultaneously, the position and orientation of the image sensor 102 are acquired through the positioning component of the balloon-type endoscope 100;

[0049] Step 2: The image processing unit 200 preprocesses the image of the tissue surface in contact with the outer wall of the balloon, and the three-dimensional mapping system 300 performs image segmentation and feature extraction on the preprocessed image in sequence.

[0050] Step 3: Based on the extracted features and the position and orientation of the image sensor 102, the segmented image is matched and fused with the mapped cardiac cavity model, and the matched and fused mapped cardiac cavity model is displayed in real time.

[0051] The system described in this invention includes a cardiac balloon endoscope, an image processing board, and a three-dimensional mapping device. Taking the StarTrek cardiac electrophysiology three-dimensional mapping device as an example, it mainly includes a processing unit, a workstation, a display, and a power supply module. System image data is transmitted from the balloon endoscope to the workstation via the image processing board; system positioning data is transmitted from the balloon positioning electrode or magnetic positioning sensor 103 to the workstation via the processing unit; the balloon endoscope and image processing board are powered by the workstation, or can be powered by other dedicated modules.

[0052] Appendix Figure 1 This is an illustration of a balloon-type cardiac endoscope, which mainly includes a balloon, an image sensor, positioning electrodes / magnetic positioning sensors, and a perfusion pathway. The image sensor can be fixed in the center of the balloon or moved circumferentially by the handpiece mechanism. In this case, the positioning electrodes / magnetic positioning sensors can only use the magnetic positioning sensor to track the position and orientation of the image sensor in real time.

[0053] Appendix Figure 2 This diagram illustrates the system hardware composition, including a cardiac balloon endoscope, an image processing board, and a 3D mapping device. Taking the StarTrek cardiac electrophysiology 3D mapping device as an example, it mainly includes a processing unit, a workstation, a display, and a power supply module. System image data is transmitted from the balloon endoscope to the workstation via the image processing board; system positioning data is transmitted from the balloon positioning electrodes or magnetic positioning sensors to the workstation via the processing unit; the balloon endoscope and image processing board are powered by the workstation, or alternatively by other dedicated modules.

[0054] Appendix Figure 3 To compare the three-dimensional models before and after fusing the cardiac endoscopy images, the left side is the traditional three-dimensional model, and the right side is the three-dimensional model unfolded by projection after fusing the inner wall image.

[0055] Specific implementation process:

[0056] (1) Place the StarTrek system-compatible surface electrodes at predetermined locations on the patient’s torso and right leg.

[0057] (2) Connect the matching body surface electrodes to the StarTrek system through the body surface electrode connection box.

[0058] (3) Anesthetize or sedate the target (in a supine position) and perform the procedure according to the specific plan, the attending physician’s preference and / or professional guidelines.

[0059] (4) In accordance with customary practice, establish vascular access (vein and / or artery, femoral artery or jugular vein) for indwelling catheter placement.

[0060] (5) The catheter is connected to the system via a direct connection or a catheter connection box. The PFA catheter is connected to the system via a direct connection to the PFA / RF interface, and the radiofrequency ablation catheter is connected to the system via a direct RF interface.

[0061] (6) The operator inserts the catheter into the target chamber, performs imaging mapping, and obtains a three-dimensional electroanatomical map of the heart chamber.

[0062] (7) The indwelling catheter is displayed in real time in 3D on the cardiac electroanatomy map, providing an intuitive and easy-to-use navigation function.

[0063] (8) The operator performs ablation under the guidance of the three-dimensional mapping equipment. According to the user settings, the three-dimensional mapping equipment continuously stores, tracks and quantifies the catheter position and various electrophysiological parameters collected before, during and after ablation (radiofrequency, cryotherapy, pulse).

[0064] (9) The surgeon can use balloon endoscopy to visualize the cardiac structures of interest before, during and after ablation.

[0065] (10) Specifically, the balloon endoscope enters the target chamber through the established vascular access, and the balloon inflation pressure is 0.5-1.0 atm.

[0066] (11) Under the guidance of the mapping equipment, the operator moves the balloon to the cardiac cavity structure of interest. The system will automatically collect tissue images at the balloon placement point, process them, and then register and project them onto the inner wall of the mapping three-dimensional model to provide the operator with intuitive tissue structure information.

[0067] The balloon filling medium can be replaced with a transparent contrast agent to enhance image contrast; the image registration algorithm can be replaced with the ORB (Oriented Fast Rotation Briefing) algorithm, which is suitable for scenarios with low computing resources.

[0068] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A balloon-type cardiac endoscopic imaging system based on a three-dimensional mapping system, characterized in that, include: Balloon-type cardiac endoscope (100), image sensor (102), positioning sensor (103), image processing unit (200) and three-dimensional mapping system (300); The balloon endoscope (100) includes a balloon (101), a handle, and a tube (104) connecting the balloon (101) and the handle. The outer end of the handle is provided with an infusion port, which is used to inject liquid into the balloon (101) through the infusion tube, so that the balloon (101) expands and adheres to the inner wall of the heart chamber to drain the blood at the inner wall of the heart chamber. The base of the balloon (101) is provided with a positioning component (103), which is used to acquire the position and orientation of the image sensor (102) in real time. An image sensor (102) is disposed inside the balloon (101) and located at the front end of the balloon (101). The image sensor (102) is used to acquire images of the intracardiac wall tissue that is attached to the outside of the balloon (101). The image processing unit (200) is used to preprocess the image acquired by the image sensor (102) and transmit the preprocessed image to the three-dimensional mapping system (300). The three-dimensional mapping system (300) is used to establish a mapping heart cavity model. At the same time, it performs image segmentation feature extraction on the preprocessed image, and uses the extracted features combined with the position and orientation of the image sensor (102) to match and fuse the segmented image with the mapping heart cavity model, and displays the mapping heart cavity model after matching and fusing the image.

2. The balloon-type cardiac endoscopic imaging system based on a three-dimensional mapping system according to claim 1, characterized in that, The handle of the balloon-type cardiac endoscope (100) is also provided with an image acquisition cable interface (105) and a positioning component cable interface (106). The image acquisition cable interface (105) and the positioning component cable interface (106) are respectively connected to the signal output end of the image sensor and the positioning component through corresponding cables. The cables are set inside the tube (104).

3. The balloon-type cardiac endoscopic imaging system based on a three-dimensional mapping system according to claim 1 or 2, characterized in that, The positioning component is a positioning electrode or a magnetic positioning sensor (103).

4. The balloon-type cardiac endoscopic imaging system based on a three-dimensional mapping system according to claim 1 or 2, characterized in that, The balloon is made of transparent material.

5. The balloon-type cardiac endoscopic imaging system based on a three-dimensional mapping system according to claim 1 or 2, characterized in that, The image processing unit (200) preprocesses the images acquired by the image sensor (102) by enhancing and encoding the images.

6. The balloon-type cardiac endoscopic imaging system based on a three-dimensional mapping system according to claim 1 or 2, characterized in that, The three-dimensional mapping system (300) uses an improved SIFT algorithm to extract feature points in the acquired images and the mapping heart cavity model. Based on the position and orientation of the endoscope, it uses affine transformation to register the feature points of the segmented image with the feature points of the mapping heart cavity model, and maps the segmented image to the mapping heart cavity model, thus achieving matching and fusion of the segmented image and the mapping heart cavity model.

7. The balloon-type cardiac endoscopic imaging system based on a three-dimensional mapping system according to claim 1 or 2, characterized in that, The three-dimensional mapping system (300) is the StarTrek cardiac electrophysiology three-dimensional mapping system (300).

8. The balloon-type cardiac endoscopic imaging system based on a three-dimensional mapping system according to claim 1 or 2, characterized in that, The fluid injected into the balloon of the balloon-type cardiac endoscope (100) is either physiological saline or a clear contrast agent.

9. The balloon-type cardiac endoscopic imaging system based on a three-dimensional mapping system according to claim 1 or 2, characterized in that, The balloon is also equipped with a cold light source.

10. A balloon-type endoscopic imaging method for cardiac chambers using a three-dimensional mapping system, characterized in that, The method includes: Step 1: Insert the balloon-type endoscope (100) into the target position and inject liquid into the liquid infusion port (107) of the balloon-type endoscope (100). When the balloon expands, the outer wall of the balloon comes into contact with the inner wall of the heart chamber, dissipating the blood at the contact point. Control the image sensor (102) to collect the image of the tissue surface in contact with the outer wall of the balloon. Simultaneously, the position and orientation of the image sensor (102) are acquired through the positioning component of the balloon-type endoscope (100); Step 2: The image processing unit (200) is used to preprocess the image of the contact tissue surface of the outer wall of the balloon, and the three-dimensional mapping system (300) is used to perform image segmentation and feature extraction on the preprocessed image in sequence. Step 3: Based on the extracted features and the position and orientation of the image sensor (102), the segmented image is matched and fused with the mapped cardiac cavity model, and the matched and fused mapped cardiac cavity model is displayed in real time.

Images