AI-based ultra-thin electronic endoscope visual navigation system for radiofrequency ablation

Abstract

This invention discloses an artificial intelligence-based ultra-thin electronic endoscopic visual navigation system for radiofrequency ablation, belonging to the field of medical device technology. It includes an endoscopic camera system, an energy system, an illumination system, and a display system. The endoscopic camera system acquires real-time images of the human body, processes the images, and outputs the processed images and the original video images to the display terminal. The energy system provides laser energy and delivers it to the front end via an optical fiber guidewire for the doctor to perform blood vessel blockage treatment at the target location. The illumination system provides light source illumination within the field of view for image acquisition. The endoscopic camera module and the radiofrequency energy module are interconnected, automatically adjusting the required energy level based on the size of the ablation vessel in the image, achieving automatic dosage control. The automated navigation function reduces the workload of the doctor, making the operation simpler and faster. Navigation helps avoid accidental damage to normal tissue, reducing the risks during the treatment process.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to an artificial intelligence-based ultra-thin electronic endoscope visual navigation system for radiofrequency therapy. Background Technology

[0002] With advancements in medical technology, endoscopic treatment has become an important tool for disease diagnosis and treatment. However, precise navigation within the complex human body remains a challenge, especially when using fetoscopic radiofrequency ablation, where doctors need to accurately locate the lesion and implement precise treatment.

[0003] In current ultra-thin electronic endoscopy treatments, the location of the target and the amount of radiofrequency energy delivered depend on the doctor's experience and skill. There are differences between different doctors or even between different surgeries performed by the same doctor, which leads to unstable surgical results or efficacy.

[0004] Therefore, there is an urgent need for an AI-based ultra-thin electronic endoscope visual navigation system for radiofrequency ablation to change this situation. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing an artificial intelligence-based ultra-thin electronic endoscope visual navigation system for radiofrequency ablation. Its advantages include helping to avoid accidental damage to normal tissue and reducing the risks during the treatment process.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] An AI-based ultra-thin electronic endoscopic visual navigation system for radiofrequency ablation includes an endoscopic camera system, an energy system, an illumination system, and a display system. The endoscopic camera system acquires real-time images of the human body, processes the images, and outputs the processed images and the original video images to the display terminal. The energy system provides laser energy and delivers it to the front end through an optical fiber guidewire for the doctor to perform blood vessel blockage treatment at the target location. The illumination system provides light source illumination within the field of view for image acquisition.

[0008] The endoscopic camera system mainly includes an image acquisition module and an image processing module. The image acquisition module includes a rigid endoscope body with a diameter of less than 3mm at the front end and a transmission cable at the rear end, used to acquire endoscopic images, including high-resolution endoscopic images and real-time video streams. The image processing module is connected to the image acquisition module via a cable. The image processing module includes a GPU module with high-speed computing capabilities and built-in image processing model algorithms. On the one hand, it performs normal noise reduction and conversion on the acquired images and sends them to the display end for display. On the other hand, it also processes the acquired real-time images through algorithms to identify the structural features of the images and determine the target location.

[0009] The present invention is further configured such that the image processing model algorithm is used to classify all pixels of the image and identify and segment the target region; the basic image recognition network sends the image into the network encoder module, extracts feature maps with rich semantic information through each encoding module, and then sends the high-dimensional feature maps into the network decoder. At the same time, the feature information of different depths in the encoder is passed to the decoder through each skip connection of the model to obtain the extracted image.

[0010] The present invention is further configured such that the image processing model algorithm includes a blood vessel highlight detection algorithm, which classifies the fetal endoscope image into images with or without highlights through a blood vessel highlight classification network; the image determined to have highlighted blood vessels is used as input, and a corresponding highlight mask image is generated through the blood vessel highlight detection algorithm; the highlight mask image, the fetal endoscope image, and the blood vessel image extracted according to the basic recognition method are used together as input and then input into the image recognition network for training, so that it can utilize the additional information in the blood vessel highlight mask on the basis of the preliminary extraction method.

[0011] The present invention is further configured such that the blood vessel highlight detection algorithm is an upgraded algorithm based on the Frangi filter, and the detection principle of the Frangi filter is based on the Hessian matrix; the Hessian matrix is ​​composed of the second-order partial derivatives of the gray value of each pixel in each direction. That is, for a two-dimensional image, the Hessian matrix of the image pixel P(x,y) is represented as: in, These represent the four second-order partial derivatives corresponding to the two-dimensional image.

[0012] The present invention is further configured such that the eigenvalues ​​of the Hessian matrix can describe the tubular structure to be detected, and let the smaller of the two eigenvalues ​​be... The one with the larger absolute value is ,but The corresponding feature vectors represent the intensity and direction of small changes in grayscale values ​​in the image, while The eigenvectors represent the intensity directions where grayscale values ​​change significantly. Isolated points in an image exhibit large variations in grayscale values ​​across all directions, while tubular structures show smaller variations along their axial direction and larger variations perpendicular to the tube.

[0013] The invention is further configured to use a pair of Fragi filters to detect darker and brighter tubular structures in the image, respectively; eigenvalues ​​with large absolute values ​​in the Hessian matrix. The positive or negative sign indicates whether a darker or brighter tubular structure was detected; when When < 0, it corresponds to a white structure on a black background. When the value is greater than 0, it corresponds to a black structure on a white background; in a fetal microscope, the bright blood vessels are relatively bright tubular structures, while blood vessels are generally darker, thus allowing for adjustments by setting... The positive and negative signs are initially extracted respectively.

[0014] The invention is further configured such that, for detecting brighter tubular structures, the Frangi filter constructs variables. , Then the expression of the corresponding function is: ;

[0015] For detecting darker tubular structures, the formula above will be... >0 changed to <0. Then the filter result is converted into a binary image, and the corresponding binary images of the brighter and darker tubular structures are respectively... and ,in, and To adjust the parameters, the sensitivity and intensity threshold of the tubular morphology were controlled separately.

[0016] The present invention is further configured to, wherein the... The outer edges of the image are removed. A low threshold i is set for the input fetal endoscope image, where pixels with values ​​below i are set to 0, and others to 1. Then, a dilation operation is performed on this region to obtain a visible area mask. ,Will and Multiplying pixel by pixel yields the edge removal result. Then, in order to eliminate some of the isolated points caused by impurities and noise, [the following was done]... and All are subjected to etching operations to remove isolated points. and Because bright blood vessels can create gaps in the darker blood vessel structures being detected, further... Perform a closing operation to obtain Finally, and Taking the union of the values, we obtain the mask for the highlighted blood vessel region: ;

[0017] Since the highlighted blood vessel area is generally located inside the blood vessel. This can be considered as a blood vessel region, so the result of taking the union of the two images is used as the output of the mask image of the highlighted blood vessel region;

[0018] For images that the blood vessel highlighting classification network determines to have no blood vessel highlights, a basic image recognition network is used to extract the target region.

[0019] The present invention is further configured such that the energy system includes an energy generation module, an energy control module, and an energy transmission module. The energy control module adjusts the energy power output by the energy generation module by adjusting the magnitude of the current. The energy transmission module transmits the energy to the front-end application through a laser guide wire. The energy control module has a signal connection with the camera system and achieves adaptive energy output power according to the different diameters of the blood vessels that need to be ablated by energy, as identified by the camera system.

[0020] The invention is further configured such that the lighting system is a medical cold light source system, which sends light through a beam guide to the light source connection port of the ultra-fine endoscope at the front end, and the beam guide inside the endoscope sends the light into the cavity to achieve illumination of the cavity.

[0021] The beneficial effects of this invention are as follows: This invention discloses an artificial intelligence-based ultra-thin electronic endoscope visual navigation system for radiofrequency ablation. In addition to the original output video image, it adds output navigation images, automatically identifying the patient's structural features and location, providing navigation for doctors to locate the target location and improving treatment effectiveness. The endoscope camera module and the radiofrequency energy module are interconnected, automatically adjusting the required energy level based on the size of the ablation vessel in the image, achieving automatic dosage control. The automated navigation function reduces the workload of doctors, making operation simpler and faster. Navigation helps avoid accidental damage to normal tissue, reducing the risks during treatment. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the overall structure of the artificial intelligence-based ultra-thin electronic endoscope visual navigation system for radiofrequency therapy proposed in this invention;

[0023] Figure 2 This is a schematic diagram of the front-end end structure of the artificial intelligence-based radiofrequency ablation ultra-thin electronic endoscope visual navigation system proposed in this invention.

[0024] Figure 3 for Figure 2 Enlarged structural diagram;

[0025] Figure 4 This is a schematic diagram of the algorithm flow for the AI-based ultra-thin electronic endoscope visual navigation system for radiofrequency therapy proposed in this invention.

[0026] In the diagram: 1. Lighting system access port; 2. Endoscope system wiring terminal; 3. Energy system output terminal; 4. Light output terminal; 5. Laser guide wire exit channel; 6. Front-end camera acquisition terminal. Detailed Implementation

[0027] The technical solution of this patent will be further described in detail below with reference to specific embodiments.

[0028] The embodiments of this patent are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this patent, and should not be construed as limiting this patent.

[0029] Reference Figure 1 and Figure 4 This is an AI-based ultra-thin electronic endoscopic visual navigation system for radiofrequency ablation, comprising an endoscopic camera system, an energy system, an illumination system, and a display system. The endoscopic camera system acquires real-time images of the body's interior, processes these images, and outputs both the processed images and the original video images to the display. The energy system provides laser energy and delivers it to the front end via fiber optic guidewires for the physician to perform blood vessel blockage treatment at the target location. The illumination system provides light source illumination within the field of view for image acquisition.

[0030] The endoscopic camera system mainly includes an image acquisition module and an image processing module. The image acquisition module consists of a rigid endoscope body with a diameter of less than 3mm at the front end and a transmission cable at the rear end, used to acquire endoscopic images, including high-resolution endoscopic images and real-time video streams.

[0031] The image processing module and the image acquisition module are connected by a cable. The image processing module includes a GPU module with high-speed computing power and built-in massive model algorithms. On the one hand, it performs normal noise reduction and conversion on the acquired images and sends them to the display end for display. On the other hand, it also processes the acquired real-time images through algorithms, identifies the structural features of the images, identifies the target location, and displays it on the display end, providing guidance for doctors to apply energy therapy in the future.

[0032] Image processing algorithms classify all pixels in an image and segment target regions. A basic image recognition network feeds the image into a network encoder module, extracts feature maps rich in semantic information through various encoding modules, and then sends these high-dimensional feature maps to the network decoder. Simultaneously, different depth feature information from the encoder is passed to the decoder through various skip connections in the model, resulting in the extracted image. However, in images from ultra-fine endoscopes, placental vessels generally appear darker than the surrounding background. Once the light source enters the cavity through the beam guide inside the electron microscope, the light shines on the relatively smooth vessel walls. Due to the good light reflection in smooth areas, these areas become highlighted, and the highlighted vessels are incorrectly identified as background. When the highlight is in the middle of a vessel, the model may identify one vessel as two; when the highlight is near the edge of the vessel, the model may miss the vessel edge, causing problems for subsequent treatments (commonly laser vascular occlusion). Therefore, to more accurately identify placental vessels, a vessel highlight detection algorithm is added to the basic image recognition algorithm to enhance the accuracy of vessel identification.

[0033] The process consists of three parts: First, the fetal endoscope image is classified into images with or without highlighted blood vessels using a blood vessel highlighting classification network. Then, images identified as having highlighted blood vessels are used as input, and a blood vessel highlighting detection algorithm generates corresponding highlight mask images. Finally, the highlight mask images, the fetal endoscope images, and the blood vessel images extracted using basic recognition methods are fed together as input to the image recognition network for training. This allows the network to utilize additional information from the blood vessel highlighting masks, building upon the initial extraction method, thereby improving image segmentation. The blood vessel highlighting classification network is a discrimination algorithm developed through multiple training iterations using a massive number of images with and without highlighted blood vessels. The blood vessel highlighting detection algorithm is an upgraded version of the Frangi filter.

[0034] Representative structure <![CDATA[λ1]]> <![CDATA[λ2]]> tubular structure Small big isolated point big big background Small Small Due to the poor contrast of fetal endoscopy images, significant variations in the shape and scale of blood vessels, and the fact that blood vessels are not simply dark tubular structures but may also contain highlights, this design upgrades the Frangi filter: a pair of Frangi filters are first used to detect the darker and brighter tubular structures in the image respectively. The eigenvalues ​​with large absolute values ​​in the Hessian matrix are also considered. The positive or negative sign can indicate whether a darker or brighter tubular structure was detected. When When < 0, it corresponds to a white structure on a black background. When the value is greater than 0, it corresponds to a black structure on a white background. In a fetal microscope, highlighted blood vessels appear as relatively bright tubular structures, while other blood vessels are generally darker. This can be achieved by adjusting the settings. The positive and negative values ​​are initially extracted separately. For detecting brighter tubular structures, variables are constructed. , Then the expression of the corresponding function is: For detecting darker tubular structures, the formula above will be... >0 changed to <0. Then the filter result is converted into a binary image, and the corresponding binary images of the brighter and darker tubular structures are respectively... and ,in, and To adjust the parameters, the sensitivity and intensity threshold of the tubular morphology were controlled separately, and then... The outer edges of the image are removed. A low threshold i is set for the input fetal endoscope image, where pixels with values ​​below i are set to 0, and others to 1. Then, a dilation operation is performed on this region to obtain a visible area mask. ,Will and Multiplying pixel by pixel yields the edge removal result. Then, in order to eliminate some of the isolated points caused by impurities and noise, [the following was done]... and All are subjected to etching operations to remove isolated points. and Because bright blood vessels can create gaps in the darker blood vessel structures being detected, further... A closing operation is performed to remove these gaps while avoiding increasing the extent of the vascular region, resulting in... Finally, and Taking the union of the values, we obtain the mask for the highlighted blood vessel region: Highlighted blood vessels are generally located inside the blood vessels. This can be considered as a blood vessel region, so the result of taking the union of the two images is output as a mask image of the highlighted blood vessel region.

[0035] For images that the blood vessel highlighting classification network determines to have no blood vessel highlights, a basic image recognition network is used to extract the target region.

[0036] The camera system transmits the navigation image and the original video image together to the display terminal via a data cable, allowing medical staff to see real-time video information and real-time extracted navigation information.

[0037] The energy system comprises an energy generation module, an energy control module, and an energy transmission module. The energy control module adjusts the output power of the energy generation module by regulating the current, while the energy transmission module transmits the energy to the front-end application via a laser guidewire. The energy control module is signal-connected to the camera system, allowing for adaptive energy output power based on the diameter of the blood vessel requiring energy ablation, as identified by the camera system. The illumination system is a medical cold light source system. This system transmits light through a beam guide to the light source connection port of the ultra-fine endoscope at the front end. The beam guide inside the endoscope then directs the light into the cavity, illuminating the interior of the cavity.

[0038] A display system, or monitor, is used for the real-time display of video and recognition images.

[0039] Reference Figure 2 and Figure 3 The front end of the endoscope consists of an illumination system input port 1, an endoscope system wiring terminal 2, an energy system output terminal 3, a light output terminal 4, a laser guidewire exit channel 5, and a front-end camera acquisition terminal 6. During surgery, the lens tip is inserted into the human cavity. The light source enters from the illumination system input terminal 1 and exits from the light output terminal 4. The light is transmitted to the front end through the optical fiber inside the endoscope to illuminate the cavity. The laser guidewire enters from the energy system output terminal 3 and exits from the laser guidewire exit channel 5. High-energy laser energy is emitted from the laser guidewire and irradiates the blood vessel, coagulating the blood vessel through the photothermal effect. The camera acquisition terminal collects information from the lens end and sends it to the camera system for processing via a rear cable.

[0040] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An AI-based ultra-thin electronic endoscope visual navigation system for radiofrequency ablation, characterized in that, The device includes an endoscopic camera system, an energy system, an illumination system, and a display system. The endoscopic camera system acquires real-time images of the human body, processes the images, and outputs the processed images and the original video images to the display terminal. The energy system provides laser energy and delivers it to the front end through an optical fiber guidewire for the doctor to perform blood vessel blockage treatment at the target location. The illumination system provides light source illumination within the field of view for image acquisition. The endoscopic camera system mainly includes an image acquisition module and an image processing module. The image acquisition module includes a rigid endoscope body with a diameter of less than 3mm at the front end and a transmission cable at the rear end, used to acquire endoscopic images, including high-resolution endoscopic images and real-time video streams. The image processing module is connected to the image acquisition module via a cable. The image processing module includes a GPU module with high-speed computing capabilities and built-in image processing model algorithms. On the one hand, it performs normal noise reduction and conversion on the acquired images and sends them to the display end for display. On the other hand, it also processes the acquired real-time images through algorithms to identify the structural features of the images and determine the target location. The image processing model algorithm is used to classify all pixels in an image and identify and segment target regions. The basic image recognition network feeds the image into the network encoder module, extracts feature maps with rich semantic information through various encoding modules, and then feeds the high-dimensional feature maps into the network decoder. At the same time, the feature information of different depths in the encoder is passed to the decoder through various skip connections of the model to obtain the extracted image. The image processing model algorithm includes a blood vessel highlighting detection algorithm. This algorithm classifies the fetal endoscope image into images with or without highlights using a blood vessel highlighting classification network. The image identified as having highlighted blood vessels is used as input, and the blood vessel highlighting detection algorithm generates a corresponding highlighting mask image. The highlighting mask image, the fetal endoscope image, and the blood vessel image extracted using the basic recognition method are then input again into the image recognition network for training, enabling it to utilize additional information from the blood vessel highlighting mask based on the initial extraction method. The proposed blood vessel highlight detection algorithm is an upgraded version of the Frangi filter. The detection principle of the Frangi filter is based on the Hessian matrix. The Hessian matrix is ​​composed of the second-order partial derivatives of the gray values ​​of each pixel in each direction. That is, for a two-dimensional image, the Hessian matrix of the image pixel P(x,y) is represented as: in, These represent the four second-order partial derivatives corresponding to the two-dimensional image; The eigenvalues ​​of the Hessian matrix can describe the tubular structure to be detected. Let the smaller of the two eigenvalues ​​be denoted as . The one with the larger absolute value is ,but The corresponding feature vectors represent the intensity and direction of small changes in grayscale values ​​in the image, while The eigenvectors represent the intensity directions where grayscale values ​​change significantly; isolated points in the image exhibit large variations in grayscale values ​​in all directions, while tubular structures show smaller variations along their axial direction and larger variations perpendicular to the tube; a pair of Fragi filters are used to detect darker and brighter tubular structures in the image, respectively; eigenvalues ​​with large absolute values ​​in the Hessian matrix... The positive or negative sign indicates whether a darker or brighter tubular structure was detected; when When < 0, it corresponds to a white structure on a black background. When the value is greater than 0, it corresponds to a black structure on a white background; in a fetal microscope, the bright blood vessels are relatively bright tubular structures, while blood vessels are generally darker, thus allowing for adjustments by setting... The positive and negative signs are initially extracted respectively; The Frangi filter constructs variables for detecting brighter tubular structures. , Then the expression of the corresponding function is: For detecting darker tubular structures, the formula above will be... >0 changed to <0; then the filter result is converted into a binary image, let the corresponding binary images of the brighter and darker tubular structures be respectively and ,in, and To adjust the parameters, the sensitivity and intensity threshold of the tubular morphology are controlled separately; The The outer edges of the image are removed. A low threshold i is set for the input fetal endoscope image, where pixels with values ​​below i are set to 0, and others to 1. Then, a dilation operation is performed on this region to obtain a visible area mask. ,Will and Multiplying pixel by pixel yields the edge removal result. Then, in order to eliminate some of the isolated points caused by impurities and noise, [the following was done]... and All are subjected to etching operations to remove isolated points. and Because bright blood vessels can create gaps in the darker blood vessel structures being detected, further... Perform a closing operation to obtain Finally, and Taking the union of the values, we obtain the mask for the highlighted blood vessel region: Since the highlighted blood vessel area is generally located inside the blood vessel. This can be viewed as a blood vessel region, so the result of taking the union of the two images is output as a mask image of the highlighted blood vessel region; For images that the blood vessel highlighting classification network determines to have no blood vessel highlights, a basic image recognition network is used to extract the target region.

2. The artificial intelligence-based ultra-thin electronic endoscope visual navigation system for radiofrequency ablation according to claim 1, characterized in that, The energy system includes an energy generation module, an energy control module, and an energy transmission module. The energy control module adjusts the energy output power of the energy generation module by regulating the current. The energy transmission module transmits the energy to the front-end application through a laser guidewire. The energy control module is connected to the camera system and adapts the energy output power according to the different diameters of the blood vessels that need to be ablated by the camera system.

3. The artificial intelligence-based ultra-thin electronic endoscope visual navigation system for radiofrequency ablation according to claim 2, characterized in that, The lighting system is a medical cold light source system. The cold light source system sends light through a beam guide to the light source connection port of the ultra-fine endoscope at the front end. The beam guide inside the endoscope sends the light into the cavity, thereby illuminating the inside of the cavity.

Images