Radio frequency treatment superfine electronic endoscope visual navigation system based on artificial intelligence

Abstract

The invention discloses a radio frequency treatment superfine electronic endoscope visual navigation system based on artificial intelligence, and relates to the technical field of medical instruments, the radio frequency treatment superfine electronic endoscope visual navigation system comprises an endoscope camera system, an energy system, an illumination system and a display system, outputting the processed image and the original video image to a display end for displaying; the energy system provides laser energy and transmits the energy to the front end through an optical fiber guide wire so that a doctor can conduct blood vessel blocking treatment on a target position. And the illumination system provides light source illumination in a visual field for image acquisition. The camera shooting module and the radio frequency energy module are interconnected, the required energy is automatically adjusted according to the size of an ablation blood vessel in an image, and automatic control over the feeding amount is achieved; the automatic navigation function reduces the workload of doctors, so that the operation is simpler, more convenient and quicker; navigation is beneficial to avoiding accidental injury of normal tissues, and risks in the treatment process are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of medical equipment, and in particular to an artificial intelligence-based radio frequency treatment ultra-thin electronic endoscope visual navigation system. Background Art

[0002] With the advancement of medical technology, endoscopic treatment has become an important means of disease diagnosis and treatment. However, precise navigation in the complex human body structure is still a challenge, especially when fetoscopic radiofrequency treatment is performed, doctors need to accurately locate the lesion and implement precise treatment.

[0003] In current ultra-fine electronic endoscope treatment, the location of the target and the amount of radiofrequency energy administered depend on the doctor's experience and skills. There are differences between different doctors or different operations performed by the same doctor, resulting in unstable surgical results or efficacy.

[0004] Therefore, an artificial intelligence-based ultra-fine electronic endoscope visual navigation system for radiofrequency therapy is urgently needed to change this situation. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and propose an artificial intelligence-based radiofrequency treatment ultra-fine electronic endoscope visual navigation system. Its advantage is that it helps to avoid accidental injury to normal tissues and reduces the risk during the treatment process.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The artificial intelligence-based radiofrequency treatment ultra-fine electronic endoscope visual navigation system includes an endoscope camera system, an energy system, a lighting system and a display system. The endoscope camera system collects images of the human body in real time and processes the images, and outputs the processed images and the original video images to the display end for display; the energy system provides laser energy and transmits the energy to the front end through the optical fiber guide wire for the doctor to perform vascular occlusion treatment on the target position; the lighting system provides light source illumination within the field of view for image acquisition;

[0008] The endoscopic camera system mainly includes an image acquisition module and an image processing module. The image acquisition module includes a hard lens with a diameter of less than 3 mm at the front end and a transmission cable at the rear end, which is used to obtain endoscopic images, including high-resolution endoscopic images and real-time video streams; the image processing module and the image acquisition module are connected by a cable, and the image processing module includes a GPU module with high-speed computing capabilities and a built-in image processing model algorithm. On the one hand, the collected pictures are subjected to normal noise reduction and conversion and sent to the display end for display; on the other hand, the collected real-time images are processed by algorithms to identify the structural features of the images and distinguish the target positions.

[0009] The present invention is further configured such that the image processing model algorithm is used to classify all pixels of the image and identify and segment the target area; the basic image recognition network is to send the image into the network encoder module, extract a feature map with rich semantic information through each encoding module, and then send the high-dimensional feature map into the network decoder, and at the same time pass the different depth feature information in the encoder to the decoder through each jump connection of the model to obtain the extracted image.

[0010] The present invention is further configured such that the image processing model algorithm includes a vascular highlight detection algorithm, which classifies the fetoscopic image into an image with highlight or an image without highlight through a vascular highlight classification network; uses the image judged to have highlighted blood vessels as input, and generates a corresponding highlight mask image through the vascular highlight detection algorithm; and inputs the highlight mask image, the fetoscopic image, and the vascular image extracted according to the basic recognition method together as input into the image recognition network again for training, so that it can utilize the additional information in the vascular highlight mask on the basis of the preliminary extraction method.

[0011] The present invention is further configured that the blood vessel highlight detection algorithm is an upgraded algorithm based on the Franging filter, and the detection principle of the Franging filter is based on the Hessian matrix; the Hessian matrix is ​​composed of the second-order partial derivatives of each pixel gray value in each direction. That is, for a two-dimensional image, the Hessian matrix of the image pixel point P (x, y) is expressed as:

[0012]

[0013] Among them, represent the four second-order partial derivatives corresponding to the two-dimensional image.

[0014] The present invention is further configured such that the eigenvalues ​​of the Hessian matrix can describe the tubular structure to be detected, and the two eigenvalues ​​with smaller absolute values ​​are λ1 and the two eigenvalues ​​with larger absolute values ​​are λ2, then λ1 and its corresponding eigenvector represent the intensity and direction of the grayscale value change in the image, while λ2 and its eigenvector represent the intensity direction of the grayscale value change. The grayscale values ​​of isolated points in each direction in the image have a large change, while the tubular structure has a small change in the axial direction and a large change in the direction perpendicular to the tubular shape.

[0015] The present invention is further configured to use a pair of the Franging filters to detect the darker tubular structure and the brighter tubular structure in the image respectively; the positive or negative value of the eigenvalue λ2 with a larger absolute value in the Hessian matrix indicates whether the detected one is a darker tubular structure or a brighter tubular structure; when λ2<0, it corresponds to a white structure on a black background, and when λ2>0, it corresponds to a black structure on a white background; the highlighted blood vessels in the fetoscope are relatively brighter tubular structures, while the blood vessels are generally darker, so that they are preliminarily extracted respectively by setting the positive or negative value of λ2.

[0016] The present invention is further configured such that, for the case of detecting a brighter tubular structure, the Franging filter constructs a variable R b =λ2 / λ1, Then the expression of the corresponding function is:

[0017]

[0018] For the case of detecting darker tubular structures, λ2>0 in the above formula is changed to λ2<0. Then the filter result is converted into a binary image, and the corresponding binary images of the brighter and darker tubular structures are L1 and D1 respectively.

[0019] The present invention is further configured to remove the outer edge in D1, set a lower threshold i for the input fetoscopic image, set the points with pixel values ​​lower than i in the image to 0, and the others to 1, and then perform an expansion operation on this area to obtain a visible area mask I V , will I V Multiplying L1 pixel by pixel with D1 can obtain the edge removal result D2; then, in order to eliminate some isolated points caused by impurity tissue and noise, corrosion operations are performed on L1 and D2 to obtain L2 and D3 with isolated points removed; since the highlighted blood vessels will cause gaps in the middle of the darker blood vessel structure detected, a closing operation is performed on D3 to obtain D4; finally, the union of L2 and D4 is taken to obtain the highlighted blood vessel area mask:

[0020] I h =L2∪D4;

[0021] Since the highlighted blood vessel area is generally inside the blood vessel, D4 can be regarded as the blood vessel area, so the result of taking the union of the two images is used as the output mask image of the highlighted blood vessel area;

[0022] For images that are judged as having no vascular highlight by the vascular highlight classification network, a basic image recognition network is used to extract the target area.

[0023] The present invention is further configured as follows: the energy system includes an energy generation module, an energy control module and an energy transmission module. The energy control module adjusts the energy power output by the energy generation module by adjusting the magnitude of the current, and the energy transmission module transmits the energy to the front-end application through a laser guide wire; the energy control module has a signal connection with the camera system, and the energy output power is adapted according to the different diameters of the blood vessels that need to be energy-fusing as identified by the camera system.

[0024] The present invention is further configured such that the lighting system is a medical cold light source system, which sends light to the light source connection port of the ultra-fine endoscope at the front end through a light guide beam, and the light guide beam inside the endoscope sends light to the inside of the cavity, thereby realizing illumination of the inside of the cavity.

[0025] The beneficial effects of the present invention are as follows: the present invention discloses an artificial intelligence-based ultra-fine electronic endoscope visual navigation system for radio frequency treatment, which adds an output navigation picture on the basis of the output original video picture, automatically identifies the patient's structural characteristics and position, provides navigation for doctors to find the target position, and improves the treatment effect; the endoscope camera module and the radio frequency energy module are interconnected, and the required energy size is automatically adjusted according to the size of the ablated blood vessel in the image, thereby realizing automatic control of the dosage; the automated navigation function reduces the doctor's workload and makes the operation simpler and faster; navigation helps to avoid accidental injury to normal tissue and reduces the risk during the treatment process. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a schematic diagram of the overall structure of the artificial intelligence-based ultra-fine electronic endoscope visual navigation system for radiofrequency treatment proposed by the present invention;

[0027] Figure 2 This is a schematic diagram of the structure of the front-end end of the ultra-fine electronic endoscope visual navigation system for radiofrequency treatment based on artificial intelligence proposed by the present invention;

[0028] Figure 3 for Figure 2 The enlarged structural diagram is shown in the figure;

[0029] Figure 4 This is a schematic diagram of the algorithm flow of the artificial intelligence-based ultra-fine electronic endoscope visual navigation system for radiofrequency treatment proposed in the present invention.

[0030] In the figure: 1. Lighting system access port; 2. Endoscope system connection terminal; 3. Energy system output terminal; 4. Light output terminal; 5. Laser guide wire outlet channel; 6. Front-end camera acquisition terminal. DETAILED DESCRIPTION

[0031] The technical solution of this patent is further described in detail below in conjunction with specific implementation methods.

[0032] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0033] Reference Figure 1 and Figure 4 , an artificial intelligence-based radiofrequency treatment ultra-fine electronic endoscope visual navigation system, an endoscope camera system, an energy system, a lighting system and a display system. The endoscope camera system collects images of the human body in real time and processes the images, and outputs the processed images and the original video images to the display end for display; the energy system provides laser energy and transmits the energy to the front end through the optical fiber guide wire for the doctor to perform vascular occlusion treatment at the target location; the lighting system provides light source illumination within the field of view for image acquisition.

[0034] The endoscopic camera system mainly includes an image acquisition module and an image processing module. The image acquisition module includes a hard mirror with a diameter of less than 3mm at the front end and a transmission cable at the back end, which is used to obtain endoscopic images, including high-resolution endoscopic images and real-time video streams.

[0035] The image processing module and the image acquisition module are connected by cables. The image processing module includes a GPU module with high-speed computing capabilities and a built-in massive model algorithm. On the one hand, it performs normal noise reduction and conversion on the acquired images and sends them to the display end for display; on the other hand, it also processes the acquired real-time images through algorithms, identifies the structural features of the images, distinguishes the target location, and displays it on the display end, providing navigation for doctors to subsequently apply energy treatment.

[0036] The algorithmic processing of the image is to classify all the pixels of the image and identify the target area for segmentation. The basic image recognition network is to send the image into the network encoder module, extract the feature map with rich semantic information through each encoding module, and then send the high-dimensional feature map to the network decoder. At the same time, the feature information of different depths in the encoder is passed to the decoder through each jump connection of the model to obtain the extracted image. However, in the image of the ultra-fine endoscope, the placental blood vessels generally appear darker than the surrounding background. Once the light source enters the cavity through the light guide inside the electronic mirror, the light shines on the relatively smooth blood vessel wall. Since the smooth area has a better reflection effect on the light, the image of the smooth area will appear highlighted. At this time, the highlighted blood vessels will be mistakenly identified as the background. When the highlight position is in the middle of a blood vessel, the model may identify one blood vessel as two blood vessels; when the highlight position is close to the edge of the blood vessel, the model recognition may cause the loss of the blood vessel edge, which will cause trouble for subsequent treatment (commonly laser blocking of blood vessels). Therefore, in order to more accurately identify the placental blood vessels, a blood vessel highlight detection algorithm is added to the basic image recognition algorithm to enhance the accuracy of blood vessel recognition.

[0037] It is divided into three parts: first, the fetoscopic image is classified into an image with highlight or an image without highlight through the vascular highlight classification network; the image judged to have highlighted blood vessels is used as input, and the corresponding highlight mask image is generated through the vascular highlight detection algorithm; the highlight mask image, the fetoscopic image and the vascular image extracted according to the basic recognition method are input again into the image recognition network for training, so that it can use the additional information in the vascular highlight mask on the basis of the preliminary extraction method, thereby improving the image segmentation effect. Among them, the formation of the vascular highlight classification network is a discrimination algorithm formed by multiple training models of massive vascular images with and without highlight. The vascular highlight detection algorithm is an upgraded algorithm based on the Franging filter.

[0038] The detection principle of the Frangi filter is based on the Hessian matrix. The Hessian matrix is ​​composed of the second-order partial derivatives of each pixel gray value in each direction. That is, for a two-dimensional image, the Hessian matrix of the image pixel point P(x,y) is expressed as:

[0039]

[0040] where f xx ,f xy ,f yx ,f yyRespectively represent the four second-order partial derivatives corresponding to the two-dimensional image. The eigenvalues ​​of the Hessian matrix can describe the tubular structure that needs to be detected. Let the two eigenvalues ​​with smaller absolute values ​​be λ1 and the larger absolute value be λ2. Then λ1 and its corresponding eigenvector represent the intensity and direction of the grayscale value change in the image, while λ2 and its eigenvector represent the intensity direction of the grayscale value change. The grayscale values ​​of isolated points in the image have large changes in all directions, while the tubular structure has a small change in the axial direction and a large change in the direction perpendicular to the tubular. Therefore, the eigenvalues ​​of the Hessian matrix can be used to detect the tubular structure in the image. The relationship between the size of the eigenvalue and the tubular structure in the image is as follows:

[0041]

[0042]

[0043] Due to the poor contrast of fetoscopic images, the great variation in the shape and scale of blood vessels in the images, and the fact that blood vessels are not single dark tubular structures but may have highlights. This design scheme has upgraded the Frangi filter: first, a pair of Frangi filters are set up to detect the darker tubular structures and the brighter tubular structures in the image respectively. The positive or negative sign of the eigenvalue λ2 with a larger absolute value in the Hessian matrix can indicate whether a darker tubular structure or a brighter tubular structure is detected. When λ2<0, it corresponds to a white structure on a black background, and when λ2>0, it corresponds to a black structure on a white background. The highlighted blood vessels in the fetoscopic image are relatively bright tubular structures, while blood vessels are generally darker, so they can be preliminarily extracted separately by setting the positive or negative value of λ2. For the case of detecting brighter tubular structures, the variable R is constructed. b =λ2 / λ1, Then the expression of the corresponding function is:

[0044]

[0045] For the detection of darker tubular structures, change λ2>0 in the above formula to λ2<0. Then convert the filter result into a binary image, and assume that the corresponding binary images of the brighter and darker tubular structures are L1 and D1 respectively. Then remove the outer edge in D1, set a lower threshold i for the input fetoscopic image, set the points with pixel values ​​lower than i in the image to 0, and the others to 1, and then dilate this area to obtain the visible area mask I V , will I VMultiplying pixel by pixel with D1, we can get the edge removal result D2. Then, in order to eliminate some isolated points caused by impurity tissue and noise, corrosion operations are performed on L1 and D2 to obtain L2 and D3 without isolated points. Since the highlighted blood vessels will cause gaps in the middle of the darker blood vessel structure detected, a closing operation is performed on D3 to remove these gaps while avoiding increasing the blood vessel area range, and D4 is obtained. Finally, L2 and D4 are taken as the union to obtain the highlighted blood vessel area mask:

[0046] I h =L2∪D4

[0047] Since the highlighted blood vessel area is generally inside the blood vessel, D4 can be regarded as the blood vessel area, so the result of taking the union of the two images is used as the output blood vessel highlight area mask image.

[0048] For images that are judged as having no vascular highlight by the vascular highlight classification network, a basic image recognition network is used to extract the target area.

[0049] The camera system sends the navigation image and the original video image to the display end through the data cable for display, so that medical staff can see the real-time video information and the real-time extracted navigation information.

[0050] The energy system includes an energy generation module, an energy control module and an energy transmission module. The energy control module adjusts the energy power output by the energy generation module by adjusting the current, and the energy transmission module transmits the energy to the front-end application through the laser guide wire. The energy control module has a signal connection with the camera system, and the energy output power is adaptively adjusted according to the different diameters of the blood vessels that need to be energy-fusing identified by the camera system.

[0051] The lighting system is a medical cold light source system. The cold light source system sends light to the light source connection port of the ultra-fine endoscope at the front end through a light guide. The light guide inside the endoscope sends light to the inside of the cavity to achieve lighting inside the cavity.

[0052] The display system is a display that is used for real-time display of video images and recognition images.

[0053] Reference Figure 2 and Figure 3The front end of the scope is composed of an illumination system access port 1, an endoscope system terminal 2, an energy system output terminal 3, a light output terminal 4, a laser guide wire outlet channel 5 and a front end camera acquisition terminal 6; when used for surgery, the front end of the lens is extended into the human body cavity, the light source enters from the illumination system access terminal 1, and is output from the light output terminal 4. The light is transmitted to the front end through the optical fiber inside the scope to realize the illumination of the cavity interior; the laser guide wire enters from the energy system output terminal 3, and exits from the laser guide wire outlet channel 5. High-energy laser energy is emitted from the laser guide wire and irradiates the blood vessel, and the blood vessel is coagulated by the thermal effect of light; the camera acquisition terminal sends the information collected at the lens end to the camera system for processing through the back-end cable.

[0054] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. The artificial intelligence-based ultra-fine electronic endoscope visual navigation system for radiofrequency treatment is characterized by: It includes an endoscopic camera system, an energy system, a lighting system and a display system. The endoscopic camera system collects images of the human body in real time and processes the images, and outputs the processed images and the original video images to the display end for display; the energy system provides laser energy and transmits the energy to the front end through the optical fiber guide wire for the doctor to perform vascular occlusion treatment on the target position; the lighting system provides light source illumination within the field of view for image acquisition; The endoscopic camera system mainly includes an image acquisition module and an image processing module. The image acquisition module includes a hard lens with a diameter of less than 3 mm at the front end and a transmission cable at the rear end, which is used to obtain endoscopic images, including high-resolution endoscopic images and real-time video streams; the image processing module and the image acquisition module are connected by a cable, and the image processing module includes a GPU module with high-speed computing capabilities and a built-in image processing model algorithm. On the one hand, the collected pictures are subjected to normal noise reduction and conversion and sent to the display end for display; on the other hand, the collected real-time images are processed by algorithms to identify the structural features of the images and distinguish the target positions.

2. The artificial intelligence-based radiofrequency treatment ultra-fine electronic endoscope visual navigation system according to claim 1 is characterized in that: The image processing model algorithm is used to classify all pixels of the image and identify the segmentation target area; the basic image recognition network is to send the image into the network encoder module, extract the feature map with rich semantic information through each encoding module, and then send the high-dimensional feature map into the network decoder, and at the same time pass the different depth feature information in the encoder to the decoder through each jump connection of the model to obtain the extracted image.

3. The artificial intelligence-based radiofrequency treatment ultra-fine electronic endoscope visual navigation system according to claim 2 is characterized in that: The image processing model algorithm includes a blood vessel highlight detection algorithm, which classifies the fetoscopic image into an image with highlight or an image without highlight through a blood vessel highlight classification network; uses the image judged to have highlighted blood vessels as input, and generates a corresponding highlight mask image through the blood vessel highlight detection algorithm; and inputs the highlight mask image, the fetoscopic image and the blood vessel image extracted according to the basic recognition method as input to the image recognition network again for training, so that it can use the additional information in the blood vessel highlight mask on the basis of the preliminary extraction method.

4. The artificial intelligence-based radiofrequency treatment ultra-fine electronic endoscope visual navigation system according to claim 3 is characterized in that: The blood vessel highlight detection algorithm is an upgraded algorithm based on the Franging filter. The detection principle of the Franging filter is based on the Hessian matrix; the Hessian matrix is ​​composed of the second-order partial derivatives of each pixel gray value in each direction. That is, for a two-dimensional image, the Hessian matrix of the image pixel point P(x,y) is expressed as: Among them, represent the four second-order partial derivatives corresponding to the two-dimensional image.

5. The artificial intelligence-based radiofrequency treatment ultra-fine electronic endoscope visual navigation system according to claim 4 is characterized in that: The eigenvalues ​​of the Hessian matrix can describe the tubular structure to be detected. Let the absolute value of the two eigenvalues ​​be λ1, and the absolute value be λ2. Then λ1 and its corresponding eigenvector represent the intensity and direction of the gray value change in the image, while λ2 and its eigenvector represent the intensity direction of the gray value change. The gray values ​​of isolated points in each direction in the image have a large change, while the tubular structure has a small change in the axial direction and a large change in the direction perpendicular to the tubular.

6. The artificial intelligence-based radiofrequency treatment ultra-fine electronic endoscope visual navigation system according to claim 5 is characterized in that: A pair of the Franging filters are used to detect the darker tubular structure and the brighter tubular structure in the image respectively; the positive or negative value of the eigenvalue λ2 with a larger absolute value in the Hessian matrix indicates whether the detected tubular structure is a darker tubular structure or a brighter tubular structure; when λ2<0, it corresponds to a white structure on a black background, and when λ2>0, it corresponds to a black structure on a white background; the highlighted blood vessels in the fetoscope are relatively brighter tubular structures, while blood vessels are generally darker, so they are preliminarily extracted by setting the positive or negative value of λ2.

7. The artificial intelligence-based radiofrequency treatment ultra-thin electronic endoscope visual navigation system according to claim 6 is characterized in that: The Franging filter constructs the variable R for detecting brighter tubular structures. b =λ2 / λ1, Then the expression of the corresponding function is: For the case of detecting darker tubular structures, λ2>0 in the above formula is changed to λ2<0. Then the filter result is converted into a binary image, and the corresponding binary images of the brighter and darker tubular structures are L1 and D1 respectively.

8. The artificial intelligence-based radiofrequency treatment ultra-thin electronic endoscope visual navigation system according to claim 7 is characterized in that: The outer edge of D1 is removed, a lower threshold i is set for the input fetoscopic image, and the points with pixel values ​​lower than i in the image are set to 0, and the others are set to 1. Then, this area is expanded to obtain the visible area mask I V , will I V Multiplying L1 pixel by pixel with D1 can obtain the edge removal result D2; then, in order to eliminate some isolated points caused by impurity tissue and noise, corrosion operations are performed on L1 and D2 to obtain L2 and D3 with isolated points removed; since the highlighted blood vessels will cause gaps in the middle of the darker blood vessel structure detected, a closing operation is performed on D3 to obtain D4; finally, the union of L2 and D4 is taken to obtain the highlighted blood vessel area mask: I h =L2∪D4; Since the highlighted blood vessel area is generally inside the blood vessel, D4 can be regarded as the blood vessel area, so the result of taking the union of the two images is used as the output mask image of the highlighted blood vessel area; For images that are judged as having no vascular highlight by the vascular highlight classification network, a basic image recognition network is used to extract the target area.

9. The artificial intelligence-based radiofrequency treatment ultra-thin electronic endoscope visual navigation system according to claim 1 is characterized in that: The energy system includes an energy generation module, an energy control module and an energy transmission module. The energy control module adjusts the energy power output by the energy generation module by adjusting the magnitude of the current, and the energy transmission module transmits the energy to the front-end application through a laser guide wire. The energy control module has a signal connection with the camera system, and the energy output power is adaptively adapted according to the different diameters of the blood vessels that need to be energy-fusing identified by the camera system.

10. The artificial intelligence-based radiofrequency treatment ultra-thin electronic endoscope visual navigation system according to claim 9 is characterized in that: The lighting system is a medical cold light source system, which sends light to the light source connection port of the ultra-fine endoscope at the front end through a light guide, and the light guide inside the endoscope sends light to the inside of the cavity to realize illumination of the inside of the cavity.

Images