Appendix flushing endoscope system and control method thereof

Abstract

The invention provides an appendix flushing endoscope system, which comprises a balloon airway, a balloon, a flushing pipeline, a two-in-one pipeline connector and a processor, and can automatically identify the position and size of coprophilith in an image shot by an endoscope through image acquisition and identification, control the balloon pressurization variable based on identification data, and improve the operation efficiency and effect.

Description

Technical Field

[0001] The present invention relates to the fields of medicine and artificial intelligence, and particularly to an appendix flushing endoscope system and a control method thereof. Background Art

[0002] Acute appendicitis (AA) is a common acute abdominal disease clinically, characterized by sudden onset and rapid progression of the condition. If not diagnosed and treated in time, it is likely to cause serious consequences such as diffuse peritonitis and acute suppurative perforated appendicitis, endangering the life safety of patients. Blockage of the appendix lumen by appendicolith is one of the common causes of acute appendicitis. Existing technologies usually use saline flushing to remove the appendicolith from the appendix lumen, but this method cannot remove larger appendicoliths. When removing large appendicoliths endoscopically, the removal tool may damage the appendix lumen. Therefore, there is an urgent need for a means to safely remove appendicoliths endoscopically. Moreover, for appendicoliths of different sizes, how to use more appropriate, fast, and safe means to remove them is also an urgent problem to be solved.

[0003] In this process, identifying the appendicolith and measuring its size are also problems that need to be urgently solved. There is no means in the existing technology to automatically identify and accurately measure appendicoliths. Although there are methods for identifying and measuring lesions endoscopically in the field of medical devices, they are not applicable to the identification and measurement of appendicoliths. This is because appendicoliths are non-tissue products, which are free from tissues, often roll, and thus the morphology of appendicoliths in the narrow endoscopic field of view is uncertain, which is completely different from tissue lesions. Therefore, how to accurately identify and measure appendicoliths endoscopically is also an urgent problem to be solved.

[0004] For small-sized appendicoliths, using tools to remove them will cause unnecessary harm to patients; for large-sized appendicoliths, the selection of tool size also needs to be considered. Therefore, how to accurately judge the size of appendicoliths so as to select a safe and convenient removal method is an urgent problem to be solved. Summary of the Invention

[0005] To solve the problems, the present invention provides an appendix flushing endoscope system and a control method thereof, which can automatically identify the position and size of the appendicolith in the image captured by the endoscope through image acquisition and recognition, and control the variable balloon pressurization based on the recognition data, improving the surgical efficiency and effect.

[0006] An appendix flushing endoscope system includes a balloon airway, a balloon, a flushing pipeline, and a two-in-one pipeline connector;

[0007] One end of the balloon airway is provided with an air path connection valve, and the other end is connected to the balloon through the two-in-one pipeline connector;

[0008] One end of the flushing pipe is connected to a three-way valve, and the other end is connected to the appendix cavity flushing port through a two-in-one pipe connector;

[0009] The two-in-one pipe connector enables the flushing pipe to enter the interior of the balloon airway;

[0010] The gas path connection valve is connected to an external controllable gas source, and the three-way valve can be respectively connected to a physiological saline pump and a vacuum pump system;

[0011] The distal end of the entire system is integrally arranged in the outer sheath and is sent into the body through the outer sheath; the balloon and the appendix cavity flushing port can freely extend from the top of the outer sheath;

[0012] The balloon can be inflated through the balloon airway, thus expanding into a mushroom head shape to grab the fecalith.

[0013] The gas path connection valve provides an appropriate air pressure for the airbag to expand to a predetermined size.

[0014] The three-way valve supplies physiological saline for flushing to the flushing pipe and can suck back the sewage from the appendix cavity flushing port through the vacuum effect.

[0015] The distal pipe forms a double-layer form.

[0016] The outer layer of the pipe is the balloon airway and the inner layer is the flushing pipe.

[0017] In the distal flushing pipe, it passes through the center of the balloon and is connected to the appendix cavity flushing port.

[0018] By pulling the balloon airway, the mushroom head-shaped balloon that has expanded outside can be pulled back into the outer sheath, and during this process, larger fecaliths can be pulled back into the outer sheath by the pulling method.

[0019] The outer sheath is equipped with an endoscope.

[0020] A method for controlling the system, the processor analyzes the size and position of the fecalith according to the image transmitted back by the endoscope: for smaller fecaliths, the processor controls the three-way valve to open the physiological saline flushing and vacuum suction modes; for still larger fecaliths, the balloon pressurized expansion mode is used.

[0021] The processing method of the processor also includes:

[0022] (1) Using an adjacent multi-frame image optimization preprocessing method to divide the original input image data into valid frames and invalid frames;

[0023] (2) Constructing a neural network model to obtain the key image regions from the valid frames;

[0024] (3) Judging the fecalith cleaning mode according to the size of the region where the fecalith is located in the valid frames and controlling the balloon pressurization variable.

[0025] The invention points and technical effects of the present invention:

[0026] 1. By combining two methods of flushing vacuum and pressurizing balloon for removing fecaliths, a convenient, fast and safe cleaning method for fecaliths of different sizes is provided, improving the diagnosis and treatment efficiency and effect of appendiceal lesions.

[0027] 2. Through image acquisition and recognition, the position and size of fecaliths in the images taken by the endoscope are automatically recognized, and the balloon pressurization variable is controlled based on the recognition data, improving the surgical efficiency and effect. In particular, based on the adjacent frame hypothesis, the original input image data is divided into valid frames and non-valid frames, thereby obtaining stable image data with high signal-to-noise ratio and improving the recognition effect of image targets; a lighter computational model based on a single-layer three-dimensional convolutional kernel improves the computational efficiency of the existing method, enabling the detection of regions of interest on an embedded front-end edge platform and improving the usability of the device.

[0028] 3. Based on the camera model, the spatial mapping relationship between the image and the reality is calculated, and further, the balloon pressurization amount is inferred according to the image size of the fecalith, achieving the goal of automatic control. By automatically controlling the balloon pressure, the surgical efficiency and effect are improved. Description of the Drawings

[0029] Figure 1 They are the overall and partial structural design drawings of this design;

[0030] Figure 2 They are the partial enlarged drawings of Area A;

[0031] Figure 3 They are the schematic structural drawings of the balloon in the contracted state in Area B;

[0032] Figure 4 They are the schematic structural drawings of the balloon in the expanded state;

[0033] Among them: 1. Balloon airway; 2. Balloon; 3. Flushing pipeline; 4. Pipeline connector; 5. Three-way valve; 6. Gas path connection valve; 7. Appendiceal cavity flushing port. Detailed Implementation Modes

[0034] The purpose of the present invention is to propose an appendiceal flushing endoscope system and its control method. Through image acquisition and recognition, the position and size of fecaliths in the images taken by the endoscope are automatically recognized, and the balloon pressurization variable is controlled based on the recognition data, improving the surgical efficiency and effect. The specific structure is as follows.

[0035] The appendiceal flushing endoscope system includes a balloon airway 1, a balloon 2, a flushing pipeline 3, and a two-in-one pipeline connector 4.

[0036] One end of the balloon airway 1 is provided with an air path connection valve 6, and the other end is connected to the balloon 2 through a two-in-one pipe connector 4.

[0037] One end of the flushing pipe 3 is connected to the three-way valve 5, and the other end is connected to the appendix cavity flushing port 7 through a two-in-one pipe connector 4.

[0038] The two-in-one pipe connector 4 enables the flushing pipe 3 to enter the interior of the balloon airway 1, thus forming a double-layer form in the distal pipe of the device, that is, the outer layer of the pipe is the balloon airway and the inner layer is the flushing pipe. The distal flushing pipe passes through the center of the balloon 2 and is connected to the appendix cavity flushing port 7.

[0039] The air path connection valve 6 is connected to an external controllable gas source, so as to provide a suitable air pressure for the airbag to expand to a predetermined size.

[0040] The three-way valve 5 can be respectively connected to a physiological saline pump and a vacuum pump system, so that physiological saline for flushing can be provided to the flushing pipe 3, and sewage, fecal water, etc. can be sucked back from the appendix cavity flushing port 7 through the vacuum effect.

[0041] The distal end of the whole system is integrally arranged in the outer sheath body and is sent into the body through the outer sheath body. The balloon 2 and the appendix cavity flushing port 7 can freely extend from the top of the outer sheath body. An endoscope is also arranged at the top of the outer sheath body for observing the morphology of the appendix and the situation of fecal stones.

[0042] The balloon 2 can be inflated through the balloon airway 1 and thus expand into a mushroom head shape. In this way, by pulling the balloon airway 1, the mushroom head-shaped balloon that has expanded outside can be pulled back into the outer sheath. During this process, larger fecal stones can be pulled back into the outer sheath by the pulling method.

[0043] Operating mode:

[0044] (1) The balloon 2 and the balloon airway 1 are sent into the body through the outer sheath body.

[0045] (2) Observe the appendix situation according to the endoscope on the outer sheath body, collect endoscopic images, and transmit them to the processor.

[0046] (3) The processor analyzes the size and position of the fecal stones according to the images transmitted back by the endoscope.

[0047] (4) For smaller fecal stones, the processor controls the three-way valve to open the physiological saline flushing mode, flushes the appendix fecal stones through the appendix cavity flushing port 7 for 1 - 3 seconds, then controls the three-way valve to open the vacuum suction mode, sucks back the fecal stone debris and sewage together through the appendix cavity flushing port 7, and sucks them into the vacuum pump system through the three-way valve.

[0048] (5) For relatively large fecaliths, extend the balloon out of the outer sheath, and the processor controls the gas path connection valve 6 to inflate the balloon. Select the inflation pressure according to the size of the fecalith so that the balloon expands to the corresponding mushroom head size. Manually or using a robotic arm, pull the balloon airway so that the mushroom-shaped balloon retracts into the outer sheath, thereby bringing the relatively large fecalith back into the outer sheath.

[0049] It can be understood that the time for flushing the appendiceal fecalith can be extended, thereby turning more fecaliths into very small debris, reducing the difficulty of removing large fecaliths, and also avoiding damage to the appendiceal inner membrane when removing the fecalith.

[0050] The specific method by which the processor judges the size and position of the fecalith based on the endoscopic image and controls the balloon expansion air pressure is as follows:

[0051] Step 1 Preprocessing of endoscopic image data.

[0052] To address the impact of blurred and unclear endoscopic images on automatic recognition, an optimized preprocessing method using adjacent multiple frames of images is adopted to improve the recognition effect of image targets.

[0053] Let

[0054] M i (u, v)

[0055] represent a pixel in a frame of image captured by the endoscopic camera. Among them, u and v represent the position coordinates of the image pixel. i represents the capture sequence. Therefore, the pixel values of the previous frame and the next frame of the image are respectively denoted as M i-1 (u, v), M i+1 (u, v). Correspondingly, take the symbol M i to represent the set of all pixels in the corresponding frame of image, that is, this frame of image.

[0056] Assume that the spatial displacement of adjacent frames of images is unchanged or extremely small, that is, when the coordinates are close, the pixel differences within a certain range of the corresponding positions in adjacent frames are extremely small.

[0057] Define the evaluation function

[0058]

[0059] where u′, v′ represent the pixel coordinates in the neighborhood near u, v, and the definitions of each sub-function are as follows.

[0060]

[0061] Among them, the (i - 1)-th frame is the frame before the i-th frame.

[0062] According to Equation 1, perform frame-by-frame and pixel-by-pixel evaluation on the captured endoscopic images.

[0063] For the i-th frame M i , calculate f according to Equation 1 i (u, v) value.

[0064] Set the threshold δ 1 , δ 2 , δ 1 > 0 is called the evaluation threshold, and δ 2 > 0 is called the hit threshold. For M i , set the counter c i . If f i (u, v)> δ 1 , then increment the frame counter by 1. Traverse all pixels of this frame. If the final value of the counter c i > δ 2 , this frame is called a valid frame. Otherwise, this frame is called an invalid frame.

[0065] Through the preprocessing and evaluation in Step 1, the original input image data is divided into valid frames and invalid frames, thereby obtaining stable image data with high signal-to-noise ratio and improving the recognition effect of image targets.

[0066] Step 2: Obtain the key image area from the valid frames.

[0067] The key image area refers to the subset area in the endoscopic valid frame image that may contain fecaliths.

[0068] Classic methods for obtaining regions of interest from images include template matching method, correlation filtering method, etc. These methods have high computational efficiency, but the matching effect is greatly affected by image noise and image distortion, and the matching accuracy is relatively low. In recent years, with the development of computing technology, methods based on convolutional neural networks have effectively improved the performance of region of interest extraction, with a large improvement in accuracy and robustness. However, due to higher computational complexity, the computational efficiency is correspondingly reduced. A large number of existing methods based on convolutional neural networks need to run on high-performance graphics computing devices, with high computational overhead, and the image data needs to be transmitted back to the backend computing center for centralized processing, which is not suitable for clinical surgery use. To solve the above problems, a more lightweight computational model is proposed, which improves the computational efficiency of existing methods, enabling the detection of regions of interest to be realized on an embedded front-end edge platform and improving the usability of the device.

[0069] Based on the preprocessing of the valid frames, the present invention proposes a lightweight neural network computational model to extract the region of interest containing fecaliths from the valid frame image to identify the size of the fecalith.

[0070] Let M i be a valid frame. M ii′ be one of its subsets, M ii′(u′, v′) is a pixel in the subset, and u′, v′ are the coordinates of the pixel in the subset.

[0071] The first layer of the neural network calculation model is implemented as follows.

[0072]

[0073] α 1 is a three-dimensional convolution kernel. j, k correspond to the coordinates in the spatial dimension, and the size (side length) of the convolution kernel is taken as the detection input (i.e., M ii′ ) of n corresponds to the third-dimensional coordinate, and n takes 32 groups from 1, 2, …, 32. β 0 is the corresponding linear bias. μ is the activation function of the neural network model. And it is agreed that:

[0074] α 1 (j, k, n) = (-1) n ·α 1 (k, j, n)

[0075]

[0076] ∈ takes a positive real number close to 0 as a control parameter, which helps to improve the sample gradient convergence performance. By adopting a symmetric convolution kernel and the above activation function, the recognition performance of the model is improved, and at the same time, the calculation efficiency of a single layer is increased.

[0077] The second layer of the model is implemented as follows.

[0078]

[0079] α 2 is the linear coefficient of the second layer, and β 1 is the corresponding linear bias. μ is the activation function. Different from the multi-layer two-dimensional convolution layers of the classical convolution network, the first layer of the present invention adopts a single-layer three-dimensional convolution kernel to form a lighter convolution layer, and combines the fully connected layer of the second layer for feature extraction, and the number of parameters is significantly reduced, thereby improving the calculation efficiency. The output vector N 2 , N 2 (p) represents its p-th component.

[0080] The output layer of the neural network model is as follows.

[0081]

[0082] α 3 is the linear coefficient of the output layer, and β 2 is the corresponding linear bias. μ is the activation function. The output N 3 corresponds to whether the input belongs to the region of interest (fecalith region).

[0083] The above model is trained using the loss function E:

[0084]

[0085] N 3 is the true label of the sample, is the label predicted by the model. The model weights are continuously adjusted through backpropagation and gradient descent optimization algorithms to minimize the loss function E.

[0086] After training, the above model is used to traverse each subset of the valid frames to obtain the region where the fecalith is located.

[0087] Step 3: Determine the fecalith cleaning mode based on the size of the region where the fecalith is located in the valid frame, and control the balloon pressurization variable.

[0088] Based on the camera calibration model, calculate the relationship between the image pixel size and the physical size in the real world.

[0089]

[0090] According to Equation 7, the size corresponding to the image subset in the real world can be calculated, that is

[0091]

[0092] where s is the scale factor of the camera image, l x and l y are the focal length parameters of the camera, and the above data are obtained through pre-calibration. Δu and Δv are the two side lengths representing the image subset, and ΔX and ΔY correspondingly represent the side lengths of the above region in the real world.

[0093] According to the side lengths ΔX and ΔY, the size (area) of the fecalith can be calculated. Based on the size of the fecalith, the normal saline flushing and vacuum recovery mode can be selected, or the balloon inflation retrieval mode can be selected. At the same time, the pressurization amount can also be obtained by looking up the table according to the area based on the size of the fecalith.

[0094] A new method for automatic image recognition and control based on an appendiceal endoscope is proposed. Through image acquisition and recognition, the position and size of the fecalith in the image taken by the endoscope are automatically recognized, and the balloon pressurization variable is controlled based on the recognition data to improve the surgical efficiency and effect. Experiments (Table 1) show that this method improves the calculation efficiency of the classical model and achieves the same recognition accuracy, with good results.

[0095] Table 1

[0096]

[0097] In each embodiment of the present application, each functional module can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Therefore, the technical solution of the present application can be embodied in the form of a software product, or the computer software product is stored in a storage medium, or the computer software product runs on a computer device, and such computer devices include not only personal computers and servers but also mobile terminals.

[0098] In several embodiments provided by the present application, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. Therefore, the above description is only for the embodiments of the present application and is not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various changes and modifications. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An appendix flushing endoscope system, characterized in that: Including balloon airway, balloon, flushing tube, two-in-one tube connector; One end of the balloon airway is provided with an airway connection valve, and the other end is connected to the balloon through a two-in-one pipeline connector; One end of the flushing pipeline is connected to the three-way valve, and the other end is connected to the appendix cavity flushing port through a two-in-one pipeline connector; The two-in-one tubing connector allows the flushing tubing to enter the interior of the balloon airway; The gas circuit connecting valve is connected to an external controllable gas source, and the three-way valve can be connected to a saline pump and a vacuum pump system respectively; The distal end of the entire system is integrally arranged in the outer sheath and is delivered into the body through the outer sheath; the balloon and the appendix cavity flushing port can be freely extended from the top of the outer sheath; The balloon can be inflated through the balloon airway, causing it to expand into a mushroom-shaped shape to capture fecal stones.

2. The system according to claim 1, characterized in that: The air circuit connecting valve provides appropriate air pressure to the air bag to expand it to a predetermined size.

3. The system according to claim 1, characterized in that: The three-way valve provides saline solution for flushing to the flushing pipeline and can suck back sewage from the flushing port of the appendix cavity through vacuum action.

4. The system according to claim 1, characterized in that: The distal end pipeline is formed into a double layer.

5. The system according to claim 4, characterized in that: The outer layer of the tube is the balloon airway, and the inner layer is the flushing tube.

6. The system according to claim 4, characterized in that: At the distal end, the irrigation tube passes through the balloon from the center and is connected to the flushing port of the appendix cavity.

7. The system of claim 1, wherein: The inflated mushroom-head balloon can be pulled back into the outer sheath by pulling the balloon airway. In the process, larger fecal stones can be pulled back into the outer sheath.

8. The system of claim 1, wherein: The outer sheath is provided with an endoscope.

9. A method for controlling the system according to claims 1-8, characterized in that: The processor analyzes the size and location of the fecal stone based on the images sent back by the endoscope: for smaller fecal stones, the processor controls the three-way valve to start the saline flushing and vacuum suction mode; for still larger fecal stones, the balloon pressurization expansion mode is used.

10. The method according to claim 1, characterized in that: The processing method of the processor also includes: (1) Using the adjacent multi-frame image optimization preprocessing method, the original input image data is divided into valid frames and non-valid frames; (2) Build a neural network model to obtain key image areas from valid frames; (3) Determine the fecal stone cleaning mode based on the size of the area where the fecal stone is located in the effective frame, and control the balloon pressurization variable.

Images