Endoscope flushing method and system

Abstract

The present invention relates to the field of medical devices and medical image processing technology, and more specifically, to an endoscope flushing method and system. Blur analysis is performed on images during surgery to determine whether the flushing conditions are currently met. In order to prevent automatic flushing from interfering with the doctor's ongoing operations, the present invention uses a surgical operation judgment algorithm that combines depth estimation and semantic segmentation, and installs a gyroscope on the endoscope to sense the movement state of the endoscope, so as to determine whether the current doctor is performing a surgical operation. Only when it is determined that the current doctor is not performing a surgical operation and the endoscopic image reaches the blurriness threshold will the automatic flushing switch be triggered, thereby achieving automatic flushing without interfering with the doctor's normal operation.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices and medical image processing, and more particularly to an endoscope flushing method and system. Background Art

[0002] In the past, some nasal endoscope irrigators could clean the lens without removing the endoscope from the nasal cavity. However, these irrigators required manual control of the flushing switch and could not automatically determine whether the endoscope needed cleaning. Some existing nasal endoscope irrigators simply install a device inside the nasal cavity to clean the endoscope, but still require manual control of the flushing switch. They do not have the function of automatically flushing when the lens is blurred, making them not truly automatic irrigators. Summary of the Invention

[0003] In order to overcome the above-mentioned defects in the prior art, the present invention provides an endoscope flushing method and system, which realizes automatic flushing of the endoscope without interfering with the normal operation of the doctor.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is: an endoscope flushing method, comprising the following steps:

[0005] S1. During surgery, the endoscope image is transmitted back to the industrial computer in real time;

[0006] S2. The industrial computer processes the endoscopic image at a rate of n frames per second using an image blur determination algorithm, calculates a blur score, and determines images exceeding a threshold score as blurred.

[0007] S3. Simultaneously with the fuzzy judgment, the industrial computer processes the endoscopic image using a surgical operation judgment algorithm that combines depth estimation and semantic segmentation to determine whether the doctor is currently performing an important, uninterruptible surgical procedure.

[0008] S4. The gyroscope mounted on the endoscope transmits the endoscope's motion information to the industrial computer in real time. The industrial computer determines whether the endoscope is moving slowly based on the gyroscope's angular velocity.

[0009] S5. When more than a plurality of frames of images are judged to be blurred, the industrial computer determines that the endoscope is in a blurred state and then checks the doctor's operation and endoscope movement information;

[0010] S6. When the industrial computer determines that the endoscope is currently in a blurred state, the doctor is not performing a critical surgical operation, and the endoscope is moving slowly, the flushing device switch installed outside the endoscope is triggered to flush the lens; if the above conditions are not met at the same time, the flushing switch is not triggered.

[0011] In this invention, a system consisting of four main devices is constructed: an endoscope, an endoscope flushing device, a gyroscope, and an industrial computer. During endoscopic surgery, the endoscope image is transmitted back to the industrial computer in real time. The industrial computer determines whether the endoscope currently requires flushing by judging image blur. Then, using a surgical operation judgment algorithm that combines depth estimation and semantic segmentation, and the gyroscope's sensing of the endoscope's motion state, it determines whether the endoscope currently meets the flushing conditions. When the endoscope requires flushing and meets the flushing conditions, the endoscope flushing device is triggered to automatically flush the endoscope.

[0012] Furthermore, the image blurring algorithm specifically includes the following steps:

[0013] S21. The industrial computer performs convolution on the received endoscopic image using the Laplacian operator, calculates the variance, and obtains the blur score of the image.

[0014] S22. Determine whether the blur score of the image exceeds a threshold; if so, determine the image to be blurred.

[0015] Furthermore, in step S2, the endoscopic image is processed at a speed of 1 frame / second; in step S5, when more than 3 frames of images are judged to be blurred images, the industrial computer determines that the current endoscope is in a blurred state.

[0016] Furthermore, the surgical operation judgment algorithm combining depth estimation and semantic segmentation specifically includes the following steps:

[0017] S31. The industrial computer performs semantic segmentation on the received endoscopic image to separate the instrument and tissue components.

[0018] S32. Estimating the depth of the instrument and tissue respectively to obtain the instrument depth value and the tissue depth value;

[0019] S33. Calculate the distance D between the instrument and the tissue based on the instrument depth value and the tissue depth value;

[0020] S34. Determine the size of D. If D is less than the threshold ε, determine that "a critical operation is being performed." Otherwise, determine that "a critical operation is not being performed."

[0021] Furthermore, the step S4 specifically includes:

[0022] S1. The gyroscope installed on the endoscope senses the angular velocity of the endoscope and transmits it back to the industrial computer;

[0023] S2. The industrial computer determines whether the angular velocity of the endoscope is less than a threshold value ε2; if so, the endoscope is determined to be moving slowly; otherwise, the endoscope is determined to be moving quickly and is not suitable for flushing.

[0024] The present invention also provides an endoscope flushing system, comprising:

[0025] Data acquisition module: used to transmit the images collected by the endoscope back to the industrial computer in real time during the operation; and used to collect the motion information of the gyroscope installed on the endoscope;

[0026] The data analysis module includes an image blur judgment unit, a surgical operation judgment unit combining depth estimation and semantic segmentation, and a speed judgment unit. The image blur judgment unit is used by the industrial computer to process the endoscopic image at a speed of n frames per second using an image blur judgment algorithm, calculate a blur score, and judge images exceeding a threshold score as blurred images; the semantic segmentation surgical operation judgment unit is used by the industrial computer to process the endoscopic image using a surgical operation judgment algorithm combining depth estimation and semantic segmentation to determine whether the current doctor is performing an important and uninterruptible surgical operation; and the speed judgment unit is used by the industrial computer to determine whether the endoscope is moving slowly based on the angular velocity of the gyroscope.

[0027] Decision control module: used to determine whether the flushing conditions are met; when more than a plurality of frames of images are judged to be blurred images, the industrial computer determines that the current endoscope is in a blurred state, and then checks the doctor's operation and endoscope movement information; when the industrial computer determines that the current endoscope is in a blurred state, the doctor is not performing a critical surgical operation, and the endoscope moves slowly, the flushing device switch installed outside the endoscope is triggered to flush the lens; if the above conditions are not met at the same time, the flushing switch will not be triggered.

[0028] Furthermore, the image blur judgment unit specifically includes:

[0029] Fuzzy calculation unit: used by the industrial computer to perform convolution with the received endoscope image using the Laplace operator, and then calculate the variance to obtain the fuzzy score of the image

[0030] Blur score judgment unit: used to judge whether the blur score of the image exceeds the threshold, and if so, it is judged as a blurred image.

[0031] Furthermore, the surgical operation judgment unit combining depth estimation and semantic segmentation includes:

[0032] Semantic segmentation unit: used by the industrial computer to perform semantic segmentation on the received endoscopic images, dividing them into instrument and tissue parts;

[0033] Depth estimation unit: used to estimate the depth of the instrument and tissue respectively, and obtain the instrument depth value and tissue depth value;

[0034] Distance calculation unit: used to calculate the distance value D between the instrument and the tissue according to the instrument depth value and the tissue depth value;

[0035] Judgment unit: used to judge the size of D. If D is less than the threshold ε, it is determined that "a critical operation is being performed", otherwise it is determined that "a critical operation is not being performed".

[0036] Furthermore, the speed determination unit includes:

[0037] Velocity acquisition unit: used to transmit the angular velocity of the endoscope sensed by the gyroscope installed on the endoscope back to the industrial computer;

[0038] Angular velocity judgment unit: used by the industrial computer to judge whether the angular velocity of the endoscope is less than the threshold ε2; if it is less than the threshold, it is judged that the endoscope is moving slowly; otherwise, it is judged that the endoscope is moving fast and is not suitable for flushing.

[0039] The present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the endoscope flushing method described above.

[0040] Compared with the prior art, the beneficial effects are as follows: the present invention provides an endoscope flushing method and system, which performs blurriness analysis on images during surgery to determine whether the flushing conditions are currently met. In order to prevent automatic flushing from interfering with the doctor's ongoing operations, the present invention uses a surgical operation judgment algorithm that combines depth estimation and semantic segmentation, and installs a gyroscope on the endoscope to sense the movement state of the endoscope, which can determine whether the current doctor is performing a surgical operation. Only when it is determined that the current doctor is not performing a surgical operation and the endoscopic image reaches the blurriness threshold will the automatic flushing switch be triggered, thereby achieving automatic flushing without interfering with the doctor's normal operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic flow chart of the method of the present invention.

[0042] Figure 2 It is a schematic diagram of the relationship between various devices of the present invention.

[0043] Figure 3 It is a flowchart of the surgical operation judgment algorithm combining depth estimation and semantic segmentation of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The present invention is described in one of the embodiments below in combination with the specific implementation methods. Among them, the drawings are only for illustrative purposes and represent only schematic diagrams rather than physical drawings, and cannot be understood as limitations on this patent; in order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0045] In the description of the present invention, it should be understood that if the terms "upper," "lower," "left," "right," etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, they are only for the purpose of facilitating the description of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, the terms describing the positional relationship in the accompanying drawings are only for illustrative purposes and should not be construed as limiting this patent. For those skilled in the art, the specific meanings of the above terms can be understood based on the specific circumstances. In addition, if there are descriptions of "first," "second," etc. in the embodiments of the present invention, the descriptions of "first," "second," etc. are only for descriptive purposes and should not be construed as indicating or implying their relative importance or implicitly indicating the number of the technical features indicated. Therefore, the definition of "first" or "second" may explicitly or implicitly include at least one of the features. In addition, the meaning of "and / or" appearing throughout the text includes three parallel solutions. For example, "A and / or B" includes solution A, solution B, or solutions that meet both A and B.

[0046] Example 1:

[0047] like Figure 1 and Figure 2 As shown, an endoscope flushing method first constructs a system consisting of four main devices: an endoscope, an endoscope flushing device, a gyroscope, and an industrial computer. The gyroscope is installed on the endoscope, and the endoscope lens captures images and transmits them to the endoscope imaging system. The endoscope imaging system is connected to the industrial computer for communication; the endoscope flushing device includes a flushing sheath, a flushing control unit, and a flushing execution unit. The industrial computer communicates with the flushing control unit, and the flushing control unit is connected to the flushing execution unit. The flushing sheath switch is controlled by the flushing execution unit to achieve flushing of the endoscope. The method includes the following steps:

[0048] S1. During surgery, the endoscope image is transmitted back to the industrial computer in real time;

[0049] S2. The industrial computer processes the endoscopic image at a rate of n frames per second using an image blur determination algorithm, calculates a blur score, and determines images exceeding a threshold score as blurred.

[0050] S3. Simultaneously with the fuzzy judgment, the industrial computer processes the endoscopic image using a surgical operation judgment algorithm that combines depth estimation and semantic segmentation to determine whether the doctor is currently performing an important, uninterruptible surgical procedure.

[0051] S4. The gyroscope mounted on the endoscope transmits the endoscope's motion information to the industrial computer in real time. The industrial computer determines whether the endoscope is moving slowly based on the gyroscope's angular velocity.

[0052] S5. When more than a plurality of frames of images are judged to be blurred, the industrial computer determines that the endoscope is in a blurred state and then checks the doctor's operation and endoscope movement information;

[0053] S6. When the industrial computer determines that the endoscope is currently in a blurred state, the doctor is not performing a critical surgical operation, and the endoscope is moving slowly, the flushing device switch installed outside the endoscope is triggered to flush the lens; if the above conditions are not met at the same time, the flushing switch is not triggered.

[0054] In this invention, during endoscopic surgery, the endoscope image is transmitted back to the industrial computer in real time. The computer determines whether the endoscope requires flushing by determining image blur. Then, using a surgical operation judgment algorithm that combines depth estimation and semantic segmentation, along with gyroscope sensing of the endoscope's motion, the computer determines whether the endoscope meets flushing requirements. If flushing is required and meets the flushing requirements, the endoscope flushing device switches on, automatically flushing the endoscope.

[0055] Specifically, the image blurring algorithm includes the following steps:

[0056] S21. The industrial computer performs convolution on the received endoscopic image using the Laplacian operator, calculates the variance, and obtains the blur score of the image.

[0057] S22. Determine whether the blur score of the image exceeds a threshold; if so, determine the image to be blurred.

[0058] In addition, in step S2, the endoscopic image is processed at a speed of 1 frame / second; in step S5, when more than 3 frames of images are judged to be blurred images, the industrial computer determines that the current endoscope is in a blurred state.

[0059] The surgical operation judgment algorithm combining depth estimation and semantic segmentation specifically includes the following steps:

[0060] S31. The industrial computer performs semantic segmentation on the received endoscopic image to separate the instrument and tissue components.

[0061] S32. Estimating the depth of the instrument and tissue respectively to obtain the instrument depth value and the tissue depth value;

[0062] S33. Calculate the distance D between the instrument and the tissue based on the instrument depth value and the tissue depth value;

[0063] S34. Determine the size of D. If D is less than the threshold ε, determine that "a critical operation is being performed." Otherwise, determine that "a critical operation is not being performed."

[0064] In addition, the step S4 specifically includes:

[0065] S1. The gyroscope installed on the endoscope senses the angular velocity of the endoscope and transmits it back to the industrial computer;

[0066] S2. The industrial computer determines whether the angular velocity of the endoscope is less than a threshold value ε2; if so, the endoscope is determined to be moving slowly; otherwise, the endoscope is determined to be moving quickly and is not suitable for flushing.

[0067] This embodiment performs a blurriness analysis on the images during the surgical process to determine whether the flushing conditions are currently met. To prevent automatic flushing from interfering with the doctor's ongoing operations, the present invention uses a surgical operation judgment algorithm that combines depth estimation and semantic segmentation, and installs a gyroscope on the endoscope to sense the endoscope's motion state, which can determine whether the doctor is currently performing a surgical operation. The automatic flushing switch is only triggered when it is determined that the doctor is not currently performing a surgical operation and the endoscopic image reaches the blurriness threshold, achieving automatic flushing without interfering with the doctor's normal operations.

[0068] Example 2

[0069] This embodiment provides an endoscope flushing system, comprising:

[0070] Data acquisition module: used to transmit the images collected by the endoscope back to the industrial computer in real time during the operation; and used to collect the motion information of the gyroscope installed on the endoscope;

[0071] The data analysis module includes an image blur judgment unit, a surgical operation judgment unit combining depth estimation and semantic segmentation, and a speed judgment unit. The image blur judgment unit is used by the industrial computer to process the endoscopic image at a speed of n frames per second using an image blur judgment algorithm, calculate a blur score, and judge images exceeding a threshold score as blurred images; the semantic segmentation surgical operation judgment unit is used by the industrial computer to process the endoscopic image using a surgical operation judgment algorithm combining depth estimation and semantic segmentation to determine whether the current doctor is performing an important and uninterruptible surgical operation; and the speed judgment unit is used by the industrial computer to determine whether the endoscope is moving slowly based on the angular velocity of the gyroscope.

[0072] Decision control module: used to determine whether the flushing conditions are met; when more than a plurality of frames of images are judged to be blurred images, the industrial computer determines that the current endoscope is in a blurred state, and then checks the doctor's operation and endoscope movement information; when the industrial computer determines that the current endoscope is in a blurred state, the doctor is not performing a critical surgical operation, and the endoscope moves slowly, the flushing device switch installed outside the endoscope is triggered to flush the lens; if the above conditions are not met at the same time, the flushing switch will not be triggered.

[0073] The image blur judgment unit specifically includes:

[0074] Fuzzy calculation unit: used by the industrial computer to perform convolution with the received endoscope image using the Laplace operator, and then calculate the variance to obtain the fuzzy score of the image

[0075] Blur score judgment unit: used to judge whether the blur score of the image exceeds the threshold, and if so, it is judged as a blurred image.

[0076] In addition, the surgical operation judgment unit combining depth estimation and semantic segmentation includes:

[0077] Semantic segmentation unit: used by the industrial computer to perform semantic segmentation on the received endoscopic images, dividing them into instrument and tissue parts;

[0078] Depth estimation unit: used to estimate the depth of the instrument and tissue respectively, and obtain the instrument depth value and tissue depth value;

[0079] Distance calculation unit: used to calculate the distance value D between the instrument and the tissue according to the instrument depth value and the tissue depth value;

[0080] Judgment unit: used to judge the size of D. If D is less than the threshold ε, it is determined that "a critical operation is being performed", otherwise it is determined that "a critical operation is not being performed".

[0081] Specifically, the speed determination unit includes:

[0082] Velocity acquisition unit: used to transmit the angular velocity of the endoscope sensed by the gyroscope installed on the endoscope back to the industrial computer;

[0083] Angular velocity judgment unit: used by the industrial computer to judge whether the angular velocity of the endoscope is less than the threshold ε2; if it is less than the threshold, it is judged that the endoscope is moving slowly; otherwise, it is judged that the endoscope is moving fast and is not suitable for flushing.

[0084] Example 3

[0085] This embodiment provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The processor executes the computer program to implement the endoscope flushing method described in Example 1.

[0086] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. An endoscope flushing system, characterized in that: include: Data acquisition module: used to transmit the images collected by the endoscope back to the industrial computer in real time during the operation; and for collecting motion information from a gyroscope mounted on an endoscope; The data analysis module includes an image blur judgment unit, a surgical operation judgment unit combining depth estimation and semantic segmentation, and a speed judgment unit. The image blur judgment unit is used by the industrial computer to process the endoscopic image at a speed of 1 frame per second using an image blur judgment algorithm, calculate a blur score, and judge images exceeding a threshold score as blurred images; the surgical operation judgment unit combining depth estimation and semantic segmentation is used by the industrial computer to process the endoscopic image using a surgical operation judgment algorithm combining depth estimation and semantic segmentation to judge whether the current doctor is performing an important and uninterruptible surgical operation; the speed judgment unit is used by the industrial computer to judge whether the endoscope is in a slow-moving state based on the angular velocity of the gyroscope; Decision control module: used to determine whether the flushing conditions are met; When more than three frames of images are judged to be blurred, the industrial computer determines that the current endoscope is in a blurred state, and then checks the doctor's operation and endoscope movement information; when the industrial computer determines that the current endoscope is in a blurred state, the doctor is not performing a critical surgical operation, and the endoscope moves slowly, it triggers the flushing device switch installed outside the endoscope to flush the lens; If the above conditions are not met at the same time, the flush switch will not be triggered; The image blur judgment unit specifically includes: Fuzzy calculation unit: used for industrial computers to use the Laplace operator to perform convolution with the received endoscope image, and then calculate the variance to obtain the fuzzy score of the image Blur score judgment unit: used to judge whether the blur score of the image exceeds a threshold, and if so, it is judged as a blurred image; The surgical operation judgment unit combining depth estimation and semantic segmentation includes: Semantic segmentation unit: used by the industrial computer to perform semantic segmentation on the received endoscopic images, dividing them into instrument and tissue parts; Depth estimation unit: used to estimate the depth of the instrument and tissue respectively, and obtain the instrument depth value and tissue depth value; Distance calculation unit: used to calculate the distance value D between the instrument and the tissue according to the instrument depth value and the tissue depth value; Judgment unit: used to judge the size of D. If D is less than the threshold ε, it is determined that "a critical operation is being performed", otherwise it is determined that "a critical operation is not being performed".

2. The endoscope flushing system according to claim 1, characterized in that: The speed judgment unit includes: Speed ​​acquisition unit: used to transmit the angular velocity of the endoscope sensed by the gyroscope installed on the endoscope back to the industrial computer; Angular velocity judgment unit: used by the industrial computer to judge whether the angular velocity of the endoscope is less than the threshold ε2; if it is less than the threshold, the endoscope is judged to be moving slowly; otherwise, the endoscope is judged to be moving quickly and is not suitable for flushing.

3. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement an endoscope flushing method, comprising the following steps: S1. During surgery, the endoscope image is transmitted to the industrial computer in real time. S2. The industrial computer processes the endoscopic image at a rate of 1 frame per second using an image blur determination algorithm, calculates a blur score, and determines images with a score exceeding a threshold as blurred. The image blur determination algorithm specifically comprises: S21. The industrial computer convolves the received endoscopic image with a Laplace operator, then calculates the variance to determine the image blur score; S22. Determines whether the image blur score exceeds a threshold; if so, determines the image as blurred. S3. Simultaneously with the fuzzy judgment, the industrial computer processes the endoscopic image using a surgical operation determination algorithm that combines depth estimation and semantic segmentation to determine whether the surgeon is currently performing a critical, uninterruptible surgical operation. The surgical operation determination algorithm that combines depth estimation and semantic segmentation specifically comprises the following steps: S31. The industrial computer performs semantic segmentation on the received endoscopic image, separating the instrument and tissue components; S32. Depth estimation is performed on the instrument and tissue, respectively, to obtain instrument and tissue depth values; S33. Based on the instrument and tissue depth values, a distance D between the instrument and tissue is calculated; S34. Determine the magnitude of D. If D is less than a threshold ε, the operator is deemed to be performing a critical operation; otherwise, the operator is deemed to be not performing a critical operation. S4. The gyroscope mounted on the endoscope transmits real-time motion information to the industrial computer. The computer then determines whether the endoscope is moving slowly based on the gyroscope's angular velocity. S5. If more than three frames are judged to be blurred, the industrial computer determines that the endoscope is currently blurred and then checks the doctor's operation and endoscope movement information; S6. When the industrial computer determines that the endoscope is currently in an obscured state, the doctor is not performing a critical surgical procedure, and the endoscope is moving slowly, it triggers a flushing device switch installed on the outside of the endoscope to flush the lens. If the above conditions are not met simultaneously, the flushing switch will not be triggered.

4. The electronic device according to claim 3, wherein: The step S4 specifically includes: S41. A gyroscope mounted on the endoscope senses the endoscope's angular velocity and transmits it to the industrial computer. S42. The industrial computer determines whether the angular velocity of the endoscope is less than a threshold value ε2. If so, the endoscope is determined to be moving slowly. Otherwise, the endoscope is determined to be moving rapidly and unsuitable for flushing.

Images