Digestive endoscope based on multi-modal perception and adaptive airflow cleaning control and automatic cleaning method thereof

Abstract

The application discloses a kind of based on multi-modal perception and adaptive airflow cleaning control's digestive endoscope and its automatic cleaning method, endoscope includes insertion part and handle, the distal end of insertion part is equipped with image sensor, cover mirror window and cleaning fluid passage, the proximal end of insertion part is connected with the distal end of handle, digestive endoscope further includes image processing unit and control component, image processing unit is configured with pre-trained AI model;Image sensor is configured to acquire the image of the operation area of the object, and input it to image processing unit;Image processing unit is configured to analyze the operation area image using AI model to estimate the definition of cover mirror window and convex deformation variable;If the identification result of AI model is that the definition of cover mirror window is lower than the preset definition threshold, and convex deformation variable reaches preset deformation variable threshold, then trigger cleaning instruction;In response to cleaning instruction, control component controls cleaning fluid passage and cleaning fluid source communication.

Description

Technical Field

[0001] This invention relates to the field of digestive tract examination equipment, and more particularly to a digestive endoscope based on multimodal sensing and adaptive airflow cleaning control and its automatic cleaning method. Background Technology

[0002] Digestive endoscopes, as minimally invasive diagnostic and therapeutic instruments, are widely used for the examination and treatment of cavities such as the esophagus, stomach, and colon. During clinical procedures, the endoscope's endoscope window (or protective glass, lens) is easily contaminated by blood, mucus, feces, or tissue debris from the patient's cavities. This contamination directly leads to blurred images, uneven lighting, or even complete obstruction of the surgical field, severely impacting the doctor's diagnostic accuracy and the safety of the procedure. To address this issue, current endoscopes are generally equipped with a cleaning system, typically including a nozzle at the tip, a fluid channel, and an external water / air pump.

[0003] However, typical cleaning triggering mechanisms are often passive and delayed: cleaning operations of traditional endoscopes (such as water spraying and air spraying) usually rely on the doctor's subjective judgment and are manually triggered by the doctor using a foot switch or handle button. This manual intervention not only increases the doctor's workload and distracts their attention, but more importantly, there is a significant delay between the occurrence of endoscope window contamination and the doctor's awareness and execution of the cleaning action, resulting in blurred or invalid images of the surgical area for a considerable period of time, posing potential medical risks.

[0004] Secondly, current technologies lack the ability to accurately identify the causes of blurriness: Endoscopic image blurriness can be caused by various factors, including contaminants adhering to the endoscope window surface and "tissue defocusing blur" caused by improper distance between the endoscope tip and the observed tissue. These two types of blurriness may appear similar visually, but their physical causes are entirely different. Existing automated cleaning solutions (such as timed cleaning or simple image grayscale thresholding) cannot effectively distinguish between "lens contamination" and "tissue defocusing." If tissue defocusing is mistakenly identified as lens contamination and cleaning is initiated, it not only fails to improve image quality but also wastes valuable cleaning fluids (such as water and air), and may even cause additional irritation or damage to the patient's cavities due to unnecessary air or water sprays.

[0005] Existing cleaning control schemes for digestive endoscopes suffer from problems such as delayed response and misjudgment. The disclosure of the above background information is only for the purpose of assisting in understanding the concept and technical solution of this application, and does not necessarily provide technical instruction. Summary of the Invention

[0006] The purpose of this invention is to provide a digestive endoscope based on multimodal sensing and adaptive airflow cleaning control to solve the problem of image blurring caused by digestive fluid adhesion.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A digestive endoscope based on multimodal perception and adaptive airflow cleaning control includes an insertion part and a handle. The distal end of the insertion part is provided with an image sensor, a cover window, and a cleaning fluid channel. The proximal end of the insertion part is connected to the distal end of the handle. The digestive endoscope also includes an image processing unit and a control component. The image processing unit is configured with a pre-trained AI model. The image sensor is configured to acquire images of the surgical area of ​​the subject and input them into the image processing unit; The image processing unit is configured to analyze the surgical area image using the AI ​​model to estimate the sharpness and protrusion deformation of the capping window; If the AI ​​model's recognition result is that the clarity of the covered window is lower than the preset clarity threshold, and the protrusion deformation reaches the preset deformation threshold, then a cleaning command is triggered. In response to the cleaning command, the control component controls the cleaning fluid channel to connect with the cleaning fluid source.

[0008] Furthermore, in accordance with any or a combination of the aforementioned technical solutions, the inner surface of the cover window is provided with a plurality of grooves, the ratio of the lens thickness corresponding to the groove to the lens thickness corresponding to the non-groove is between 1:30 and 1:15, or the lens thickness corresponding to the groove is between 20μm and 100μm.

[0009] Furthermore, following any one or a combination of the aforementioned technical solutions, the edge of the cover window is provided with a plurality of the aforementioned grooves at intervals.

[0010] Furthermore, in accordance with any or a combination of the aforementioned technical solutions, the sealing window is embedded in the distal end face of the insertion part, and the distal end face is also provided with the channel outlet of the cleaning fluid channel; The number of cleaning fluid channels and channel outlets is one, and both are in the shape of a ring or arc; or, the number of cleaning fluid channels and channel outlets is multiple, and the channel outlets are distributed at intervals along the outer periphery of the cover window. The flow direction of the cleaning fluid channel at the channel outlet is closer to the center normal of the cover window.

[0011] Furthermore, following any one or a combination of the aforementioned technical solutions, the number of the cleaning fluid channels and channel outlets is multiple, and the control component includes a controller and a controlled switch corresponding to each of the cleaning fluid channels; the switching state and / or opening degree of each controlled switch are independently controlled.

[0012] Furthermore, based on any or a combination of the aforementioned technical solutions, the distribution of the adhesive on the capping window is generated according to the protrusion deformation of the capping window identified by the AI ​​model and / or by analyzing the sharpness distribution of the surgical area image. The controller opens the controlled switch of the control section according to the distribution information of the adhesive on the sealing window, so as to form a clean fluid state adapted to the current distribution of the adhesive.

[0013] Furthermore, based on any one or a combination of the aforementioned technical solutions, the channel outlet of each clean fluid channel includes a plurality of first outlets and a plurality of second outlets spaced apart. The angle between the flow direction corresponding to the first outlet and the sealing window is the first angle, and the angle between the flow direction corresponding to the second outlet and the sealing window is the second angle, wherein the first angle is smaller than the second angle.

[0014] Furthermore, based on any one or a combination of the aforementioned technical solutions, and according to the convex deformation of the sealing window identified by the AI ​​model, the corresponding cleaning instructions are determined, including: If the identified convex deformation is between δ th If δ1 is reached, then the system controls only the first outlet to output intermittent pulsed fluid, where δ th As a preset deformation threshold, δ1 is greater than δ th ; If the detected protrusion deformation is between δ1 and δ2, the first outlet is controlled to output fluid within a preset first continuous duration. If the protrusion deformation is still between δ1 and δ2, the second outlet is controlled to output fluid within a preset second continuous duration, wherein δ2 is greater than δ1. If the detected convex deformation is between δ2 and δ3, then 50% of the second outlet is controlled to output fluid within a preset third continuous duration, then intermittently for a preset fourth continuous duration, and then the remaining 50% of the second outlet is controlled to output fluid within a preset third continuous duration, and then the first outlet is controlled to output fluid within a fifth continuous duration, wherein δ3 is greater than δ2.

[0015] Furthermore, based on any one or a combination of the aforementioned technical solutions, the aperture of the channel outlet is between 0.18 mm and 0.25 mm, and the center-to-center distance between adjacent channel outlets is between 0.45 mm and 0.63 mm; The distance between each channel outlet and the outer periphery of the cover window is the same, or the distance between two adjacent channel outlets and the outer periphery of the cover window is a first distance and a second distance, respectively, wherein the first distance is 0.18 mm to 0.25 mm larger than the second distance.

[0016] Furthermore, based on any or a combination of the aforementioned technical solutions, the control component includes a controller and a controlled switch for controlling the flow rate of the cleaning fluid channel; After the cleaning command is triggered, the target cleaning fluid pressure value is determined based on the convex deformation identified by the AI ​​model. The larger the convex deformation, the larger the target cleaning fluid pressure value. The controller controls the opening degree of the controlled switch to meet the target cleaning fluid pressure value.

[0017] Furthermore, following any one or a combination of the aforementioned technical solutions, the target cleaning fluid pressure value is determined using a pre-constructed pressure model P = a + b × δ, where P is the target cleaning fluid pressure value, δ is the convex deformation of the sealing window, and the constant parameters a and b are determined in the following manner: The minimum output fluid pressure value of the clean fluid source is determined and defined as P. min ; Determine the upper limit of the safe fluid pressure value applicable to endoscopic surgery, defined as P. max ; The preset deformation threshold is defined as δ. th The upper limit of the deformation variables identified by the AI ​​model is defined as δ. max ; (δ) th ,P min ), (δ max ,P max Substitute these parameters into the pressure model to determine the constant parameters a and b.

[0018] Furthermore, based on any or a combination of the aforementioned technical solutions, the AI ​​model includes a first sub-model configured to analyze the surgical area image to estimate the clarity of the capping lens. The first sub-model uses the MobileNetV3 model, which obtains the learning samples for training the first sub-model in the following way: Obtain N compliant, privacy-free clinical endoscopic images; A sharpness label is assigned to each clinical endoscopic image as follows: the sharpness score of the endoscopic image is evaluated using the Laplace gradient variance method; the N clinical endoscopic images are sorted according to their sharpness scores, and the images ranked first are... i % and after j % of clinical endoscopic images were deleted; the sharpness scores of the remaining clinical endoscopic images were normalized and used as the sharpness labels for the corresponding clinical endoscopic images.

[0019] Furthermore, based on any or a combination of the aforementioned technical solutions, the AI ​​model includes a second sub-model, which is configured to analyze the surgical area image to estimate the convex deformation of the capping window; The second sub-model uses the MobileNetV3 model, which obtains the learning samples for training the second sub-model in the following way: Using digestive endoscopy to obtain clinical endoscopic images or simulated endoscopic images in a simulated environment I k ; The sealing window of the digestive endoscope was placed under a laser confocal microscope, and the morphology of the outer surface of the sealing window was scanned to measure the micro-protrusion deformation Δ relative to the nominal surface. h k As the image I k The raised deformation label; Repeat the above steps to obtain a sample set of multiple learning samples.

[0020] Furthermore, following any one or a combination of the aforementioned technical solutions, the AI ​​model adopts the MobileNetV3 model, and its training samples include sharp samples, tissue defocus blur samples, and lens contamination samples, wherein the sharpness label value of the sharp samples is greater than the preset sharpness threshold, and its convex deformation label value is less than the preset deformation threshold. The sharpness label value of the out-of-focus blurred sample is less than the preset sharpness threshold, and its convex deformation label value is less than the preset deformation threshold. The sharpness label value of the lens contamination sample is less than the preset sharpness threshold, while its convex deformation label value is greater than the preset deformation threshold. The learning samples used to train the AI ​​model are obtained through the following methods: Using digestive endoscopy to obtain clinical endoscopic images or simulated endoscopic images in a simulated environment I k ; The sealing window of the digestive endoscope was placed under a laser confocal microscope, and the morphology of the outer surface of the sealing window was scanned to measure the micro-protrusion deformation Δ relative to the nominal surface. h k As the image I k The image is labeled with convex deformation variables; and the Laplacian gradient variance method is used to evaluate the image. I k Clarity score; Repeat the above steps and normalize the sharpness score of each image to obtain the corresponding sharpness label.

[0021] Furthermore, following any one or a combination of the aforementioned technical solutions, the following multi-task joint loss function is used to optimize the classification and recognition of the AI ​​model: L total =λ1·L reg +λ2·L cls +λ3·R(Θ), where L total For multi-task joint loss, L reg For the sharpness regression loss term, L cls R(Θ) is the loss term for fuzzy cause classification, and R(Θ) is the regularization term. λ1, λ2, and λ3 are the balancing hyperparameters of each loss term, and their values ​​are determined by grid search combined with the performance of the validation set.

[0022] According to another aspect of the present invention, an automated cleaning method for a digestive endoscope as described above is provided, comprising the following steps: Gastrointestinal endoscopy is used to acquire images of the surgical area of ​​the patient. The pre-trained AI model is used to analyze the surgical area image to predict the sharpness and convexity of the capping window; If the clarity of the covered window is detected to be lower than a preset clarity threshold, and the convex deformation reaches a preset deformation threshold, then a cleaning fluid is sprayed onto the area in front of the covered window.

[0023] Furthermore, following any one or a combination of the aforementioned technical solutions, triggering the spraying of cleaning fluid into the area in front of the sealed mirror window includes: The target clean fluid pressure value matched by the identified convex deformation is determined by using a pre-built pressure model. If the identified convex deformation is between δ th To δ1, the first outlet of the cleaning fluid channel is controlled to output intermittent pulsed fluid, where δ th As a preset deformation threshold, δ1 is greater than δ th The angle between the flow direction of the first outlet and the sealing window is the first angle; If the detected protrusion deformation is between δ1 and δ2, the first outlet is controlled to output fluid within a preset first continuous duration. If the protrusion deformation is still between δ1 and δ2, the second outlet of the cleaning fluid channel is controlled to output fluid within a preset second continuous duration, wherein δ2 is greater than δ1, and the angle between the flow direction of the second outlet and the sealing window is a second angle greater than the first angle. If the detected convex deformation is between δ2 and δ3, then 50% of the second outlet is controlled to output fluid within a preset third continuous duration, then intermittently for a preset fourth continuous duration, and then the remaining 50% of the second outlet is controlled to output fluid within a preset third continuous duration, and then the first outlet is controlled to output fluid within a fifth continuous duration, wherein δ3 is greater than δ2.

[0024] The beneficial effects of the technical solution provided by this invention are as follows: a. Real-time acquisition of surgical area images via image sensors and synchronous analysis of image features using pre-trained AI models. No manual triggering by the doctor is required. The system automatically identifies the contamination status of the endoscope window and immediately triggers cleaning instructions, providing doctors with a more continuous and clearer intraoperative view and reducing the diagnostic and treatment risks caused by unclear vision. b. The AI ​​model is configured to simultaneously estimate the sharpness and convex deformation of the cover window, and trigger a cleaning command by determining the combination of the two (sharpness below the threshold and deformation reaching the threshold); when the image blur is caused by tissue defocus (low sharpness but deformation not reaching the threshold), the system will not erroneously start the cleaning process, thereby avoiding the waste of cleaning fluids such as water and air, reducing unnecessary physical stimulation to the patient's cavity, and improving the comfort and safety of the operation; c. The micro-protrusion deformation of the endoscope window is creatively incorporated into the decision-making dimension of cleaning control, making the triggering conditions of the cleaning system more scientific and comprehensive. Doctors do not need to pay attention to the cleaning status of the endoscope window or manually perform cleaning actions during the operation. They can concentrate their attention on the observation of the lesion and the surgical operation, thereby reducing cognitive load and improving surgical efficiency and smoothness. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A schematic diagram of the main body of a digestive endoscope provided for an exemplary embodiment of the present invention; Figure 2 A cross-sectional view of the distal end of the insertion portion of an endoscope provided as an exemplary embodiment of the present invention; Figure 3 for Figure 2 A magnified schematic diagram of a local area A in the middle; Figure 4A front view of the distal end of the insertion portion of an endoscope provided as an exemplary embodiment of the present invention; Figure 5 for Figure 4 A magnified schematic diagram of a local region B in the middle; Figure 6 This is a schematic diagram of an endoscope self-cleaning process provided as an exemplary embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0029] In one embodiment of the present invention, a digestive endoscope based on multimodal sensing and adaptive airflow cleaning control is provided, such as... Figure 1 As shown, it includes an insertion part 100 and a handle 200. The distal end of the insertion part 100 is provided with an image sensor, a cover window 110 and a cleaning fluid channel 120. The proximal end of the insertion part is connected to the distal end of the handle 200. The digestive endoscope also includes an image processing unit and a control component (not shown). The image processing unit is configured with a pre-trained AI model. The image sensor is configured to acquire images of the surgical area of ​​the subject and input them into the image processing unit; The image processing unit is configured to analyze the surgical area image using the AI ​​model to estimate the sharpness and protrusion deformation of the capping window; If the AI ​​model's recognition result is that the clarity of the covered window is lower than the preset clarity threshold, and the protrusion deformation reaches the preset deformation threshold, then a cleaning command is triggered. In response to the cleaning command, the control component controls the cleaning fluid channel 120 to connect with the cleaning fluid source.

[0030] If the two conditions are not met simultaneously—the clarity of the cover window being lower than the preset clarity threshold and the convex deformation reaching the preset deformation threshold—the cleaning command will not be triggered, meaning there is no need to control the flow of cleaning fluid to clean the cover window.

[0031] This embodiment is based on two modal perceptions: the first is the perception of the clarity of the covered window 110, and the second is the perception of the protrusion deformation of the covered window 110 due to the adhesion of digestive fluid and other substances on its surface, in order to identify the cleaning needs of the covered window 110.

[0032] like Figure 3 and Figure 5 As shown, the inner surface of the cover window 110 is provided with a plurality of grooves 112. The ratio of the lens thickness corresponding to the groove 112 to the lens thickness corresponding to the non-groove area is between 1:30 and 1:15, or the lens thickness corresponding to the groove 112 is between 20μm and 100μm. This groove structure is used to amplify the perception of bulging deformation caused by the adhesion of digestive fluids and other substances to the surface of the cover window 110.

[0033] like Figure 4 and Figure 5 As shown, the edge of the cover window 110 is provided with a plurality of grooves 112. The present invention is not limited to the grooves 112 being distributed around the entire perimeter of the cover window 110, nor is it limited to the grooves 112 being distributed only around the perimeter of the cover window 110. In different embodiments, the grooves 112 may be distributed only around the perimeter of the cover window 110, or a plurality of grooves 112 may be discretely distributed in other areas besides the edges.

[0034] like Figure 4 As shown, the sealing window 110 is embedded in the distal end face of the insertion part 100, and the distal end face is also provided with the channel outlet 122 of the cleaning fluid channel 120; there are multiple cleaning fluid channels 120 and channel outlets 122, and the channel outlets 122 are distributed at intervals along the outer periphery of the sealing window 110; the present invention does not limit the number of cleaning fluid channels 120 and channel outlets 122 to multiple. In different embodiments, there is only one cleaning fluid channel 120 and channel outlet 122, and they are all annular or arc-shaped (not shown), that is, cleaning liquid or cleaning gas can be discharged in an annular shape on the distal end face of the insertion part 100.

[0035] like Figure 2As shown, the flow direction of the cleaning fluid channel 120 at the channel outlet 122 is closer to the center normal of the cover window 110.

[0036] This embodiment assumes that there are multiple cleaning fluid channels 120 and channel outlets 122, and details the adaptive airflow cleaning control scheme for the endoscope: the control component includes a controller and controlled switches (not shown) corresponding one-to-one with each of the cleaning fluid channels 120; the on / off state and / or opening degree of each controlled switch are independently controlled. It should be noted that the following adaptive airflow cleaning control embodiments are all control actions taken by the control component in response to a cleaning command triggered.

[0037] First control embodiment: Based on the protrusion deformation of the capping window 110 identified by the AI ​​model and / or by analyzing the sharpness distribution of the surgical area image, the distribution state of the adhesive on the capping window 110 is generated, that is, the specific location of the adhesive mainly distributed on the capping window 110 is analyzed. The controller opens controlled switches based on the distribution information (orientation information) of the adhered material on the cover window 110, thereby creating a cleaning fluid state adapted to the current distribution of the adhered material. This is because when all cleaning fluid channels 120 are open, the fluid scouring force converges on the central normal of the cover window 110. When the adhered material is not adhered to the center of the cover window 110, opening all cleaning fluid channels 120 wastes cleaning fluid and does not achieve a good cleaning effect. Therefore, this embodiment of the invention proposes to open specific cleaning fluid channels based on the distribution of the adhered material. In this way, for adhered material that is off-center, the formation of a correspondingly adapted asymmetric cleaning fluid state can more effectively clean off-center adhered material.

[0038] Second control embodiment: The channel outlet 122 of each cleaning fluid channel 120 includes a plurality of first outlets and a plurality of second outlets spaced apart; the angle between the flow direction corresponding to the first outlet and the cover window is a first angle, and the angle between the flow direction corresponding to the second outlet and the cover window is a second angle, wherein the first angle is smaller than the second angle. In a specific numerical illustration embodiment, the first angle is between 15° and 25° (e.g., set to 15°), and the second angle is between 28° and 40° (e.g., set to 35°).

[0039] Based on the convex deformation of the covered window identified by the AI ​​model, the corresponding cleaning instructions are determined, including: If the detected protrusion deformation is between 0.5μm (preset deformation threshold) and 1μm, then only the first outlet is controlled to output intermittent pulse fluid, for example, the pulse width is 40ms and the pulse interval is 80ms. The purpose is to sweep the thin deposits tangentially in a short time. If the detected protrusion deformation is between 1μm and 3.5μm, the first outlet is controlled to continuously output fluid for 200ms. If the protrusion deformation is still between 1μm and 3.5μm, the second outlet is controlled to continuously output fluid for 100ms. This process is repeated until the protrusion deformation is less than 1μm. Compared to the tangential sweep of the first outlet, the normal impact force of the second outlet is stronger.

[0040] If the identified protrusion deformation is between 3.5μm and 5μm (the upper limit of deformation that the AI ​​model in this embodiment can identify), then 50% of the second outlet is controlled to continuously output fluid for 120ms, followed by a 100ms interval, and then the remaining 50% of the second outlet is controlled to continuously output fluid for 120ms, and then the first outlet is controlled to continuously output fluid for 300ms. The misalignment control here refers to: assuming there are 40 channel outlets 122 distributed around the perimeter, numbered 1 through 40; odd-numbered outlets are the first outlets, and even-numbered outlets are the second outlets. The first batch of 50% of the second outlets under misalignment control is assumed to be outlets 2, 6, 10, 14, and 18; correspondingly, the remaining 50% of the second outlets under misalignment control are outlets 4, 8, 12, 16, and 20. Compared to the decrease in single-port flow rate when all outlets are open, the advantage of time-sharing staggered alternating control is that it can enhance the normal cleaning force of single-port airflow on the one hand, and reduce the probability of abdominal distension or spasm in patients on the other hand. Furthermore, when all outlets are open, the superposition of the normal component of airflow at the center of the mirror window will cause the image to turn white instantly, leading to misjudgment by the AI ​​model.

[0041] Third control embodiment: When the AI ​​model's recognition result is that the clarity of the cover window is lower than a preset clarity threshold and the convex deformation reaches a preset deformation threshold, a cleaning command is triggered. In this case, the target cleaning fluid pressure value is determined according to the convex deformation identified by the AI ​​model. The larger the convex deformation, the larger the target cleaning fluid pressure value. The controller controls the opening degree of the controlled switch to meet the target cleaning fluid pressure value.

[0042] Specifically, the target cleaning fluid pressure value is determined using a pre-constructed pressure model P = a + b × δ, where P is the target cleaning fluid pressure value, δ is the convex deformation of the sealing window, and the constant parameters a and b are determined in the following way: The minimum output fluid pressure value of the clean fluid source is determined and defined as P. min This depends on the operating characteristics of the clean fluid source, to be less than 1.2 cm. 3 Taking a miniature air pump as an example, its minimum output fluid pressure is 0.84 kPa; Determine the upper limit of the safe fluid pressure value applicable to endoscopic surgery, defined as P. max The pressure limit of 1.2 kPa specified in the clinical safety guidelines is adopted here, which complies with the safety standards for Class II medical devices. The preset deformation threshold is defined as δ. th As in the second control embodiment above, the value is set to 0.5 μm; the upper limit of the deformation detected by the AI ​​model is defined as δ. max As in the second control embodiment above, it is set to 5μm; Substitute (0.5μm, 0.84kPa) and (5μm, 1.2kPa) into the pressure model to determine the constant parameters a=0.8 and b=0.08; The convex deformation δ identified by the AI ​​model in real time will then be... pre Substituting into P = 0.8 + 0.08 × δ, the corresponding target clean fluid pressure value P is calculated. pre .

[0043] In the above multiple control embodiments, the aperture of the channel outlet 122 is between 0.18mm and 0.25mm (for example, set to 0.2mm), and the center distance between adjacent channel outlets 122 is between 0.45mm and 0.63mm (for example, set to 0.5mm), to ensure that the airflow coverage of adjacent air holes does not overlap or have blind spots, while avoiding mutual airflow interference; The distance between each channel outlet 122 and the outer periphery of the cover window 110 is the same; Alternatively, the distance between the first outlet and the outer periphery of the covered mirror window 110 is a first distance, and the distance between the second outlet and the outer periphery of the covered mirror window 110 is a second distance, wherein the first distance is 0.18 mm to 0.25 mm larger than the second distance. The advantage of this is that the jet from the first outlet, which performs tangential blowing, lands at the edge of the mirror window, while the jet from the second outlet, which performs stronger normal blowing, lands near the center of the edge of the mirror window, physically creating a cleaning effect of internal lifting and tangential sweeping of the outer edge.

[0044] The following is a detailed introduction to the AI ​​model: First Model Implementation Example: The AI ​​model in this example adopts the MobileNetV3 model. Its training samples include sharp samples, tissue defocus blur samples, and lens contamination samples. The sharpness label value of the sharp samples is greater than the preset sharpness threshold, and the convex deformation label value is less than the preset deformation threshold (i.e., sharpness S>0.6, convex deformation δ<0.5μm). The sharpness label value of the out-of-focus blurred sample is less than the preset sharpness threshold, and its convex deformation label value is less than the preset deformation threshold (i.e., sharpness S < 0.6, convex deformation δ < 0.5μm). The sharpness label value of the lens contamination sample is less than the preset sharpness threshold, and its convex deformation label value is greater than the preset deformation threshold (i.e., sharpness S < 0.6, convex deformation δ > 0.5 μm).

[0045] The learning samples used to train the AI ​​model are obtained through the following methods: Using digestive endoscopy to obtain clinical endoscopic images or simulated endoscopic images in a simulated environment I k ; Then, the capping window of the digestive endoscope was placed under a laser confocal microscope, and the morphology of the outer surface of the capping window was scanned to measure the micro-protrusion deformation Δ relative to the nominal surface. h k As the image I k The image is labeled with convex deformation variables; and the Laplacian gradient variance method is used to evaluate the image. I k The sharpness score; it should be noted that the deformation Δ of the micro-protrusions was measured using a laser confocal microscope. h k Not synchronized with the image acquisition in the same frame I k ; Repeat the above steps and normalize the sharpness score of each image as the corresponding sharpness label. This way, the image... I k , convex deformation label Δ h k A learning sample is composed of clarity labels.

[0046] Using the Laplacian gradient variance (Variance of Laplacian) to determine endoscopic image sharpness is one of the most commonly used and lightweight no-reference focus evaluation functions in industrial / medical imaging: the sharper the image, the higher the Laplacian gradient variance (V), which is used as the sharpness score. For example, after calculating the Laplacian gradient variance of 1000 clinical or simulated endoscopic images, 1000 variance values ​​are obtained, which are then normalized: V norm =(VV min ) / (V max -V min ), where V norm V is the normalized sharpness label value, and V is the calculated Laplacian gradient variance value. max V is the maximum value among 1000 variance values. min It is the minimum value among 1000 variance values.

[0047] In a further embodiment, after filtering out, for example, 2% of extreme frames from both ends of the variance value sorting sequence, normalization is performed: 1000 clinical or simulated endoscopic images are sorted by variance value from low to high (or from high to low), the 20 with the smallest variance value and the 20 with the largest variance value are deleted, and then the maximum value V is determined from the remaining 960 variance values. max and minimum value V min Then, normalization calculations are performed.

[0048] In this embodiment, the following multi-task joint loss function is used to optimize the classification and recognition of the AI ​​model: L total =λ1·L reg +λ2·L cls +λ3·R(Θ), where L total For multi-task joint loss, L reg For the sharpness regression loss term, L cls R(Θ) is the loss term for fuzzy cause classification, and R(Θ) is the regularization term. λ1, λ2, and λ3 are the balancing hyperparameters for each loss term, and their values ​​are determined through grid search combined with validation set performance. Specifically, for example, λ1 is set as the baseline weight, fixed at 1.0; λ2 is set to have multiple candidate values, and λ3 is similarly set to have multiple candidate values. The model needs to be evaluated on the validation set for performance on multiple tasks simultaneously. The comprehensive evaluation metric can be defined as: using the root mean square error (RMSE) as the sharpness regression accuracy, and the classification accuracy (Acc) as the classification accuracy. cls As the accuracy of fuzzy classification, in order to select the optimal balanced hyperparameter, a comprehensive index is determined, such as Score = α·(1-RMSE). norm )+β·Acc cls , among which, RMSEnorm This refers to the normalized root mean square error (RMSE). The specific normalization method is RMSE. norm =RMSE / RMSE ref RMSE ref Used to eliminate the influence of dimensions, RMSE ref Take the RMSE of the single task baseline on the validation set (since the baseline itself has converged, the RMSE is...). ref (This will not be excessively large) Alternatively, take the 99th percentile of the RMSE of the single-task baseline on the training set (there are usually 1% of poorly performing samples in the training set that raise the percentile value; taking the 99th percentile value can prevent excessively high RMSE). ref (Excessively large).

[0049] Since the previous rounds emphasized classification accuracy, the weight of β can be appropriately increased, for example, Score = 0.3 * (1 - RMSE / RMSE) ref )+0.7·Acc cls Then, iterate through all combinations of λ2 and λ3, training the model on the training set for each combination with λ1=1.0, and calculating the overall score on an independent validation set. Select the set of λ1, λ2, λ3 that results in the highest overall score on the validation set as the final balancing hyperparameters.

[0050] Second Model Implementation Example: Unlike the AI ​​model in the first model implementation example which performs multi-task recognition, in this example, the AI ​​model includes a first sub-model and a second sub-model, which respectively undertake the tasks of recognizing the sharpness of the cover window and recognizing the protruding deformation of the cover window. Specifically, the first sub-model can also adopt the MobileNetV3 model, which is used to recognize the sharpness of the cover window. It obtains the learning samples used to train the first sub-model through the following method: Obtain N compliant, privacy-free clinical endoscopic images; A sharpness label is assigned to each clinical endoscopic image as follows: the sharpness score of the endoscopic image is evaluated using the Laplace gradient variance method; the N clinical endoscopic images are sorted according to their sharpness scores, and the images ranked first are... i % and after j % of clinical endoscopic images were deleted; the sharpness scores of the remaining clinical endoscopic images were normalized and used as the sharpness labels for the corresponding clinical endoscopic images.

[0051] The second sub-model can also use the MobileNetV3 model, which is used to identify the convex deformation of the cover window. The training samples for training the second sub-model are obtained in the following way: Using digestive endoscopy to obtain clinical endoscopic images or simulated endoscopic images in a simulated environment I k ; The sealing window of the digestive endoscope was placed under a laser confocal microscope, and the morphology of the outer surface of the sealing window was scanned to measure the micro-protrusion deformation Δ relative to the nominal surface. h k As the image I k The raised deformation label; Repeat the above steps to obtain a sample set of multiple learning samples.

[0052] For the first sub-model, the following loss function is used to optimize its classification and recognition: L total =λ1·L reg +λ2·R(Θ), where L total For the total loss, L reg R(Θ) is the sharpness regression loss term, R(Θ) is the regularization term, and λ1 and λ2 are the balancing hyperparameters of each loss term. For the second sub-model, the following loss function is used to optimize its classification and recognition: L total =λ3·L cls +λ4·R(Θ), where L total For the total loss, L cls R(Θ) is the loss term for fuzzy cause classification, R(Θ) is the regularization term, and λ3 and λ4 are the balancing hyperparameters for each loss term.

[0053] In one embodiment of the present invention, an automated cleaning method for a digestive endoscope as described above is provided, such as... Figure 6 As shown, the cleaning method includes the following steps: Gastrointestinal endoscopy is used to acquire images of the surgical area of ​​the patient. The pre-trained AI model is used to analyze the surgical area image to predict the sharpness and convexity of the capping window; If the clarity of the covered window is detected to be lower than a preset clarity threshold, and the convex deformation reaches a preset deformation threshold, then a cleaning fluid is sprayed onto the area in front of the covered window.

[0054] The process of triggering the spraying of cleaning fluid into the area in front of the sealed mirror window includes: The target clean fluid pressure value matched by the identified convex deformation is determined by using a pre-built pressure model. If the identified convex deformation is between δ th To δ1, the first outlet of the cleaning fluid channel is controlled to output intermittent pulsed fluid, where δ th As a preset deformation threshold, δ1 is greater than δ th The angle between the flow direction of the first outlet and the sealing window is the first angle; If the detected protrusion deformation is between δ1 and δ2, the first outlet is controlled to output fluid within a preset first continuous duration. If the protrusion deformation is still between δ1 and δ2, the second outlet of the cleaning fluid channel is controlled to output fluid within a preset second continuous duration, wherein δ2 is greater than δ1, and the angle between the flow direction of the second outlet and the sealing window is a second angle greater than the first angle. If the detected convex deformation is between δ2 and δ3, then 50% of the second outlet is controlled to output fluid within a preset third continuous duration, then intermittently for a preset fourth continuous duration, and then the remaining 50% of the second outlet is controlled to output fluid within a preset third continuous duration, and then the first outlet is controlled to output fluid within a fifth continuous duration, wherein δ3 is greater than δ2.

[0055] The automatic cleaning method for the digestive endoscope provided in this embodiment belongs to the same inventive concept as the digestive endoscope based on multimodal sensing and adaptive airflow cleaning control provided in the above embodiment. Hereinafter, all contents of the digestive endoscope embodiment based on multimodal sensing and adaptive airflow cleaning control are incorporated into this embodiment of the automatic cleaning method for the digestive endoscope by reference, and will not be repeated.

[0056] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0057] The above description is only a specific embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A digestive endoscope based on multimodal sensing and adaptive airflow cleaning control, comprising an insertion part and a handle, wherein the distal end of the insertion part is provided with an image sensor, a closure window, and a cleaning fluid channel, and the proximal end of the insertion part is connected to the distal end of the handle, characterized in that, The inner surface of the cover window is provided with a number of grooves, and the ratio of the lens thickness corresponding to the groove to the lens thickness corresponding to the non-groove is between 1:30 and 1:15, or the lens thickness corresponding to the groove is between 20μm and 100μm. The digestive endoscope also includes an image processing unit and a control component, wherein the image processing unit is configured with a pre-trained AI model; The image sensor is configured to acquire images of the surgical area of ​​the subject and input them into the image processing unit; The image processing unit is configured to analyze the surgical area image using the AI ​​model to estimate the sharpness and protrusion deformation of the capping window; If the AI ​​model's recognition result is that the clarity of the covered window is lower than the preset clarity threshold, and the protrusion deformation reaches the preset deformation threshold, then a cleaning command is triggered. In response to the cleaning command, the control component controls the cleaning fluid channel to connect with the cleaning fluid source.

2. The digestive endoscope based on multimodal sensing and adaptive airflow cleaning control according to claim 1, characterized in that, The edge of the cover window is provided with a plurality of grooves at intervals.

3. The digestive endoscope based on multimodal sensing and adaptive airflow cleaning control according to claim 1, characterized in that, The cover window is embedded in the distal end face of the insertion part, and the distal end face is also provided with the channel outlet of the cleaning fluid channel; The number of cleaning fluid channels and channel outlets is one, and both are in the shape of a ring or arc; or, the number of cleaning fluid channels and channel outlets is multiple, and the channel outlets are distributed at intervals along the outer periphery of the cover window. The flow direction of the cleaning fluid channel at the channel outlet is closer to the center normal of the cover window.

4. The digestive endoscope based on multimodal sensing and adaptive airflow cleaning control according to claim 3, characterized in that, The number of cleaning fluid channels and channel outlets are both multiple, and the control component includes a controller and a controlled switch corresponding to each cleaning fluid channel; the switching state and / or opening degree of each controlled switch are independently controlled.

5. The digestive endoscope based on multimodal sensing and adaptive airflow cleaning control according to claim 4, characterized in that, Based on the protrusion deformation of the capping window identified by the AI ​​model and / or by analyzing the sharpness distribution of the surgical area image, the distribution of the adhesive on the capping window is generated. The controller opens the controlled switch of the control section according to the distribution information of the adhesive on the sealing window, so as to form a clean fluid state adapted to the current distribution of the adhesive.

6. The digestive endoscope based on multimodal sensing and adaptive airflow cleaning control according to claim 4, characterized in that, Each clean fluid channel has multiple first outlets and multiple second outlets spaced apart. The angle between the flow direction corresponding to the first outlet and the sealing window is the first angle, and the angle between the flow direction corresponding to the second outlet and the sealing window is the second angle, wherein the first angle is smaller than the second angle.

7. The digestive endoscope based on multimodal sensing and adaptive airflow cleaning control according to claim 6, characterized in that, Based on the convex deformation of the covered window identified by the AI ​​model, the corresponding cleaning instructions are determined, including: If the identified convex deformation is between δ th If δ1 is reached, then the system controls only the first outlet to output intermittent pulsed fluid, where δ th As a preset deformation threshold, δ1 is greater than δ th ; If the detected protrusion deformation is between δ1 and δ2, the first outlet is controlled to output fluid within a preset first continuous duration. If the protrusion deformation is still between δ1 and δ2, the second outlet is controlled to output fluid within a preset second continuous duration, wherein δ2 is greater than δ1. If the detected convex deformation is between δ2 and δ3, then 50% of the second outlet is controlled to output fluid within a preset third continuous duration, then intermittently for a preset fourth continuous duration, and then the remaining 50% of the second outlet is controlled to output fluid within a preset third continuous duration, and then the first outlet is controlled to output fluid within a fifth continuous duration, wherein δ3 is greater than δ2.

8. The digestive endoscope based on multimodal sensing and adaptive airflow cleaning control according to any one of claims 4 to 7, characterized in that, The aperture of the channel outlet is between 0.18 mm and 0.25 mm, and the center-to-center distance between adjacent channel outlets is between 0.45 mm and 0.63 mm; The distance between each channel outlet and the outer periphery of the cover window is the same, or the distance between two adjacent channel outlets and the outer periphery of the cover window is a first distance and a second distance, respectively, wherein the first distance is 0.18 mm to 0.25 mm larger than the second distance.

9. The digestive endoscope based on multimodal sensing and adaptive airflow cleaning control according to any one of claims 1 to 7, characterized in that, The control components include a controller and a controlled switch that controls the flow rate of the cleaning fluid channel; After the cleaning command is triggered, the target cleaning fluid pressure value is determined based on the convex deformation identified by the AI ​​model. The larger the convex deformation, the larger the target cleaning fluid pressure value. The controller controls the opening degree of the controlled switch to meet the target cleaning fluid pressure value.

10. The digestive endoscope based on multimodal sensing and adaptive airflow cleaning control according to claim 9, characterized in that, The target cleaning fluid pressure value is determined using a pre-constructed pressure model P = a + b × δ, where P is the target cleaning fluid pressure value, δ is the convex deformation of the sealing window, and the constant parameters a and b are determined in the following way: The minimum output fluid pressure value of the clean fluid source is determined and defined as P. min ; Determine the upper limit of the safe fluid pressure value applicable to endoscopic surgery, defined as P. max ; The preset deformation threshold is defined as δ. th The upper limit of the deformation variables identified by the AI ​​model is defined as δ. max ; (δ) th ,P min ), (δ max ,P max Substitute these parameters into the pressure model to determine the constant parameters a and b.

11. The digestive endoscope based on multimodal sensing and adaptive airflow cleaning control according to any one of claims 1 to 7, characterized in that, The AI ​​model uses the MobileNetV3 model, and its training samples include sharp samples, tissue out-of-focus blur samples, and lens contamination samples. Among them, the sharpness label value of the sharp samples is greater than the preset sharpness threshold, and its convex deformation label value is less than the preset deformation threshold. The sharpness label value of the out-of-focus blurred sample is less than the preset sharpness threshold, and its convex deformation label value is less than the preset deformation threshold. The sharpness label value of the lens contamination sample is less than the preset sharpness threshold, while its convex deformation label value is greater than the preset deformation threshold. The learning samples used to train the AI ​​model are obtained through the following methods: Using digestive endoscopy to obtain clinical endoscopic images or simulated endoscopic images in a simulated environment I k ; The sealing window of the digestive endoscope was placed under a laser confocal microscope, and the morphology of the outer surface of the sealing window was scanned to measure the micro-protrusion deformation Δ relative to the nominal surface. h k As the image I k The image is labeled with convex deformation variables; and the Laplacian gradient variance method is used to evaluate the image. I k Clarity score; Repeat the above steps and normalize the sharpness score of each image to obtain the corresponding sharpness label.

12. The digestive endoscope based on multimodal sensing and adaptive airflow cleaning control according to claim 11, characterized in that, The following multi-task joint loss function is used to optimize the classification and recognition of the AI ​​model: L total =λ1·L reg +λ2·L cls +λ3·R(Θ), where L total For multi-task joint loss, L reg For the sharpness regression loss term, L cls R(Θ) is the loss term for fuzzy cause classification, and R(Θ) is the regularization term. λ1, λ2, and λ3 are the balancing hyperparameters of each loss term, and their values ​​are determined by grid search combined with the performance of the validation set.

13. The digestive endoscope based on multimodal sensing and adaptive airflow cleaning control according to any one of claims 1 to 7, characterized in that, The AI ​​model includes a first sub-model configured to analyze the surgical area image to estimate the sharpness of the capping window; The first sub-model uses the MobileNetV3 model, which obtains the learning samples for training the first sub-model in the following way: Obtain N compliant, privacy-free clinical endoscopic images; A sharpness label is assigned to each clinical endoscopic image by evaluating its sharpness score using the Laplace gradient variance method. The N clinical endoscopic images are sorted according to their sharpness score, and the images ranked first are... i % and after j % of clinical endoscopic images were deleted; the sharpness scores of the remaining clinical endoscopic images were normalized and used as the sharpness labels for the corresponding clinical endoscopic images.

14. The digestive endoscope based on multimodal sensing and adaptive airflow cleaning control according to any one of claims 1 to 7, characterized in that, The AI ​​model includes a second sub-model configured to analyze the surgical area image to estimate the convex deformation of the capping window; The second sub-model uses the MobileNetV3 model, which obtains the learning samples for training the second sub-model in the following way: Using digestive endoscopy to obtain clinical endoscopic images or simulated endoscopic images in a simulated environment I k ; The sealing window of the digestive endoscope was placed under a laser confocal microscope, and the morphology of the outer surface of the sealing window was scanned to measure the micro-protrusion deformation Δ relative to the nominal surface. h k As the image I k The raised deformation label; Repeat the above steps to obtain a sample set of multiple learning samples.

15. An automated cleaning method for a digestive endoscope as described in any one of claims 1 to 14, characterized in that, Includes the following steps: Gastrointestinal endoscopy is used to acquire images of the surgical area of ​​the patient. The pre-trained AI model is used to analyze the surgical area image to predict the sharpness and convexity of the capping window; If the clarity of the covered window is detected to be lower than a preset clarity threshold, and the convex deformation reaches a preset deformation threshold, then a cleaning fluid is sprayed onto the area in front of the covered window.

16. The automatic cleaning method for a digestive endoscope according to claim 15, characterized in that, Triggering the spraying of cleaning fluid into the area in front of the covered mirror window includes: The target clean fluid pressure value matched by the identified convex deformation is determined by using a pre-built pressure model. If the identified convex deformation is between δ th To δ1, the first outlet of the cleaning fluid channel is controlled to output intermittent pulsed fluid, where δ th As a preset deformation threshold, δ1 is greater than δ th The angle between the flow direction of the first outlet and the sealing window is the first angle; If the detected protrusion deformation is between δ1 and δ2, the first outlet is controlled to output fluid within a preset first continuous duration. If the protrusion deformation is still between δ1 and δ2, the second outlet of the cleaning fluid channel is controlled to output fluid within a preset second continuous duration, wherein δ2 is greater than δ1, and the angle between the flow direction of the second outlet and the sealing window is a second angle greater than the first angle. If the detected convex deformation is between δ2 and δ3, then 50% of the second outlet is controlled to output fluid within a preset third continuous duration, then intermittently for a preset fourth continuous duration, and then the remaining 50% of the second outlet is controlled to output fluid within a preset third continuous duration, and then the first outlet is controlled to output fluid within a fifth continuous duration, wherein δ3 is greater than δ2.

Images