Gastrointestinal endoscope operation auxiliary system and method

Abstract

The invention belongs to the field of medical operation systems, and particularly relates to a gastrointestinal endoscope operation assisting system and method.The system comprises a hardware control module, an intelligent analysis module and a man-machine interaction module; the hardware control module comprises a mechanical arm execution unit and a self-adaptive adjusting mechanism, and the mechanical arm execution unit adopts a six-degree-of-freedom mechanical arm to carry an endoscope body to achieve rapid adaptation with existing gastrointestinal endoscope equipment through an electromagnetic lock catch; the intelligent analysis module comprises a real-time navigation system and an AI focus recognition system, and the real-time navigation system constructs a 3D alimentary canal model based on a preoperative CT image and realizes endoscopic positioning and path planning through an SLAM algorithm during an operation; according to the AI focus recognition system, an improved YOLOv7 model is adopted to carry out real-time analysis on white light / NBI images. According to the multi-device cooperative control technology, an endoscope, an ultrasonic probe and a treatment instrument are integrated through a unified communication protocol, and one-stop operation of'inspection-diagnosis-treatment 'is achieved.

Description

Technical Field

[0001] The present invention relates to the field of medical equipment, and in particular to a gastroenteroscopic operation auxiliary system and method. Background Art

[0002] Evolution of traditional gastroenterological technology: Rigid gastroscopy (1868): German inventor Kussmaul invented the metal tube gastroscopy, which was eliminated due to its easy damage to tissues; Flexible gastroscopy (1950): Japanese inventor Uji Tatsuro developed the fiber endoscope, which achieved flexible observation and became the basis of modern gastroenterological endoscopy; Electronic gastroscopy (1980s): CCD cameras were introduced, and the resolution was increased to megapixels, supporting digital image processing; Capsule endoscopy (after 2000): Wireless swallowable design broke through the limitations of invasiveness, but it cannot be actively controlled and treated.

[0003] Patent 1: CN202023098097.4, Abstract: Integrates a sampling sponge ball, an elastic balloon bag, and dual-channel drug delivery. The balloon pressure is used to control the uniform coating of the drug on the gastric mucosa, solving the problem of the single function of traditional gastroscopy.

[0004] Patent 2: CN213883147U, Abstract: A screw-driven piston plate is used to control the amount of liquid medicine delivered, and precise drug delivery is achieved through scale lines, reducing operational errors;

[0005] Patent three: CN33563688, Abstract: Design of an air chamber structure at the front end of a flexible gastroscope tube, adjusting the bending length through the inflatable tube to improve the permeability of the digestive tract.

[0006] Current gastrointestinal endoscopy has the following pain points: it relies on the doctor's experience, and novices are prone to missing lesions or causing damage to the patient's mucosa; manual operation is time-consuming and consumes a lot of physical energy for the doctor; complex cases (such as gastrointestinal bleeding, large foreign objects) require the coordination of multiple devices, and the operation is cumbersome.

[0007] Therefore, to address the above problems, a gastrointestinal endoscopy operation assistance system and method are proposed. Summary of the Invention

[0008] The present invention specifically relates to a gastroenteroscopic operation assistance system that integrates robotic arm control, real-time image analysis, intelligent navigation and pressure feedback, and is suitable for gastroenteroscopic examination and minimally invasive surgery scenarios.

[0009] The technical solution adopted by the present invention to solve its technical problems is as follows: a gastroenteroscopic operation assistance system and method described in the present invention includes a hardware control module, an intelligent analysis module, and a human-computer interaction module; the hardware control module includes a robotic arm execution unit and an adaptive adjustment mechanism, and the robotic arm execution unit uses a 6-degree-of-freedom robotic arm to carry the endoscope body and achieve rapid adaptation to existing gastroenteroscopic equipment through electromagnetic locks;

[0010] The intelligent analysis module includes a real-time navigation system and an AI lesion recognition system. The real-time navigation system constructs a 3D digestive tract model based on preoperative CT images and uses the SLAM algorithm to achieve endoscopic positioning and path planning during surgery. The AI ​​lesion recognition system uses an improved YOLOv7 model to perform real-time analysis of white light / NBI images.

[0011] The human-computer interaction module includes an augmented reality operating console and an emergency takeover mechanism, and the emergency takeover mechanism is provided with dual-mode switching.

[0012] Preferably, the robotic arm execution unit integrates a multimodal sensing probe and a dual-channel treatment tube at the end; the multimodal sensing probe includes a white light / NBI light source, a pressure sensor and a miniature ultrasound probe; the main channel of the dual-channel treatment tube is used for biopsy forceps / aspirator, and the secondary channel is equipped with a targeted contrast agent sprayer.

[0013] Preferably, the adaptive adjustment mechanism is an inflatable airbag belt provided on the outside of the endoscope body, which automatically adjusts the pressure according to the curvature of the intestine to reduce friction damage.

[0014] Preferably, the real-time navigation system includes a multi-objective optimization algorithm and a resistance feedback mechanism; the multi-objective optimization algorithm balances lesion coverage and operation time to generate an optimal inspection path; the resistance feedback mechanism automatically pauses and vibrates to remind when the robotic arm propulsion resistance is greater than 5N.

[0015] Preferably, the AI ​​lesion recognition system includes the ability to classify and identify lesions, automatically mark areas, and generate three-dimensional thermal maps.

[0016] Preferably, the augmented reality console is integrated with a HoloLens head display to achieve:

[0017] (1) Real-time display of endoscopic images + AI analysis results + 3D navigation path;

[0018] (2) Gesture control of the robot arm’s motion trajectory (clench your fist to pause / slide your finger to adjust the viewing angle).

[0019] Preferably, the dual-mode switching of the emergency takeover mechanism is:

[0020] Assisted mode: The doctor takes the lead in the operation, and the system provides intelligent suggestions;

[0021] Automatic mode: For standardized inspection processes (such as early cancer screening), the robotic arm executes according to preset programs.

[0022] A method for a gastrointestinal endoscope operation assisting system, characterized by comprising the following steps:

[0023] S1. Multimodal data fusion processing

[0024] (1) Preoperative 3D modeling

[0025] Based on the CT image segmentation of the digestive tract contour, the Delaunay triangulation algorithm is used to generate the topological structure and mark the high-risk areas (such as the high-incidence areas of ulcers).

[0026] (2) Real-time navigation during surgery

[0027] The SLAM algorithm integrates endoscopic images, IMU inertial data, and airbag pressure values, updating the 3D positioning model every second:

[0028] Path planning: Using improved A* algorithm to avoid areas with curvature greater than 120°

[0029] When the propulsion resistance is greater than 5N, reverse creep compensation is triggered;

[0030] S2. AI dynamic lesion recognition

[0031] (1) YOLOv7 enhanced model

[0032] The training set contains 100,000 annotated images (20 types of lesions). The attention mechanism CBAM module is introduced to output the following in real time:

[0033] Lesion bounding box (confidence ≥ 0.92 marked in red)

[0034] Heat map of malignancy probability (red - high risk, blue - low risk).

[0035] (2) Ultrasound data-assisted diagnosis

[0036] For tiny lesions (<3mm), the high-frequency ultrasound mode is automatically switched to measure the blood flow velocity in the submucosal layer (>2cm / s indicates the possibility of malignancy).

[0037] The present invention is beneficial in that:

[0038] 1. Multi-device collaborative control technology: Integrates endoscopes, ultrasound probes, and treatment equipment through a unified communication protocol to achieve a one-stop "examination-diagnosis-treatment" operation.

[0039] 2. Adaptive mucosal protection technology: The airbag pressure is dynamically related to the scope body advancement speed, and the pressure fluctuation range is controlled within 0.3-0.8kPa.

[0040] 3. Cross-modal data fusion algorithm: Fuse and analyze white light images, ultrasound data, and mechanical feedback information to improve the detection rate of tiny lesions. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 Schematic diagram of the framework structure of the gastrointestinal endoscope operation auxiliary system of the present invention. DETAILED DESCRIPTION

[0043] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0044] Example 1: Hardware Control Method

[0045] 1. Dynamic adaptation and control of the robotic arm

[0046] The rapid installation and calibration uses an electromagnetic lock to fix the 6-DOF robotic arm to the endoscope body. The bending angle of the body is detected by a laser calibrator (with an accuracy of ±0.5°) and the motion compensation parameters of the robotic arm are automatically generated.

[0047] When the multimodal sensing linkage white light / NBI light source is switched, the ultrasound probe frequency (adjustable from 5 to 12 MHz) and the pressure sensor sensitivity threshold (dynamic range of 0.1 to 1N) are adjusted synchronously.

[0048] 2. Adaptive mucosal protection execution

[0049] Airbag pressure closed-loop control

[0050] The annular airbag is installed outside the scope. Every 5cm of advancement triggers an intestinal diameter measurement (based on ultrasound echo time difference) and dynamically adjusts the inflation pressure:

[0051] Straight pipe section: 0.3kPa (reduced friction)

[0052] Bending section: 0.8kPa (to prevent the mirror body from impacting)

[0053] When resistance suddenly increases: automatically reduce pressure and pause propulsion.

[0054] 3. Collaborative operation of therapeutic devices

[0055] Dual-channel linkage mechanism

[0056] The main channel (3.2mm diameter) is embedded with a biopsy forceps magnetic guide module, and the secondary channel (1.8mm diameter) is equipped with a pulsed contrast agent nozzle that triggers the injection of contrast agent (0.2ml / pulse) by pressure difference.

[0057] Example 2: Human-computer interaction method

[0058] 1. Augmented reality operation process

[0059] Gesture Control Protocol

[0060] Fist: Freeze robotic arm movement

[0061] Draw a circle with your index finger: switch to NBI / white light mode

[0062] Push your palm forward: Start the automatic sampling program.

[0063] Multi-view display strategy

[0064] The split screen displays real-time endoscopy images (60% area), 3D navigation map (20%), and AI analysis report (20%), and supports voice command zooming.

[0065] 2. Dual-mode security switching mechanism

[0066] Assisted Mode

[0067] The doctor manually controls the advancement speed with a joystick (adjustable from 0 to 2 cm / s). The AI ​​provides:

[0068] Vibration feedback (resistance warning)

[0069] Voice prompt ("Polyp found 15mm ahead").

[0070] Automatic mode

[0071] Preset parameters for standardized inspection processes (such as early cancer screening):

[0072] Propulsion speed: 1.2cm / s

[0073] Dwell time per quadrant: 8 seconds

[0074] Sampling interval: Biopsy is completed within 5 seconds after the suspicious area is automatically marked.

[0075] Example 3: Quality Control Method

[0076] 1. Operational integrity verification

[0077] Part coverage detection

[0078] The system has a built-in list of anatomical structures (9 key areas such as esophagus / cardia / gastric body, etc.). Uncovered areas will trigger a buzzer alarm and mark navigation gaps.

[0079] 2. Emergency Response Protocol

[0080] Bleeding Event Response

[0081] When the hemoglobin spectral characteristics (415nm absorption peak) are detected:

[0082] (1) Automatically switch the secondary channel to spray thrombin (0.5ml / s)

[0083] (2) Pressurize the airbag to 1.2kPa to stop bleeding

[0084] (3) Start negative pressure suction (-80kPa) to remove blood clots

[0085] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention, and such changes and modifications fall within the scope of the invention as claimed.

Claims

1. A gastrointestinal endoscope operation assistance system, characterized by: The system includes hardware control module, intelligent analysis module, and human-computer interaction module; The hardware control module includes a robotic arm execution unit and an adaptive adjustment mechanism. The robotic arm execution unit uses a 6-degree-of-freedom robotic arm to carry the endoscope body and achieve rapid adaptation to existing gastrointestinal endoscope equipment through electromagnetic locks; The intelligent analysis module includes a real-time navigation system and an AI lesion recognition system. The real-time navigation system constructs a 3D digestive tract model based on preoperative CT images and uses the SLAM algorithm to achieve endoscopic positioning and path planning during surgery. The AI ​​lesion recognition system uses an improved YOLOv7 model to perform real-time analysis of white light / NBI images. The human-computer interaction module includes an augmented reality operating console and an emergency takeover mechanism, and the emergency takeover mechanism is provided with dual-mode switching.

2. The gastrointestinal endoscopy operation assistance system according to claim 1, characterized in that: The robotic arm execution unit integrates a multimodal sensing probe and a dual-channel treatment tube at the end; the multimodal sensing probe includes a white light / NBI light source, a pressure sensor and a miniature ultrasound probe; the main channel of the dual-channel treatment tube is used for biopsy forceps / aspirator, and the secondary channel is equipped with a targeted contrast agent sprayer.

3. The gastrointestinal endoscopy operation assistance system according to claim 1, characterized in that: The self-adaptive adjustment mechanism is provided with an inflatable airbag belt outside the endoscope body, which automatically adjusts the pressure according to the curvature of the intestine to reduce friction damage.

4. The gastrointestinal endoscopy operation assistance system according to claim 1, characterized in that: The real-time navigation system includes a multi-objective optimization algorithm and a resistance feedback mechanism; the multi-objective optimization algorithm balances lesion coverage and operation time to generate an optimal inspection path; the resistance feedback mechanism automatically pauses and vibrates to remind when the robotic arm's propulsion resistance is greater than 5N.

5. The gastrointestinal endoscopy operation assisting system and method according to claim 1, characterized in that: The AI ​​lesion recognition system includes the ability to classify and identify lesions, automatically mark areas, and generate three-dimensional thermal maps.

6. The gastrointestinal endoscopy operation assisting system and method according to claim 1, characterized in that: The augmented reality console integrates the HoloLens head display to achieve: (1) Real-time display of endoscopic images + AI analysis results + 3D navigation path; (2) Gesture control of the robot arm’s motion trajectory (clench your fist to pause / slide your finger to adjust the viewing angle).

7. The gastrointestinal endoscopy operation assisting system and method according to claim 1, characterized in that: The dual-mode switching of the emergency takeover mechanism is: Assisted mode: The doctor takes the lead in the operation, and the system provides intelligent suggestions; Automatic mode: For standardized inspection processes (such as early cancer screening), the robotic arm executes according to preset programs.

8. The method of the gastrointestinal endoscopy operation assisting system according to claim 1, characterized in that: The following steps are involved: S1. Multimodal data fusion processing (1) Preoperative 3D modeling Based on the CT image segmentation of the digestive tract contour, the Delaunay triangulation algorithm is used to generate the topological structure and mark the high-risk areas (such as the high-incidence areas of ulcers). (2) Real-time navigation during surgery The SLAM algorithm integrates endoscopic images, IMU inertial data, and airbag pressure values, updating the 3D positioning model every second: Path planning: Using improved A* algorithm to avoid areas with curvature greater than 120° When the propulsion resistance is greater than 5N, reverse creep compensation is triggered; S2. AI dynamic lesion recognition (1) YOLOv7 enhanced model The training set contains 100,000 annotated images (20 types of lesions). The attention mechanism CBAM module is introduced to output the following in real time: Lesion bounding box (confidence ≥ 0.92 marked in red) Heat map of malignancy probability (red - high risk, blue - low risk). (2) Ultrasound data-assisted diagnosis For tiny lesions (<3mm), the high-frequency ultrasound mode is automatically switched to measure the blood flow velocity in the submucosal layer (>2cm / s indicates the possibility of malignancy).

Images