Gastrointestinal endoscope body and knob torsion monitoring method and device based on multi-modal sensing

Abstract

The invention relates to a multimodal sensing-based gastrointestinal endoscope body and knob torsion monitoring method and device. Hardware transformation is performed on an existing mature gastrointestinal endoscope, a rotary encoder, a piezoresistive torque sensor and a magnetorheological damper are arranged at the knob end of the gastrointestinal endoscope, and an IMU (inertial measurement unit) is implanted into the endoscope body end of the gastrointestinal endoscope. On one hand, PWM control can be carried out on the magnetorheological damper according to rotation angle information obtained at the knob end, so that tactile resistance changes are fed back to an operator, and the operator can conveniently carry out tactile perception on the rotation degree of the knob; on the other hand, the tissue resistance coefficient is calculated through the pressure information, and the threshold value is dynamically updated based on the tissue resistance coefficient, so that torsion safety monitoring is more accurate; and effective early warning reminding is carried out on the user through a preset grading early warning triggering mechanism, so that the whole operation process is ensured to be carried out stably.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and particularly to a method and device for monitoring the body and knob torsion of a gastroscope and colonoscope based on multimodal sensing. It is applicable to the operation scenarios of electronic gastroscopes, colonoscopes, and other endoscopes. Background Art

[0002] Traditional gastroscope and colonoscope operations have the following deficiencies. 1. Blind spot risk: The operator cannot intuitively perceive the cumulative angle of the body torsion in the body. A typical prior art is the content disclosed in patent CN202411208858.7. The technical solution it discloses only focuses on the adjustment of the medical bed and does not solve the problem of body mechanics monitoring. 2. Over - operation injury: Continuous knob torsion can easily lead to mechanical fatigue of the endoscope body. A typical prior art is the content disclosed in patent CN201020239339.4. The torsion mechanism disclosed in its technical solution lacks a real - time feedback mechanism and there is a risk of torsion damage to the electron microscope sample stage. 3. Complications induction: Existing literature shows that excessive torsion of the endoscope body can lead to iatrogenic perforation. When the torsion angle of the endoscope body > 180°, the perforation risk increases by 3 times. Summary of the Invention

[0003] The purpose of the present invention is to at least solve one of the deficiencies of the prior art, and provide a method and device for monitoring the body and knob torsion of a gastroscope and colonoscope based on multimodal sensing.

[0004] To achieve the above - mentioned purpose, the present invention adopts the following technical solutions: Specifically, a method for monitoring the body and knob torsion of a gastroscope and colonoscope based on multimodal sensing is proposed, including the following: Obtain the target action scenario, and determine the reference body torsion threshold and reference knob torsion threshold for each operation part according to the target action scenario; Obtain the rotation angle information and pressure information at the knob end, and obtain the inertial measurement data at the body end; Perform PWM control on the magnetorheological damper based on the rotation angle information to cause a change in tactile resistance and feedback it to the operator; Determine the body torsion angle based on the inertial measurement data; Calculate the tissue resistance coefficient based on the pressure information, and dynamically update the reference body torsion threshold and reference knob torsion threshold for each operation part based on the tissue resistance coefficient; Determine the current operation part, and obtain the dynamically updated body torsion threshold and knob torsion threshold for the current operation part; Based on the magnitude relationship between the rotation angle information and the knob torsion threshold, as well as the magnitude relationship between the mirror body torsion angle and the mirror body torsion threshold, early warning reminders are made according to a preset hierarchical early warning trigger mechanism.

[0005] Further, specifically, the target application scenarios include a routine examination scenario, a complex lesion treatment scenario, and a special anatomical structure scenario; In the routine examination scenario, the reference mirror body torsion threshold for the esophageal operation site is 1.5 turns, the reference knob torsion threshold is 90°, the reference mirror body torsion threshold for the gastric cavity operation site is 2 turns, the reference knob torsion threshold is 120°, the reference mirror body torsion threshold for the duodenal operation site is 1 turn, and the reference knob torsion threshold is 60°, where one turn is 360°; In the complex lesion treatment scenario, in the bleeding treatment scenario, the reference knob torsion threshold is 180°, and it is necessary to ensure that the pressure information is lower than the pressure threshold of 8 N. In the stenosis passage scenario, it is specified that the single - turn torsion angle of the mirror body cannot exceed 30°; In the special anatomical structure scenario, a 3D model is established based on the patient's CT / MRI data. Based on the 3D model, the reference mirror body torsion threshold and the reference knob torsion threshold are pre - calculated, and it is specified that the single - turn torsion angle of the mirror body cannot exceed 30°. And based on the patient's X - ray image, image recognition is performed, and the reference mirror body torsion threshold of the ileocecal region in the image recognition is set to 0.8 turns.

[0006] Further, specifically, PWM control is performed on the magnetorheological damper based on the rotation angle information, causing a change in the tactile resistance and feeding it back to the operator, including, Calculating the target damping force according to the rotation angle information, and then performing PWM control on the magnetorheological damper according to the target damping force, causing a change in the magnetic field strength, and further dynamically adjusting the viscosity of the magnetorheological fluid. The change in the viscosity of the magnetorheological fluid causes a change in the tactile resistance and is fed back to the operator.

[0007] Further, specifically, the reference mirror body torsion threshold and the reference knob torsion threshold for each operation site are dynamically updated based on the tissue resistance coefficient, including, Denote the tissue resistance coefficient as tissue_resistance, and the reference mirror body torsion threshold or the reference knob torsion threshold before update as base_angle. Then the updated mirror body torsion threshold or knob torsion threshold is base_angle * (1 - 0.15*tissue_resistance).

[0008] Further, the method further includes, Set a mechanical safety threshold, which includes that the mirror body cannot be continuously twisted in one direction by more than 3 turns, and the left / right rotation angle of the knob cannot exceed 150°. When any one of the rotation angle information or the mirror body torsion angle exceeds the mechanical safety threshold, a warning reminder is given.

[0009] Furthermore, specifically, the preset hierarchical warning trigger mechanism includes: Insert the first intermediate value and the second intermediate value into the range from 0 to the updated mirror body torsion threshold and divide it into three intervals, which are recorded as the first safety interval, the first yellow warning interval, and the first red warning interval in ascending order; insert the third intermediate value and the fourth intermediate value into the range from 0 to the updated knob torsion threshold and divide it into three intervals, which are recorded as the second safety interval, the second yellow warning interval, and the second red warning interval in ascending order; If at least one of the rotation angle information or the mirror body torsion angle is within the red warning interval, it is a level 3 warning, and a corresponding warning reminder is given; If both parameters of the rotation angle information or the mirror body torsion angle are within the yellow warning interval, it is a level 2 warning, and a corresponding warning reminder is given; If only a single parameter of the rotation angle information or the mirror body torsion angle is within the yellow warning interval, it is a level 1 warning, and a corresponding warning reminder is given.

[0010] Furthermore, the method further includes: Pre-store the historical curve of the torsion parameters in the expert standard mode, compare the deviation value between the current operation and the historical curve of the torsion parameters, and give a warning reminder when the deviation value is greater than the deviation threshold.

[0011] Furthermore, the method further includes that when the forward pressure of the mirror body recorded by the pressure information is greater than the pressure threshold, the mechanical conduction path is marked by a heat map for visual display.

[0012] The present invention also proposes a gastroscope mirror body and knob torsion monitoring device based on multi-modal sensing, including the following: A rotary encoder, a piezoresistive torque sensor, and a magnetorheological damper are arranged at the rotary end of the gastroscope, and an IMU is implanted at the mirror body end of the gastroscope; A processing module For obtaining the target action scenario, and determining the reference mirror body torsion threshold and the reference knob torsion threshold of each operation part according to the target action scenario; Obtain the rotation angle information and the pressure information at the knob end through the rotary encoder and the piezoresistive torque sensor, and obtain the inertial measurement data at the mirror body end through the IMU; Perform PWM control on the magnetorheological damper based on the rotation angle information to cause a change in the tactile resistance and feedback it to the operator; Determine the torsional angle of the endoscope body based on the inertial measurement data; Calculate the tissue resistance coefficient based on the pressure information, and dynamically update the reference torsional threshold of the endoscope body and the reference torsional threshold of the knob for each operation site based on the tissue resistance coefficient; Determine the current operation site, and obtain the dynamically updated torsional threshold of the endoscope body and the torsional threshold of the knob at the current operation site; According to the magnitude relationship between the rotation angle information and the torsional threshold of the knob, and the magnitude relationship between the torsional angle of the endoscope body and the torsional threshold of the endoscope body, issue a warning reminder according to the preset hierarchical warning trigger mechanism.

[0013] The present invention also proposes a gastroscope endoscope body and knob torsional monitoring device based on multimodal sensing, including the following: A data acquisition module for acquiring multi-source associated data of a target user; A multimodal processing module for performing normalization processing on the multi-source associated data and converting it into a text vector based on a multimodal processing model; A score evaluation module for obtaining an emotion type score by classifying the text vector based on the emotion type through a large model, obtaining an emotion intensity score by classifying the emotion intensity, obtaining a behavior pattern score by recognizing the behavior pattern, and obtaining a keyword recognition score by recognizing keywords; A final score calculation module for accumulating the emotion type score, the emotion intensity score, the behavior pattern score, and the keyword recognition score according to a preset weight coefficient to obtain a final score; A result judgment module for judging whether the final score is higher than a preset threshold. If so, it is determined that the target user is a high-risk user of abnormal speech. If not, it is determined that the target user is a low-risk user of abnormal speech; A result output module for outputting the judgment result.

[0014] The beneficial effects of the present invention are as follows: The present invention proposes a gastroscope endoscope body and knob torsional monitoring method and device based on multimodal sensing. By performing hardware transformation on the existing mature gastroscope, a rotary encoder, a piezoresistive torque sensor, and a magnetorheological damper are provided at the rotary end of the gastroscope, and an IMU is implanted at the endoscope body end of the gastroscope. On the one hand, the rotation angle information obtained at the knob end can be used to perform PWM control on the magnetorheological damper to cause a change in tactile resistance and feedback it to the operator to facilitate the operator's tactile perception of the degree of knob rotation; on the other hand, the tissue resistance coefficient is calculated based on the pressure information, and the threshold is dynamically updated based on this to make the torsional safety monitoring more accurate; an effective warning reminder is also issued to the user through a preset hierarchical warning trigger mechanism to ensure the stable progress of the entire operation process. Description of the Drawings

[0015] By describing the embodiments shown in the accompanying drawings in detail, the above and other features of the present disclosure will become more apparent. The same reference numerals in the drawings of the present disclosure denote the same or similar elements. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings. In the drawings: Figure 1 The flowchart of the gastroscope body and knob torsion monitoring method based on multi-modal sensing according to the present invention is shown; Figure 2 The schematic diagram of the change feedback of tactile resistance caused by PWM control of the magnetorheological damper in the present invention is shown; Figure 3 The structural schematic diagram of the gastroscope body and knob torsion monitoring system based on multi-modal sensing according to the present invention is shown; Figure 4 The signal flow description when the gastroscope body and knob torsion monitoring method based on multi-modal sensing according to the present invention is applied is shown. Specific embodiments

[0016] The concept, specific structure and technical effects of the present invention will be clearly and completely described below in conjunction with the embodiments and the drawings to fully understand the purpose, solution and effects of the present invention. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0017] Embodiment 1, referring to Figure 1 , the present invention proposes a gastroscope body and knob torsion monitoring method based on multi-modal sensing, including the following: Step 110: Obtain the target action scenario (which can be directly obtained by manual selection from the operator), and determine the reference gastroscope body torsion threshold and the reference knob torsion threshold for each operation part according to the target action scenario; Step 120: Obtain the rotation angle information and pressure information at the knob end, and obtain the inertial measurement data at the gastroscope body end; Step 130: Perform PWM control on the magnetorheological damper based on the rotation angle information to cause a change in tactile resistance and feedback it to the operator; The following is the principle of calculating the damping force in step 130: I. Control principle architecture design 1. Multi-modal feedback closed-loop system Sensor layer: Rotary encoder (1000PPR accuracy) + IMU (200Hz sampling rate); Control layer: STM32H7 series MCU (main frequency 480MHz); Execution layer: Magnetorheological damper (response time < 10ms); Feedback layer: Tactile resistance generation (adjustable range 0 - 5N); 2. Real - time control process Rotation angle θ → Differentiate to calculate angular velocity ω (rad / s) → Second - order filtering process → Damping force mapping → PWM generation; II. Construction of core mathematical model 1. Non - linear damping force model $$ F_{damp} = k_1 \cdot tanh(\frac{θ}{θ_{max}}) + k_2 \cdot ω + k_3\cdot \frac{dω}{dt} $$ Where: - $k_1 = 2.5N$ is the angle gain coefficient - $k_2 = 0.8N·s / rad$ is the viscous damping coefficient - $k_3 = 0.15N·s² / rad$ is the inertia compensation term - $θ_{max}$ is dynamically adjusted according to the anatomical site (e.g., 120° for the esophagus, 0.8 turns for the ileocecal part) 2. PWM duty cycle mapping $$ D = \begin{cases} \frac{F_{damp}}{F_{max}} \cdot 100\% & F_{damp} \leq F_{max} \\ 100\% & F_{damp} > F_{max} \end{cases} $$ (F_max = 5N, corresponding to the maximum magnetic field strength of 2T) 3. Implementation of dynamic parameter adjustment algorithm ```python def dynamic_damping_control(theta, omega, phase, tissue_resist): # Anatomical stage parameter mapping phase_params = { 'esophagus': {'k1':2.8, 'k2':0.7, 'theta_max':120}, 'ileocecal': {'k1': 3.5, 'k2': 1.2, 'theta_max': 0.8 * 360} } # Tissue resistance compensation k1_comp = phase_params[phase]['k1'] * (1 + 0.2 * tissue_resist) # Angular acceleration estimation d_omega = kalman_filter(omega_history) # Estimate the differential using Kalman filter # Damping force calculation F_damp = k1_comp * np.tanh(np.radians(theta) / phase_params[phase]['theta_max']) + phase_params[phase]['k2'] * omega + 0.15 * d_omega # Safety clamping F_clamped = np.clip(F_damp, 0, 5) # PWM conversion pwm_duty = (F_clamped / 5) * 100 return pwm_duty, F_clamped IV. Hardware feature adaptation strategy 1. Hysteresis compensation module Use the Preisach model to compensate for the hysteresis effect of magnetorheological fluid: $$ H_{eff} = H_{appl} + α \cdot \frac{dH}{dt} $$ where α = 0.03 is the empirical coefficient; 2. Temperature drift correction Implant a PT100 temperature sensor and establish a temperature drift compensation model: $$ k_{temp} = 1 - 0.015(T - 25) $$ Correct the damping force parameters in real time; 3. Multi-level early warning linkage - Yellow warning (50% threshold): Increase the PWM frequency to 20 kHz to enhance tactile perception; - Red warning (80% threshold): superimposed with 5Hz pulsed vibration feedback; - Safety threshold breached: activate the mechanical locking device.

[0018] Step 140: Determine the torsion angle of the lens body based on the inertial measurement data; Step 150: Calculate the tissue resistance coefficient based on the pressure information, and dynamically update the reference lens body torsion threshold and the reference knob torsion threshold for each operation part based on the tissue resistance coefficient; The following is the complete conversion process for calculating the tissue resistance coefficient based on pressure information, including three core links: signal processing, feature extraction, and dynamic mapping: 1. Pressure signal preprocessing flow Original signal acquisition (sampling rate: 1000Hz) raw_pressure = get_pressure_sensor_data() Moving average filtering (window size: 200ms) window_size = 200 # 200 sample points smoothed = np.convolve(raw_pressure, np.ones(window_size) / window_size, mode='valid') Baseline drift correction (dynamic threshold detection) baseline = np.percentile(smoothed, 10) # Take the 10th percentile as the base pressure adjusted_pressure = smoothed - baseline Peak hold processing (retain the maximum contact pressure) peak_hold = maximum_filter1d(adjusted_pressure, size=50) # 50ms window hold Tissue resistance feature extraction 1. Time-domain feature calculation # Calculate features within a 200ms time window window = peak_hold[-200:] # Take the latest 200ms data features = { 'mean_p': np.mean(window), # Average pressure (0-5N range) 'std_p': np.std(window), # Pressure fluctuation intensity 'slope': np.polyfit(np.arange(200), window, 1)[0], # Pressure change slope 'peak_ratio': len(window[window > 0.5*np.max(window)]) / 200 # High pressure ratio } 2. Frequency domain feature extraction fft_result = np.fft.fft(window) freq = np.fft.fftfreq(len(window), d=0.001) # Sampling interval 1ms # Extract the energy in the 3-8Hz frequency band (corresponding to tissue elastic vibration characteristics) mask = (np.abs(freq) >= 3) & (np.abs(freq) <= 8) features['freq_energy'] = np.sum(np.abs(fft_result[mask])**2) 3. Dynamic mapping of tissue resistance coefficient Physical model establishment The viscoelastic properties of tissues conform to the Kelvin-Voigt model : Among them: k: Tissue elastic coefficient (N / mm) c: Tissue viscosity coefficient (N·s / mm) Real-time parameter identification Online estimation by recursive least squares (RLS): Python # Input matrix (displacement x is calculated from IMU data) X = np.vstack([displacement, velocity]).T y = features['mean_p'] # Observed pressure # RLS parameter update theta = np.linalg.inv(X.T @ X + 1e-4*np.eye(2)) @ X.T @ y k, c = theta[0], theta[1] Normalization processing # The reference parameters are from biomechanical experiment data k_norm = k / 0.15 # The elastic coefficient of normal mucosa is 0.15 N / mm c_norm = c / 0.02 # The viscous coefficient of normal mucosa is 0.02 N·s / mm # Calculation of the comprehensive resistance coefficient tissue_resistance = 0.6*k_norm + 0.4*c_norm tissue_resistance = np.clip(tissue_resistance, 0, 1.5) # Limit it within the range of 0 - 150%. See Table 3 below for the clinical verification parameters:

[0019] Table 3 Step 160: Determine the current operation site (here, it can be obtained by directly obtaining the operator's manual selection or by collecting images with a gastroscope and determining it through image recognition), and obtain the dynamically updated mirror body torsion threshold and knob torsion threshold for the current operation site; Step 170: According to the magnitude relationship between the rotation angle information and the knob torsion threshold, and the magnitude relationship between the mirror body torsion angle and the mirror body torsion threshold, give an early warning reminder according to the preset hierarchical early warning trigger mechanism.

[0020] In this Embodiment 1, by performing hardware transformation on the existing mature gastroscope, a rotary encoder, a piezoresistive torque sensor, and a magnetorheological damper are set at the rotary end of the gastroscope, and an IMU is implanted at the mirror body end of the gastroscope. On the one hand, the rotation angle information obtained at the knob end can be used to perform PWM control on the magnetorheological damper to cause a change in tactile resistance and feedback it to the operator to facilitate the operator's tactile perception of the degree of knob rotation; on the other hand, the tissue resistance coefficient is calculated through the pressure information, and the threshold is dynamically updated based on this to make the torsion safety monitoring more accurate; an effective early warning reminder is also given to the user through the preset hierarchical early warning trigger mechanism to ensure the stable progress of the entire operation process.

[0021] As a preferred implementation manner of the present invention, specifically, the target application scenarios include conventional examination scenarios, complex lesion treatment scenarios, and special anatomical structure scenarios; In the routine examination scenario, the reference endoscope body torsion threshold for the esophageal operation site is 1.5 turns, the reference knob torsion threshold is 90°, the reference endoscope body torsion threshold for the gastric cavity operation site is 2 turns, the reference knob torsion threshold is 120°, the reference endoscope body torsion threshold for the duodenal operation site is 1 turn, and the reference knob torsion threshold is 60°, where one turn is 360°; In the complex lesion treatment scenario, in the bleeding treatment scenario, the reference knob torsion threshold is 180°, and it is necessary to ensure that the pressure information is lower than the pressure threshold of 8 N. In the stenosis passage scenario, the single - turn torsion angle of the endoscope body is limited not to exceed 30°; In the special anatomical structure scenario, a 3D model is established based on the patient's CT / MRI data. The reference endoscope body torsion threshold and the reference knob torsion threshold are pre - calculated based on the 3D model, and the single - turn torsion angle of the endoscope body is limited not to exceed 30°. And based on the patient's X - ray image, image recognition is performed, and the reference endoscope body torsion threshold of the ileocecal region in the image recognition is set to 0.8 turns.

[0022] In this preferred embodiment, 1. Routine examination scenario (esophagus / stomach / duodenum screening) Applicable scenario: newly diagnosed patients without known anatomical abnormalities or lesions; The maximum torsion angle setting is shown in Table 1 below:

[0023] Table 1 2. Complex lesion treatment scenario (bleeding / stenosis / tumor) Applicable scenario: lesion areas requiring fine operation (such as ulcer hemostasis, stenosis dilation); Parameter adjustment strategy: Bleeding treatment: The knob angle limit is increased to 180° (short - term unlocking), and it is necessary to synchronously monitor the pressure sensor (threshold ≤ 8 N)4; Stenosis passage: Enable the "micro - motion mode", the single - turn torsion of the endoscope body ≤ 30°, and cooperate with the torque feedback system (refer to 6 ultrasonic guidance strategy); 3. Special anatomical structure scenario (intestinal curvature / post - operative adhesion); Applicable scenario: high - risk patients with a history of intestinal torsion, postoperative abdominal adhesions, etc.25; Dynamic adjustment plan: Pre - operative evaluation: Establish a 3D model based on CT / MRI data and pre - calculate the safe angle range; Intra - operative control: The torsion angle of the endoscope body is automatically reduced by 30% (e.g., the original 2 turns → 1.4 turns); Image fusion: Set region - specific thresholds under X - ray fluoroscopy (such as the ileocecal region ≤ 0.8 turns)6.

[0024] As a preferred embodiment of the present invention, specifically, PWM control is performed on the magnetorheological damper based on the rotation angle information, and the tactile resistance change is fed back to the operator, including, Calculate the target damping force according to the rotation angle information, and then perform PWM control on the magnetorheological damper according to the target damping force, causing a change in the magnetic field strength, and further dynamically adjusting the viscosity of the magnetorheological fluid. The change in the viscosity of the magnetorheological fluid causes a change in the tactile resistance and is fed back to the operator.

[0025] Refer to Figure 2 , in this preferred embodiment, PWM control is performed through the magnetorheological damper to cause a change in the tactile resistance and is fed back to the operator. The specific principle is as follows. The magnetorheological damper realizes the dynamic adjustment of the damping force by regulating the rheological characteristics of the magnetorheological fluid through a magnetic field. Its core components include: Magnetorheological fluid (MRF): It is composed of ferromagnetic particles suspended in a carrier liquid. Under the action of a magnetic field, the particles form a chain-like structure, resulting in a sharp increase in the viscosity of the liquid (which can be increased by an order of magnitude of 10^4).

[0026] Electromagnetic coil: Integrated on the knob rotating shaft, the magnetic field strength is controlled by current (in the range of 0-2T).

[0027] Damping channel: Designed as a multi-stage meandering structure to increase the magnetic field action area and improve the response efficiency.

[0028] As a preferred embodiment of the present invention, specifically, the reference mirror body torsion threshold and the reference knob torsion threshold of each operation part are dynamically updated based on the tissue resistance coefficient, including, Denote the tissue resistance coefficient as tissue_resistance, the reference mirror body torsion threshold or the reference knob torsion threshold before update as base_angle, then the updated mirror body torsion threshold or knob torsion threshold is base_angle * (1 - 0.15*tissue_resistance).

[0029] Specifically, the relevant code is as follows. def safety_threshold_calc(operation_phase, tissue_resistance): # Adjust the threshold according to different operation phases (such as reducing the allowable torsion amount when passing through the stenosis section) base_angle = {"esophagus":90°, "gastric antrum":120°, "duodenum":60°}[operation_phase] return base_angle * (1 - 0.15*tissue_resistance) # The tissue resistance coefficient comes from the feedback of the pressure sensor.

[0030] Full implementation of dynamic threshold update def update_thresholds(operation_phase, tissue_resistance): # Base threshold table (unit: degree) base_thresholds = { 'esophagus': {'scope': 540, 'knob': 90}, 'stomach': {'scope': 720, 'knob': 120}, 'duodenum': {'scope': 360, 'knob': 60} } # Dynamic adjustment formula adjusted = { 'scope': base_thresholds[operation_phase]['scope'] * (1 - 0.15 * tissue_resistance), 'knob': base_thresholds[operation_phase]['knob'] * (1 - 0.10 * tissue_resistance) } # Mechanical safety limit (hard constraint) adjusted['scope'] = min(adjusted['scope'], 1080) # Not exceeding 3 turns adjusted['knob'] = min(adjusted['knob'], 150) # Patent defined value return adjusted

[0031] As a preferred embodiment of the present invention, the method further includes setting a mechanical safety threshold, which includes that the scope cannot be continuously twisted in one direction by more than 3 turns, and the left / right rotation angle of the knob cannot exceed 150°. When any one of the rotation angle information or the scope torsion angle exceeds the mechanical safety threshold, a warning reminder is given.

[0032] In this preferred embodiment, according to the patented technology of the gastroscope knob design in 4 and the clinical operation specifications, it is recommended to set the following reference parameters: 1. Mechanical safety threshold: Lens body: Unidirectional continuous torsion ≤ 3 turns (1080°); Knob: Left / right rotation angle ≤ 150°; 2. Principle of dynamic adjustment: Dynamically adjust the threshold according to the anatomical complexity of the operation site, the patient's body type, and the nature of the lesion.

[0033] As a preferred embodiment of the present invention, specifically, the preset hierarchical early warning trigger mechanism includes, Insert the first intermediate value and the second intermediate value into the updated lens body torsion threshold from 0, and divide them into three intervals, which are recorded as the first safety interval, the first yellow early warning interval, and the first red early warning interval in order of size; insert the third intermediate value and the fourth intermediate value into the updated knob torsion threshold from 0, and divide them into three intervals, which are recorded as the second safety interval, the second yellow early warning interval, and the second red early warning interval in order of size; If at least one of the rotation angle information or the lens body torsion angle is within the red early warning interval, it is a third-level early warning, and corresponding early warning reminders are given; If both parameters of the rotation angle information or the lens body torsion angle are within the yellow early warning interval at the same time, it is a second-level early warning, and corresponding early warning reminders are given; If only a single parameter of the rotation angle information or the lens body torsion angle is within the yellow early warning interval, it is a first-level early warning, and corresponding early warning reminders are given.

[0034] In this preferred embodiment, by setting the above-mentioned hierarchical early warning trigger mechanism to give corresponding early warnings, it is convenient for the operator to understand the corresponding situation. Specifically, see Table 2 below,

[0035] Table 2 As a preferred embodiment of the present invention, the method further includes, Pre-store the historical curve of the torsion parameters in the expert standard mode, compare the deviation value between the current operation and the historical curve of the torsion parameters, and give an early warning reminder when the deviation value is greater than the deviation threshold.

[0036] In this preferred embodiment, the historical curve of the torsion parameters is floatingly displayed on the right side of the screen, and the deviation value between the current operation and the expert standard mode is compared (presented in a line chart + standard deviation shaded area), and when the deviation > 30%, it is automatically marked in red.

[0037] As a preferred embodiment of the present invention, the method further includes, when the forward pressure of the lens body recorded by the pressure information is greater than the pressure threshold, marking the mechanical conduction path in the form of a heat map for visual display.

[0038] In this preferred embodiment, when the forward pressure of the endoscope body is detected to be > 8 N, the mechanical conduction path is marked in the thermal image, and the resistance is displayed in the scene of the stenosis section, facilitating the operator to check and adjust.

[0039] In addition, as an aspect of the solution proposed by the present invention, the color coding rules of the thermal image can be added: Basic color scale division, Green (safe area), yellow (warning area), red (dangerous area), see the risk level assessment for details: Dynamic area marking, According to the real-time data of the sensor, high-risk operation points are marked in the thermal image: Proximal end of the endoscope body: If the pressure of the handle part > 5 N, a pressure gradient ring (red → yellow from inside to outside) is displayed.

[0040] Knob area: When the rotation speed changes suddenly (Δ≥20° / s), a flashing warning box is triggered.

[0041] Image fusion technology A semi-transparent and semi-reflective mirror optical system is adopted to superimpose the thermal image on the main endoscopic image with a transparency of 30%, ensuring that the observation of the lesion is not blocked.

[0042] The resolution of the thermal image is synchronized with the endoscopic image (≥1920×1080), and the key areas (such as the gastric angle and the descending part of the duodenum) adopt the local enlarged picture-in-picture mode.

[0043] Refer to Figure 3 , the present invention also proposes a gastroscope body and knob torsion monitoring device based on multi-modal sensing, including the following: A rotary encoder (high-precision rotary encoder (angle resolution ±0.5°)), a piezoresistive torque sensor (range 0-10 N·m) and a magnetorheological damper are arranged at the rotary end of the gastroscope, and an IMU (9-axis MEMS inertial measurement unit, sampling rate 200 Hz) is implanted at the endoscope body end; 1. Through multi-sensor data fusion (Sensor Fusion), the real-time posture of the endoscope body in three-dimensional space can be accurately reconstructed.

[0044] 2 Miniaturization and low power consumption The MEMS process can package the IMU to a size of 3×3×1 mm³ and embed it at the proximal end of the endoscope body without affecting the operation flexibility.

[0045] The typical power consumption ≤ 10 mW, supporting continuous operation for 8 hours (meeting the requirements of long-term surgery).

[0046] 3. Compatibility with the medical environment It can withstand high-temperature and high-pressure sterilization (such as steam sterilization at 134°C) and meets the hospital infection control standards.

[0047] Anti-electromagnetic interference design to avoid signal conflicts with devices such as endoscope light sources and electrosurgical knives 7.

[0048] Processing module, used to obtain the target working scenario, and determine the reference mirror body torsion threshold and the reference knob torsion threshold for each operating part according to the target working scenario; obtain the rotation angle information and pressure information through a rotary encoder and a piezoresistive torque sensor at the knob end, and obtain inertial measurement data through an IMU at the mirror body end; perform PWM control on the magnetorheological damper based on the rotation angle information to cause a change in tactile resistance and feedback it to the operator; determine the mirror body torsion angle based on the inertial measurement data; calculate the tissue resistance coefficient based on the pressure information, and dynamically update the reference mirror body torsion threshold and the reference knob torsion threshold for each operating part based on the tissue resistance coefficient; determine the current operating part, and obtain the dynamically updated mirror body torsion threshold and knob torsion threshold for the current operating part; carry out early warning reminders according to the size relationship between the rotation angle information and the knob torsion threshold, and the size relationship between the mirror body torsion angle and the mirror body torsion threshold according to the preset hierarchical early warning trigger mechanism.

[0049] Key position description: Knob end component layout (from near to far): Rotary encoder (R): Directly installed coaxially with the operating knob Piezoresistive torque sensor (F): Nested inside the knob transmission mechanism Magnetorheological damper (MRD): Circumferentially arranged around the transmission shaft Mirror body end component layout: IMU module: Embedded 10 cm from the proximal end of the mirror body (from the knob end) Flexible circuit: Arrange signal transmission lines along the longitudinal axis of the mirror body (diameter <0.3 mm) Mechanical connection features: Clearance between the damper and the transmission shaft: 0.1 - 0.3 mm (filled with magnetorheological fluid) Sensor sampling point spacing: Strain gauge arrays are set every 5° at the knob end IMU installation angle: The X-axis coincides with the axis of the mirror body, and the Z-axis is perpendicular to the operation plane 4. See Figure 4 .

[0050] 5. See Table 4 below for dimension parameter markings:

[0051] Table 4 The method and device for monitoring the gastroscope body and knob twisting based on multimodal sensing proposed by the present invention have the following overall advantages Clinical applicability: Compatible with the transformation and adaptation of mainstream brand gastroscopes (Olympus / Fujifilm). Precise early warning: Identify high-risk operations 1.2 - 1.8 seconds earlier than traditional empirical judgment). Teaching value: Can export operation curves for skill assessment.

[0052] In addition, in each embodiment of the present invention, each functional module can be integrated in a processing module, can also exist physically alone for each module, or two or more modules can be integrated in one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module.

[0053] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above method embodiments of the present invention, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or system, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0054] Although the description of the present invention has been quite detailed and several of the described embodiments have been described in particular, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but should be regarded as providing a broad interpretation of these claims in light of the prior art by reference to the appended claims, so as to effectively cover the intended scope of the present invention. In addition, the present invention has been described above with embodiments foreseeable by the inventor for the purpose of providing a useful description, and those non-substantive modifications to the present invention that are not currently foreseeable may still represent equivalent modifications of the present invention.

[0055] As described above, these are only the preferred embodiments of the present invention. The present invention is not limited to the above-described embodiments. As long as the same means are used to achieve the technical effects of the present invention, they should fall within the protection scope of the present invention. Within the protection scope of the present invention, various modifications and variations can be made to its technical solutions and / or implementation manners.

Claims

1. A method for monitoring the endoscope body and knob rotation based on multimodal sensing, characterized in that Including the following: Obtain the target working scenario, and determine the reference mirror body torsion threshold and the reference knob torsion threshold for each operating part according to the target working scenario; Obtain the rotation angle information and the pressure information at the knob end, and obtain the inertial measurement data at the mirror body end; Perform PWM control on the magnetorheological damper based on the rotation angle information to cause a change in tactile resistance and feedback it to the operator; Determine the mirror body torsion angle based on the inertial measurement data; Calculate the tissue resistance coefficient based on the pressure information, and dynamically update the reference mirror body torsion threshold and the reference knob torsion threshold for each operating part based on the tissue resistance coefficient; Determine the current operating part, and obtain the dynamically updated mirror body torsion threshold and the knob torsion threshold for the current operating part; According to the magnitude relationship between the rotation angle information and the knob torsion threshold, and the magnitude relationship between the mirror body torsion angle and the mirror body torsion threshold, issue a warning reminder according to the preset hierarchical warning trigger mechanism.

2. The method for monitoring the gastroscope body and knob torsion based on multi-modal sensing according to claim 1, wherein Specifically, the target working scenario includes a routine examination scenario, a complex lesion treatment scenario, and a special anatomical structure scenario; In the routine examination scenario, the reference mirror body torsion threshold for the esophageal operating part is 1.5 turns, the reference knob torsion threshold is 90°, the reference mirror body torsion threshold for the gastric cavity operating part is 2 turns, the reference knob torsion threshold is 120°, and the reference mirror body torsion threshold for the duodenal operating part is 1 turn, the reference knob torsion threshold is 60°, where one turn is 360°; In the complex lesion treatment scenario, in the bleeding treatment scenario, the reference knob torsion threshold is 180°, and it is necessary to ensure that the pressure information is lower than the pressure threshold of 8N. In the stenosis passage scenario, it is limited that the single torsion angle of the mirror body cannot exceed 30°; In the special anatomical structure scenario, a 3D model is established based on the patient's CT / MRI data, the reference mirror body torsion threshold and the reference knob torsion threshold are pre-calculated based on the 3D model, and it is limited that the single torsion angle of the mirror body cannot exceed 30°, and the reference mirror body torsion threshold of the ileocecal region in the image recognition is set to 0.8 turns based on the patient's X-ray image for image recognition.

3. The method for monitoring the gastroscope body and knob torsion based on multimodal sensing according to claim 1, wherein, Specifically, performing PWM control on the magnetorheological damper based on the rotation angle information to cause a change in tactile resistance and feedback it to the operator includes, Calculating the target damping force according to the rotation angle information, and then performing PWM control on the magnetorheological damper according to the target damping force, causing a change in the magnetic field strength, and further dynamically adjusting the viscosity of the magnetorheological fluid. The change in the viscosity of the magnetorheological fluid causes a change in tactile resistance and feedbacks it to the operator.

4. The method for monitoring the endoscope body and knob rotation based on multimodal sensing according to claim 1, wherein Specifically, dynamically updating the reference mirror body torsion threshold and the reference knob torsion threshold for each operating part based on the tissue resistance coefficient includes, Denote the tissue resistance coefficient as tissue_resistance, and the reference mirror body torsion threshold or the reference knob torsion threshold before update as base_angle, then the updated mirror body torsion threshold or knob torsion threshold is base_angle * (1 - 0.15*tissue_resistance).

5. The method for monitoring the gastroscope body and knob torsion based on multimodal sensing according to claim 1, wherein The method further includes, Set a mechanical safety threshold, which includes that the mirror body cannot be continuously twisted in one direction by more than 3 turns, and the left / right rotation angle of the knob cannot exceed 150°. When any one of the rotation angle information or the mirror body twist angle exceeds the mechanical safety threshold, a warning reminder is given.

6. The method for monitoring the gastroscope body and knob torsion based on multimodal sensing according to claim 1, wherein Specifically, the preset hierarchical warning trigger mechanism includes: Insert the first intermediate value and the second intermediate value into the mirror body twist threshold from 0 to the updated value, and divide them into three intervals, which are recorded as the first safety interval, the first yellow warning interval, and the first red warning interval in order of size; insert the third intermediate value and the fourth intermediate value into the knob twist threshold from 0 to the updated value, and divide them into three intervals, which are recorded as the second safety interval, the second yellow warning interval, and the second red warning interval in order of size; If at least one parameter of the rotation angle information or the mirror body twist angle is in the red warning interval, it is a third-level warning, and a corresponding warning reminder is given; If both parameters of the rotation angle information or the mirror body twist angle are in the yellow warning interval, it is a second-level warning, and a corresponding warning reminder is given; If only a single parameter of the rotation angle information or the mirror body twist angle is in the yellow warning interval, it is a first-level warning, and a corresponding warning reminder is given.

7. The method for monitoring the endoscope body and knob rotation based on multimodal sensing according to claim 6, characterized in that, The method further includes: Pre-store the historical curve of the twist parameters in the expert standard mode, compare the deviation value between the current operation and the historical curve of the twist parameters, and give a warning reminder when the deviation value is greater than the deviation threshold.

8. The method for monitoring the endoscope body and knob rotation based on multimodal sensing according to claim 1, wherein, The method further includes that when the forward pressure of the mirror body recorded by the pressure information is greater than the pressure threshold, the mechanical conduction path is marked by a heat map for visual display.

9. Gastroscope body and knob torsion monitoring device based on multimodal sensing, characterized in that It includes the following: A rotary encoder, a piezoresistive torque sensor, and a magnetorheological damper are set at the twist end of the gastroscope, and an IMU is implanted at the mirror body end of the gastroscope; A processing module is used to obtain the target action scenario, and determine the reference mirror body twist threshold and the reference knob twist threshold of each operation part according to the target action scenario; Obtain the rotation angle information and pressure information through the rotary encoder and the piezoresistive torque sensor at the knob end, and obtain the inertial measurement data through the IMU at the mirror body end; Perform PWM control on the magnetorheological damper based on the rotation angle information to cause a change in tactile resistance and feedback it to the operator; Determine the mirror body twist angle based on the inertial measurement data; Calculate the tissue resistance coefficient based on the pressure information, and dynamically update the reference mirror body twist threshold and the reference knob twist threshold of each operation part based on the tissue resistance coefficient; Determine the current operation part, and obtain the dynamically updated mirror body twist threshold and knob twist threshold of the current operation part; According to the size relationship between the rotation angle information and the knob twist threshold, and the size relationship between the mirror body twist angle and the mirror body twist threshold, give a warning reminder according to the preset hierarchical warning trigger mechanism.

Images