An augmented reality eyewear system and infusion pump drip rate control method

Abstract

This invention discloses an augmented reality glasses system and an infusion pump drip rate control method. The system includes: a data acquisition module for acquiring multimodal data of the infusion pump process; a spatial positioning module for determining the region of interest and calculating the physical parameters driving the infusion fluid; an image processing module for quantifying and evaluating the state indicators of the irrigation fluid based on image sequence data; a calculation processing module for calculating the estimated actual volumetric flow rate of the infusion pump and calculating the theoretical volumetric flow rate; and a strategy generation module for inputting the calculated irrigation fluid state indicators and / or the estimated actual volumetric flow rate of the infusion pump into a decision model to generate drip rate adjustment suggestions and / or drip rate control commands. This invention, by integrating multimodal sensing data and establishing a gravity-driven physical model, achieves objective evaluation of the drip rate, physical identification of abnormal causes, and intelligent adjustment guided by augmented reality, thereby improving the safety, accuracy, and efficiency of the bladder irrigation process.

Description

Technical Field

[0001] This invention relates to the fields of urology and intelligent control technology, and in particular to an augmented reality glasses system and a method for controlling the drip rate of an infusion pump. Background Technology

[0002] Bladder irrigation is frequently required during urological surgeries and postoperative management, such as transurethral resection of the prostate (TURP), transurethral resection of bladder tumor (TURBT), cystoscopy, and continuous postoperative bladder irrigation (CBI). In clinical practice, the irrigation fluid drip rate / flow rate is usually adjusted by healthcare professionals based on experience, observation of the irrigation fluid color, and drainage status. Current techniques suffer from at least the following problems: highly subjective drip rate adjustment; delayed detection of risk events; significant intraoperative visual and operational burden; lack of intelligent closed-loop and standardized guidance; and difficulty in distinguishing physical causes such as insufficient potential energy, excessively tight clamps, tubing kinks, or blood clot blockage when flow rates are abnormal. Therefore, there is an urgent need for a system that can identify the irrigation fluid status in real time during / after surgery, estimate drip rate and flow rate, provide standardized drip rate adjustment suggestions, and incorporate gravitational potential energy differences based on spatial positioning to achieve interpretable physical discrimination. Summary of the Invention

[0003] To address the shortcomings of the existing technologies, the technical problem to be solved by this invention is: to propose an augmented reality glasses system and an infusion pump drip rate control method, which can achieve: real-time identification of the flushing fluid status; real-time estimation and target range control of drip rate / flow rate; early warning of risks such as blockage, poor backflow, and bladder overfilling; guide medical staff to complete drip rate adjustment under aseptic workflow through augmented reality overlay; and integrate gravitational potential energy and fluid dynamics models to achieve physical causal discrimination of abnormal flow rate, thereby improving clinical interpretability and safety.

[0004] One technical solution adopted by this invention is to provide an augmented reality glasses system, which includes the following modules: The data acquisition module is used to acquire multimodal data of the infusion pump during the infusion process; The spatial positioning module is used to determine the region of interest and calculate the physical parameters driving the infusion fluid based on spatial positioning information. The image processing module is used to quantitatively evaluate the state indicators of the flushing fluid based on image sequence data in the multimodal data. The calculation and processing module is used to calculate the estimated actual volumetric flow rate of the infusion pump based on multimodal data, and to calculate the theoretical volumetric flow rate based on the flow model. The strategy generation module is used to input the calculated flushing fluid status index and / or the estimated actual volumetric flow rate of the infusion pump into the decision model to generate drip rate adjustment suggestions and / or drip rate control instructions, and to overlay and display the target drip rate range and operation guidance in a real scene through a head-mounted near-eye display device.

[0005] Furthermore, the system also includes: The automatic adjustment module is used to determine the abnormal flow rate based on the estimated actual volumetric flow rate of the infusion pump, and upon receiving the drip rate control command, to perform automatic control on the infusion pump and / or actuator according to the drip rate control command, or to guide manual adjustment through an augmented reality interface.

[0006] Furthermore, the data acquisition module includes the following sub-modules: The image acquisition submodule is used to acquire image sequences of the dripping area and the drainage fluid area through an image acquisition device; The multimodal interface submodule is used to connect to external sensors via a multimodal sensing interface to collect external sensor data, including pipeline pressure data, and to obtain patient risk parameters. Furthermore, the spatial positioning module includes the following submodules: The region generation submodule is used to acquire spatial location information of the dripping bucket, clamping part and / or drainage container, and determine the region of interest of the dripping bucket, clamping part and / or drainage container based on the spatial location information; The height difference calculation submodule is used to calculate the height difference driven by gravity. ; in, Indicates the liquid level in the flushing fluid container. Indicates the height of the patient's urinary catheter inlet. This indicates the height difference driven by gravity.

[0007] Furthermore, the image processing module includes the following sub-modules: The feature extraction submodule is used to extract color features, transparency / turbidity features, flocculent / blood clot features, and bubble features from the image sequence of the drainage fluid area; The flushing fluid status index generation submodule is used to generate one or more of the following: bloodiness index (HI), turbidity index (TI), and clot risk index (CI) based on features extracted from the image sequence of the drainage fluid area.

[0008] Furthermore, the computation processing module includes the following sub-modules: The drip fusion submodule is used for visual drip counting based on image sequences of the dripping area, and then fused with external drip counting data to calculate the current drip rate and convert it into the actual volumetric flow rate. ; The theoretical volumetric flow rate calculation submodule is used to calculate the gravity-driven height difference. Calculate the theoretical volumetric flow rate: ; ; in, Indicates theoretical speed, This represents the correction factor, and g represents the acceleration due to gravity. The height difference is represented by gravity, A represents the effective cross-sectional area of ​​the pipe, and R represents the equivalent flow resistance coefficient. This represents the theoretical volumetric flow rate.

[0009] Furthermore, the strategy generation module includes the following sub-modules: The strategy model input submodule is used to input one or more of the bloodiness index HI, turbidity index TI, and clot risk index CI, and the actual volumetric flow rate. The theoretical volumetric flow rate Pipeline pressure data and patient risk parameter input strategy model; The strategy output submodule is used to output the target drip rate range and drip rate adjustment suggestions and / or drip rate control instructions; the control strategy includes a rule base and / or a learning model.

[0010] Furthermore, the automatic adjustment module includes the following sub-modules: The flow rate anomaly detection submodule is used to determine the flow rate based on the actual volumetric flow rate. The theoretical volumetric flow rate The pipeline pressure data is used to identify flow velocity anomalies. The augmented reality submodule is used to overlay and display the target drip rate range, current drip rate, flow rate anomaly information, and operation guidance based on the flow rate anomaly judgment in a real scene through a head-mounted near-eye display device. The automatic adjustment discrimination submodule is used to compare the operation guidance based on the flow rate anomaly discrimination with the preset automatic adjustment safety conditions to determine whether the preset automatic adjustment safety conditions are met. If they are met, the automatic adjustment submodule is entered; otherwise, manual adjustment is prompted. The automatic adjustment submodule is used to drive the infusion pump and / or actuator to automatically adjust the drip rate when the drip rate control command is received.

[0011] The second technical solution adopted in this invention is a method for controlling the drip rate of an infusion pump, which includes the following steps: S1: Collect multimodal data on the infusion process of the infusion pump; S2: Based on spatial positioning information, determine the region of interest and calculate the physical parameters driving the infusion fluid; S3: Quantitatively evaluate the state indicators of the flushing fluid based on the image sequence data in the multimodal data; S4: Calculate the estimated actual volumetric flow rate of the infusion pump based on multimodal data, and calculate the theoretical volumetric flow rate based on the flow model; S5: Input the calculated flushing fluid status index and / or the estimated actual volumetric flow rate of the infusion pump into the decision model to generate drip rate adjustment suggestions and / or drip rate control instructions, and overlay the target drip rate range and operation guidance in the real scene through an augmented reality interface.

[0012] Furthermore, the method also includes the following steps: S6: Based on the estimated actual volumetric flow rate of the infusion pump, an abnormal flow rate is determined, and upon receiving the drip rate control command, automatic control is performed on the infusion pump and / or actuator according to the drip rate control command, or manual adjustment is guided through an augmented reality interface.

[0013] The augmented reality glasses system of the present invention has at least the following beneficial effects: 1. Achieve objective and precise drip rate control: By fusing image and multi-channel sensor data, the drip rate is estimated and adjusted in real time and closed-loop, replacing subjective judgment based on experience, thus improving control accuracy and stability.

[0014] 2. Early identification of flushing fluid status and risks: Features such as color, turbidity, and clots are automatically extracted from images and quantitative indicators are generated. Changes in these indicators can be continuously monitored, thereby enabling early identification and warning of risks such as increased bloodiness and clot formation.

[0015] 3. Identify the cause of anomalies based on physical models: By calculating the gravity-driven pressure difference through spatial positioning and establishing a theoretical volumetric flow rate model, and by comparing the actual volumetric flow rate with the theoretical volumetric flow rate, combined with pipeline pressure, it is possible to distinguish whether the anomaly is caused by insufficient gravity or pipeline blockage, making the decision more interpretable and safer.

[0016] 4. Provides intelligent guidance and safe closed-loop adjustment: By overlaying and displaying targets, alarms and operation guidance through an augmented reality interface, the operational burden is reduced; the control strategy is generated by integrating rules and learning models, and the actuator is driven to achieve automatic adjustment under strict safety constraints, thereby improving efficiency and safety. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1This is a block diagram of the augmented reality glasses system of the present invention.

[0018] Figure 2 This is a flowchart of the infusion pump drip rate control method of the present invention. Detailed Implementation

[0019] The invention will now be further described with reference to the accompanying drawings.

[0020] Please see Figure 1 The present invention provides an augmented reality glasses system. The system includes the following modules: The data acquisition module is used to collect multimodal data of the infusion pump infusion process. As the "sensory" input end of the augmented reality glasses system, this data acquisition module is responsible for acquiring all raw information for analysis and decision-making, including visual images and physical sensor data, and is the basis for multimodal information fusion.

[0021] This data acquisition module may include the following sub-modules: The image acquisition submodule is used to acquire image sequences of the dripping area and the drainage fluid area using an image acquisition device. This image acquisition submodule clarifies the specific source and target of the system's "visual data." Its function is to continuously acquire image sequences of the two key areas, the dripping bucket (for drop counting) and the drainage bag (for assessing liquid properties), providing raw materials for subsequent visual drop counting and state recognition.

[0022] The multimodal interface submodule is used to connect to external sensors via a multimodal sensing interface to collect external sensor data, including pipeline pressure data, and to obtain patient risk parameters. This multimodal interface submodule defines the access channel for the system's "physical sensor data." Its function is to serve as a universal hardware interface, capable of receiving and integrating real-time measurement data from various external devices such as drop sensors, pressure sensors, and liquid level sensors, complementing and verifying visual data.

[0023] This image acquisition submodule and multimodal interface submodule can be integrated into a head-mounted glasses device. This device can acquire image sequences of the drip chamber and drainage fluid, and at the same time, it can access external sensor data (drop counting, pipeline pressure, liquid level / weight, temperature, clamp opening, etc.) through the multimodal data interface module.

[0024] The spatial positioning module is used to determine the region of interest (ROI) and calculate the physical parameters driving the infusion fluid based on spatial positioning information. As the core of the system's "spatial perception," this module's role is not only to determine the ROI for image analysis, but more importantly, to calculate the gravity-driven height difference. This provides core input parameters for the subsequent establishment of a flow model based on fluid dynamics, transforming spatial geometric information into a physically meaningful driving potential energy difference.

[0025] This spatial positioning module may include the following sub-modules: The region generation submodule is used to acquire the spatial location information of the dripping bucket, clamping device, and / or drainage container, and based on the spatial location information, determine the region of interest (ROI) of the dripping bucket, clamping device, and / or drainage container. The role of this region generation submodule is to implement an "attention mechanism." By locating key components such as the dripping bucket and clamping device in three-dimensional space, and accordingly defining the corresponding "ROI" on the two-dimensional image, subsequent image processing algorithms (such as drip counting and feature extraction) can focus on the effective area, eliminate background interference, and significantly improve the accuracy and efficiency of processing.

[0026] The height difference calculation submodule is used to calculate the height difference driven by gravity. ; in, Indicates the liquid level in the flushing fluid container. Indicates the height of the patient's urinary catheter inlet. This indicates the gravity-driven height difference. This height difference calculation submodule obtains the liquid level height in the flushing fluid bag. and the height of the inlet inside the patient's body And calculate the difference. This submodule outputs the most fundamental and critical driving parameters of the entire fluid dynamics model. There is no accurate... Subsequent theoretical volumetric flow rate calculation ( This will lose its physical basis.

[0027] The image processing module is used to quantitatively evaluate the state indicators of the flushing fluid based on image sequence data in multimodal data. This image processing module is specifically designed to process image data. Its function is to transform subjective flushing fluid properties (such as color, turbidity, and presence or absence of clots) observed by the naked eye into quantifiable and calculable objective numerical indicators (such as HI, TI, CI), thereby realizing the digital and objective evaluation of the flushing fluid state.

[0028] This image processing module may include the following sub-modules: The feature extraction submodule is used to extract color features, transparency / turbidity features, flocculent / blood clot features, and bubble features from the image sequence of the drainage fluid area. This feature extraction submodule can use image processing algorithms to extract several key visual features (color, turbidity, clots, and bubbles) that are most relevant to clinical judgment from the complex image of the drainage fluid area, and convert these features from pixel values ​​into a series of feature vectors or values ​​that can be used for calculation.

[0029] The irrigation fluid status index generation submodule generates one or more of the following: Bloodiness Index (HI), Turbidity Index (TI), and Clot Risk Index (CI), based on features extracted from image sequences of the drainage fluid area. This submodule maps the abstract features extracted in the previous step into standardized assessment indicators with clear clinical significance. For example, color features (especially the a* component) are calculated as the "Bloodiness Index (HI)". These indicators (HI, TI, CI) enable the computer to classify and quantify "bleeding severity," "turbidity," and "clot risk" like a doctor, providing direct and comparable input parameters for subsequent intelligent decision-making.

[0030] The color / bloodiness identification process involves white balance and exposure compensation of the drainage fluid ROI, followed by conversion to Lab or HSV color space. The bloodiness index HI satisfies: HI = norm(a*ROI), where a*ROI is the mean or weighted mean of the a* component of the ROI.

[0031] Turbidity / transparency identification is based on the calculation of the turbidity index TI, which is derived from contrast attenuation, scattering features, and texture energy. For example, TI = 1. norm(CROI), where CROI is the contrast or high-frequency energy within the ROI.

[0032] Agglomerate / flocculent identification is based on temporal optical flow, morphological features, or segmentation models to detect agglomerate targets, obtain agglomerate area / quantity / velocity, etc., and fuse them to form agglomerate risk index CI.

[0033] Bubble identification is based on the highlighting of circular edges and motion characteristics. Bubbles are then removed or downweighted in turbidity and clot estimation to avoid misjudgment.

[0034] Flow anomalies are detected by identifying abnormal drip intervals, discontinuous dripping, and sudden decreases in drip rate, serving as risk inputs or criteria for judging sensor effectiveness.

[0035] The calculation and processing module is used to calculate the estimated actual volumetric flow rate of the infusion pump based on multimodal data and to calculate the theoretical volumetric flow rate based on the flow model. This module is the core of the system's "data fusion and physical modeling." Its functions are: firstly, to fuse visual drip counting and other sensor data to calculate a more reliable estimate of the actual volumetric flow rate. Secondly, it utilizes the spatial positioning module. The theoretical volumetric flow rate under the current physical conditions is calculated using fluid mechanics formulas. . and The comparison is the basis for subsequent intelligent discrimination and physical causal analysis.

[0036] This calculation and processing module may include the following sub-modules: The drip fusion submodule is used for visual drip counting based on image sequences of the dripping area, and then fused with external drip counting data to calculate the current drip rate and convert it into the actual volumetric flow rate. This drip fusion submodule obtains the visual drip rate by detecting dripping events within the dripping chamber's ROI using visual drip counting. Infrared dropper output drip rate , fusion drip rate: Where w represents the change with visual confidence, and more preferably, w is adaptively determined by the gain of the gating network or Kalman filter; Indicates visual drip rate. This indicates the drop rate output by the infrared dropper. Its purpose is to improve the robustness and accuracy of flow measurement. It does not rely solely on visual drop counting (which may be affected by obstruction or lighting) or external sensors (which may malfunction or drift), but instead fuses information from both (e.g., through weighted averaging or Kalman filtering) to obtain a more stable and reliable estimate of the current actual volumetric flow rate. This is a reliable data source for the system to perceive the "current situation".

[0037] The theoretical volumetric flow rate calculation submodule is used to calculate the gravity-driven height difference. Calculate the theoretical volumetric flow rate: ; ; in, Indicates theoretical speed, This represents the correction factor, and g represents the acceleration due to gravity. The height difference is represented by gravity, A represents the effective cross-sectional area of ​​the pipe, and R represents the equivalent flow resistance coefficient. This represents the theoretical volumetric flow rate. The purpose of this theoretical volumetric flow rate calculation submodule is to establish an "ideal" physical reference benchmark for the system, by utilizing the height difference. By combining parameters such as the pipe cross-sectional area A and the flow resistance coefficient R, the theoretically expected flow rate under current gravity-driven conditions with an unobstructed pipe can be calculated using fluid dynamics formulas. .this It determines the current actual traffic. Is the physical scale normal, and if not, what could be causing it to be abnormal?

[0038] It is worth mentioning that, in order to solve the problem of fusion between image sequences and sensor data with different frequencies and delays, this computational processing module can perform time synchronization and alignment of data from different modalities: First, a timestamp t from a unified clock source is added to each image frame and each channel of sensor data; Then in the sliding window W=[t Within [Tw, t], buffer the modal data, perform interpolation / preservation on low-frequency data, and downsample / aggregate on high-frequency data to obtain the aligned {I(t), Z_sen(t)}; Finally, if there is communication delay or packet loss, late packet compensation, retransmission, or nearest neighbor replacement are used, and a validity flag (flag) and confidence level (conf) are generated for each data stream.

[0039] In addition, when lighting changes or occlusion occurs, the computational processing module adjusts the visual weight w based on the detection confidence of ROI (conf_roi) and image quality indicators (sharpness, exposure, reflectivity, etc.). When conf_roi is below a threshold, it switches to an estimation mode dominated by external sensor data. At the same time, the AR interface of the glasses can display a message "Occlusion / Illumination Abnormality".

[0040] The strategy generation module is used to input the calculated flushing fluid status indicators and / or the estimated actual volumetric flow rate of the infusion pump into the decision model to generate drip rate adjustment suggestions and / or drip rate control commands. This strategy generation module comprehensively analyzes the output results of all signed modules (status indicators, actual flow rate, theoretical flow rate, pressure, etc.), inputs them into a preset rule base or machine learning model, and finally outputs specific and actionable guidance suggestions (such as "speed up / slow down") or direct control commands, completing a closed loop from "perception analysis" to "decision output".

[0041] This strategy generation module may include the following sub-modules: The strategy model input submodule is used to input one or more of the bloodiness index HI, turbidity index TI, and clot risk index CI, and the actual volumetric flow rate. The theoretical volumetric flow rate The input submodule of this strategy model, which collects, aligns, and synchronizes data from different processing paths (state indicators obtained from image processing, actual and theoretical flow rates obtained from computational processing, pressure data directly read from sensors, and the patient's own risk parameters), forms a data package that comprehensively describes the current flushing system status, providing complete "contextual information" for the decision model.

[0042] A strategy output sub-module for outputting a target drip rate range and drip rate adjustment suggestions and / or drip rate control instructions; the control strategy includes a rule base and / or a learning model. The function of this strategy output sub-module is to execute decision logic and output executable results. It processes the input state vector using a built-in rule base (if-then rules based on a clinical knowledge base) and / or a learning model (an AI model trained based on historical data), and finally outputs two types of results: one is drip rate adjustment suggestions with a safety range for medical staff to view; the other is specific drip rate control instructions for machine execution.

[0043] In addition, this rule base can take the flushing system state vector S(t) as input and output the target drip rate range , , adjustment direction, and risk warning level. Preferably, a hierarchical threshold and hysteresis mechanism are adopted to avoid frequent jumps: a) Hemorrhagic grading: HI ∈ [0,1], set H1 < H2 < H3, and classify hemorrhage into three grades: mild / moderate / severe; when HI rises across levels and CI rises, increase Dr_max and prompt "Strengthen flushing, pay attention to clots"; b) Clot trend: The condition for CI to rise is CI(t) CI(t ΔT) ≥ β or dCI / dt ≥ β'; when CI reaches C2 and the upward trend continues for N windows, trigger a "high risk of clot" warning, and suggest temporarily increasing the drip rate or prompt manual flushing / squeezing the drainage tube; c) Turbidity / muddiness: When TI exceeds T2 and the bubble ratio is low, it is prompted that there may be flocs or precipitates, and it is recommended to check the pipeline and increase the weight for clot estimation; d) Pressure-flow coupling: When P increases and decreases, increase the "blockage / kinking of the tube" risk level; when P increases but remains unchanged, prompt "bladder pressure increase or reflux restriction", and it is recommended to check the height of the drainage bag and the position of the urinary catheter.

[0044] The above thresholds H1, H2, H3, C2, T2, β, β', etc. are configured by the background end, and different parameter groups can be configured for different liquids (normal saline / drug-containing flushing solution), different drip chamber models, and different surgical types.

[0045] This system may also include the following modules: The automatic adjustment module is used to identify flow rate anomalies based on the estimated actual volumetric flow rate of the infusion pump, and upon receiving the drip rate control command, to automatically control the infusion pump and / or perform manual adjustment according to the command, or guide manual adjustment through an augmented reality interface. This automatic adjustment module, based on all the above modules, adds execution and interaction components, upgrading the system from an analysis-aid system to a complete monitoring-decision-execution closed-loop control system.

[0046] This automatic adjustment module may include the following sub-modules: The flow rate anomaly detection submodule is used to determine the flow rate based on the actual volumetric flow rate. The theoretical volumetric flow rate The flow velocity anomaly detection submodule uses the pipeline pressure data to determine flow velocity anomalies. , The relationship between these three key physical quantities—pipeline pressure P, pipeline pressure, and pipeline pressure—is logically determined. The specific logic for this determination can be as follows: To improve the operability of the judgment, the flow ratio η is defined as: / ( +ε).

[0047] a) Insufficient gravity drive: When η<γ1 and P is in the low pressure range [0, P_low], it is determined that the gravity drive is insufficient, prompting the infusion stand to be raised or the height of the flushing fluid container to be adjusted. b) Tight clamp / bent tube / blockage: When η<γ2 and P≥P_high, it is determined that the flow resistance of the pipeline is abnormal; if the clamp opening is small, the prompt "relax clamp" will be given first, otherwise the prompt "check for bent tube / blood clot blockage" will be given. c) Sensing anomaly: When and If the long-term deviation exceeds δ and the image quality is good, it indicates that the dropper may be loose or drifting; when the P signal is saturated or lost, it enters degrade mode.

[0048] Among them, γ1, γ2, P_low, P_high, and δ are configurable parameters that can be initialized by equipment calibration or departmental experience and can be uniformly distributed in the background.

[0049] The augmented reality submodule is used to overlay and display the target drip rate range, current drip rate, flow rate anomaly information, and operation guidance based on the flow rate anomaly judgment onto a real scene using a head-mounted near-eye display device. The purpose of this augmented reality submodule is to achieve intuitive and immersive human-computer interaction. It precisely overlays abstract numerical information and decision results (target drip rate, current drip rate, anomaly type, operation guidance) onto corresponding objects (such as drip chambers, infusion tubes, and regulators) in a visual manner (such as numbers, color blocks, arrows, and text prompts) on the real scene seen by medical staff, greatly reducing the cognitive load and time cost of information understanding and operation execution.

[0050] The automatic adjustment discrimination submodule is used to compare the operation guidance based on the abnormal flow rate with the preset safety conditions for automatic adjustment to determine whether the preset safety conditions for automatic adjustment are met. If they are met, the automatic adjustment submodule is entered; otherwise, manual adjustment is prompted. This automatic adjustment discrimination submodule judges based on a series of preset hard safety conditions (such as whether the pressure exceeds the limit, whether the sensor is malfunctioning, whether the image is severely obstructed, etc.). Only when all conditions are met is automatic control mode allowed; otherwise, automatic control is immediately locked and manual intervention is prompted. This is a key design to ensure patient safety.

[0051] Upon receiving the drip rate control command, the automatic adjustment submodule drives the infusion pump and / or actuator to automatically adjust the drip rate. After passing a safety check, this automatic adjustment submodule converts the control command issued by the strategy generation module into a specific motor drive signal to control the opening and closing of the tubing clamp actuator or the rotational speed of the infusion pump, thereby actually changing the tubing resistance or driving pressure to achieve precise and automatic adjustment of the drip rate, completing the entire control closed loop.

[0052] Please see Figure 2 The present invention provides a method for controlling the drip rate of an infusion pump, as shown in the flowchart above. This method is used in the aforementioned augmented reality glasses system and may specifically include the following steps: S1: Collect multimodal data on the infusion process of the infusion pump; S2: Based on spatial positioning information, determine the region of interest and calculate the physical parameters driving the infusion fluid; S3: Quantitatively evaluate the state indicators of the flushing fluid based on the image sequence data in the multimodal data; S4: Calculate the estimated actual volumetric flow rate of the infusion pump based on multimodal data, and calculate the theoretical volumetric flow rate based on the flow model; S5: Input the calculated flushing fluid status index and / or the estimated actual volumetric flow rate of the infusion pump into the decision model to generate drip rate adjustment suggestions and / or drip rate control instructions, and overlay the target drip rate range and operation guidance in the real scene through an augmented reality interface.

[0053] The method may also include the following steps: S6: Based on the estimated actual volumetric flow rate of the infusion pump, an abnormal flow rate is determined, and upon receiving the drip rate control command, automatic control is performed on the infusion pump and / or actuator according to the drip rate control command, or manual adjustment is guided through an augmented reality interface.

[0054] The above description merely illustrates preferred embodiments of the present invention and is quite specific and detailed; however, it should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of the present invention. Therefore, the scope of protection of this invention should be determined by the appended claims.

Claims

1. An augmented reality glasses system, characterized in that, The system includes the following modules: The data acquisition module is used to acquire multimodal data of the infusion pump during the infusion process; The spatial positioning module is used to determine the region of interest and calculate the physical parameters driving the infusion fluid based on spatial positioning information. The image processing module is used to quantitatively evaluate the state indicators of the flushing fluid based on image sequence data in the multimodal data. The calculation and processing module is used to calculate the estimated actual volumetric flow rate of the infusion pump based on multimodal data, and to calculate the theoretical volumetric flow rate based on the flow model. The strategy generation module is used to input the calculated flushing fluid status index and / or the estimated actual volumetric flow rate of the infusion pump into the decision model to generate drip rate adjustment suggestions and / or drip rate control instructions, and to overlay and display the target drip rate range and operation guidance in a real scene through a head-mounted near-eye display device.

2. The augmented reality glasses system as described in claim 1, characterized in that, The system also includes: The automatic adjustment module is used to determine the abnormal flow rate based on the estimated actual volumetric flow rate of the infusion pump, and upon receiving the drip rate control command, to perform automatic control on the infusion pump and / or actuator according to the drip rate control command, or to guide manual adjustment through an augmented reality interface.

3. The augmented reality glasses system as described in claim 2, characterized in that, The data acquisition module includes the following sub-modules: The image acquisition submodule is used to acquire image sequences of the dripping area and the drainage fluid area through an image acquisition device; The multimodal interface submodule is used to connect to external sensors through a multimodal sensing interface, collect external sensing data including pipeline pressure data, and obtain patient risk parameters.

4. The augmented reality glasses system as described in claim 3, characterized in that, The spatial positioning module includes the following sub-modules: The region generation submodule is used to acquire spatial location information of the dripping bucket, clamping part and / or drainage container, and determine the region of interest of the dripping bucket, clamping part and / or drainage container based on the spatial location information; The height difference calculation submodule is used to calculate the height difference driven by gravity. ； in, Indicates the liquid level in the flushing fluid container. Indicates the height of the patient's urinary catheter inlet. This indicates the height difference driven by gravity.

5. The augmented reality glasses system as described in claim 4, characterized in that, The image processing module includes the following sub-modules: The feature extraction submodule is used to extract color features, transparency / turbidity features, flocculent / blood clot features, and bubble features from the image sequence of the drainage fluid area; The flushing fluid status index generation submodule is used to generate one or more of the following: bloodiness index (HI), turbidity index (TI), and clot risk index (CI) based on features extracted from the image sequence of the drainage fluid area.

6. The augmented reality glasses system as described in claim 5, characterized in that, The computation processing module includes the following sub-modules: The drip fusion submodule is used for visual drip counting based on image sequences of the dripping area, and then fuses them with external drip counting data to calculate the current drip rate and convert it into the actual volumetric flow rate. ; The theoretical volumetric flow rate calculation submodule is used to calculate the gravity-driven height difference. Calculate the theoretical volumetric flow rate: ； ； in, Indicates theoretical speed, This represents the correction factor, and g represents the acceleration due to gravity. The height difference is represented by gravity, A represents the effective cross-sectional area of ​​the pipe, and R represents the equivalent flow resistance coefficient. This represents the theoretical volumetric flow rate.

7. The augmented reality glasses system as described in claim 6, characterized in that, The strategy generation module includes the following sub-modules: The strategy model input submodule is used to input one or more of the bloodiness index HI, turbidity index TI, and clot risk index CI, and the actual volumetric flow rate. The theoretical volumetric flow rate Pipeline pressure data and patient risk parameter input strategy model; The strategy output submodule is used to output the target drip rate range and drip rate adjustment suggestions and / or drip rate control instructions; the control strategy includes a rule base and / or a learning model.

8. The augmented reality glasses system as described in claim 7, characterized in that, The automatic adjustment module includes the following sub-modules: The flow rate anomaly detection submodule is used to determine the flow rate based on the actual volumetric flow rate. The theoretical volumetric flow rate The pipeline pressure data is used to identify flow velocity anomalies. The augmented reality submodule is used to overlay and display the target drip rate range, current drip rate, flow rate anomaly information, and operation guidance based on the flow rate anomaly judgment in a real scene through a head-mounted near-eye display device. The automatic adjustment discrimination submodule is used to compare the operation guidance based on the flow rate anomaly discrimination with the preset automatic adjustment safety conditions to determine whether the preset automatic adjustment safety conditions are met. If they are met, the automatic adjustment submodule is entered; otherwise, manual adjustment is prompted. The automatic adjustment submodule is used to drive the infusion pump and / or actuator to automatically adjust the drip rate when the drip rate control command is received.

9. A method for controlling the drip rate of an infusion pump, characterized in that, The method includes the following steps: S1: Collect multimodal data on the infusion process of the infusion pump; S2: Based on spatial positioning information, determine the region of interest and calculate the physical parameters driving the infusion fluid; S3: Quantitatively evaluate the state indicators of the flushing fluid based on the image sequence data in the multimodal data; S4: Calculate the estimated actual volumetric flow rate of the infusion pump based on multimodal data, and calculate the theoretical volumetric flow rate based on the flow model; S5: Input the calculated flushing fluid status index and / or the estimated actual volumetric flow rate of the infusion pump into the decision model to generate drip rate adjustment suggestions and / or drip rate control instructions, and overlay the target drip rate range and operation guidance in the real scene through an augmented reality interface.

10. The infusion pump drip rate control method as described in claim 9, characterized in that, The method also includes the following steps: S6: Based on the estimated actual volumetric flow rate of the infusion pump, an abnormal flow rate is determined, and upon receiving the drip rate control command, automatic control is performed on the infusion pump and / or actuator according to the drip rate control command, or manual adjustment is guided through an augmented reality interface.

Images