An image recognition-based real-time monitoring method and system for a static therapy infusion process

Abstract

The present application relates to the technical field of intravenous infusion monitoring, and discloses a real-time monitoring method and system for intravenous infusion process based on image recognition, which acquires visible light image data, non-contact distance data and structured light image data, and adaptively selects a determination mode of liquid level according to the transparency of liquid in an infusion bag, that is, visible light and structured light image data are used when the liquid is transparent, and non-contact distance data is used when the liquid is turbid or opaque, so that the problem of inaccurate liquid level recognition of traditional single image recognition method when facing infusion liquid with different transparencies is effectively solved.

Description

Technical Field

[0001] This invention relates to the field of intravenous infusion monitoring technology, and in particular to a method and system for real-time monitoring of intravenous infusion processes based on image recognition. Background Technology

[0002] In modern medical settings, accurate and real-time monitoring of intravenous infusion processes is crucial for ensuring patient safety and treatment effectiveness. While image recognition-based automated monitoring systems offer new avenues for reducing the burden on healthcare workers and improving monitoring accuracy, these systems face numerous complex and unpredictable challenges in actual hospital ward environments. These challenges stem not only from the inherent differences in infusion devices but also from potential physical displacement of monitoring equipment during long-term operation, performance degradation of optical components, information gaps between the system and other hospital information systems, and external physical interference from patients or their families. These factors combined can lead to inaccurate monitoring data, frequent false alarms or missed alarms, thereby affecting system reliability and healthcare worker trust.

[0003] In modern medical institutions, to improve the safety and efficiency of intravenous infusion, a real-time monitoring method based on image recognition has been introduced and gradually deployed. The core of this method lies in installing a dedicated image acquisition device, typically a small camera, next to each infusion point requiring monitoring. This camera is carefully positioned to clearly capture changes in the fluid level in the infusion bag and the formation and descent of droplets in the Murphy drip chamber. Through continuously acquired image sequences, the system utilizes advanced image processing and pattern recognition technology to automatically identify and count droplets in the drip chamber, thereby accurately calculating the real-time infusion drip rate. Simultaneously, it can track the height of the fluid level in the infusion bag, thus estimating the remaining medication volume. Once the monitored drip rate deviates from the preset range (whether too fast, too slow, or completely stopped), or if the medication volume is about to run out, the system immediately issues an alarm through the central monitoring platform at the nurses' station or the mobile terminal of nursing staff. This aims to promptly remind nursing staff to intervene, effectively avoiding potential risks such as infusion interruptions, medication overdose, or underdose due to human oversight.

[0004] However, as this technology moves from the ideal laboratory environment to real-world clinical applications, a series of problems that are difficult to foresee under controlled conditions gradually emerge. First, the initial deployment and calibration of the system face significant challenges. To ensure accurate image recognition, the system needs precise knowledge of the physical dimensions, shape, and optical characteristics of the infusion bags and drippers. For example, converting pixel dimensions to actual fluid volume or drip rate requires a preset scaling factor. However, in reality, medical institutions use a wide variety of infusion bags and sets, with subtle differences between manufacturers in material transparency, bag geometry (e.g., flat bags, cylindrical bags, and deformation under different filling conditions), dripper inner diameter, and dropper design. This means that a system calibrated for a specific model of infusion set will have significantly reduced measurement accuracy when used with other models. For example, if the system defaults to a dripper inner diameter of X millimeters, but the actual dripper used has an inner diameter of Y millimeters, the image characteristics of the droplet shape and falling velocity will change, leading to deviations in drip rate calculation. Similarly, the nonlinear relationship between the liquid level and the remaining volume varies depending on the shape of the infusion bag. If the calibration model is mismatched, the estimation error of the remaining medication volume will increase. Individually calibrating and storing parameters for each infusion set, and manually selecting the correct set each time it is changed, undoubtedly increases the workload for busy nursing staff, making it difficult to achieve completely accurate matching in practical applications. Summary of the Invention

[0005] This invention provides a method and system for real-time monitoring of intravenous infusion processes based on image recognition. It aims to address the complex challenges faced by existing image recognition-based intravenous infusion monitoring systems in practical clinical applications, such as the differences in infusion devices, displacement of monitoring equipment, performance degradation of optical components, information system barriers, and external physical interference. These problems may lead to inaccurate monitoring data, false alarms, or missed alarms, thereby affecting the reliability of the system and the trust of medical staff.

[0006] In a first aspect, to solve the above-mentioned technical problems, the present invention provides a method for real-time monitoring of intravenous infusion process based on image recognition, comprising:

[0007] Acquire visible light image data, non-contact distance data, and structured light image data;

[0008] The transparency of the liquid inside the infusion bag is determined based on the visible light image data.

[0009] The liquid level inside the infusion bag is adaptively determined based on the transparency.

[0010] Specifically, when the liquid is transparent, the liquid level is determined based on the visible light image data and the structured light image data; when the liquid is turbid or opaque, the liquid level is determined based on the non-contact distance data.

[0011] Based on ambient lighting conditions and droplet characteristics, droplets within the infusion tube are adaptively identified and counted. Specifically, when the ambient lighting conditions meet a first threshold and the droplet characteristics meet a second threshold, the droplets are identified and counted based on the visible light image data; when they do not meet the threshold, the droplets are identified and counted based on the structured light image data.

[0012] The sensing parameters of the liquid level height are dynamically adjusted based on the visible light image data and the structured light image data.

[0013] Preferably, the step of adaptively determining the liquid level in the infusion bag based on the transparency includes:

[0014] The visible light image data and the structured light image data are analyzed to quantify the optical properties of the liquid inside the infusion bag and construct the liquid optical property curve.

[0015] Based on the liquid optical property curve, the confidence weights of optical sensing and non-contact distance sensing are dynamically adjusted.

[0016] Based on the confidence weight, the liquid level height is obtained by fusing the optical liquid level detection results and the non-contact distance liquid level detection results.

[0017] Preferably, the adaptive identification and counting of droplets within the infusion tube based on ambient lighting conditions and droplet characteristics includes:

[0018] Install a piezoelectric sensor at the end of the infusion drip tube;

[0019] Acquire the electrical signal output by the piezoelectric sensor;

[0020] The electrical signal is filtered to obtain a filtered electrical signal;

[0021] Set the signal threshold;

[0022] When the amplitude of the filtered electrical signal exceeds the signal threshold, it is identified as a droplet shedding event;

[0023] Analyze the waveform characteristics of the droplet detachment event;

[0024] Based on the waveform characteristics, the droplet detachment event was confirmed;

[0025] After confirming the droplet detachment event, increment the droplet counter;

[0026] The infusion drip rate is calculated based on the changes in the droplet counter within a preset time window.

[0027] Preferably, calculating the infusion drip rate based on the change of the droplet counter within a preset time window includes:

[0028] Monitor the vibration frequency spectrum of the infusion drip to identify background vibration modes that are unrelated to droplet shedding;

[0029] The length of the preset time window used to calculate the infusion drip rate is dynamically adjusted based on the vibration frequency spectrum of the infusion drip tube.

[0030] Morphological analysis was performed on the droplets at the end of the infusion tube to estimate the average volume of a single droplet;

[0031] The droplet counter is weighted and corrected based on the average volume of the individual droplets;

[0032] The infusion drip rate is calculated based on the changes in the weighted and corrected droplet counter within the preset time window.

[0033] Preferably, the step of weighting the droplet counter based on the average volume of the individual droplets includes:

[0034] Real-time monitoring of the temperature of the medication solution inside the infusion bag;

[0035] Based on the temperature, dynamically query the physical parameters of the drug solution, such as viscosity and surface tension, as a function of temperature.

[0036] The average volume estimate of the individual droplet is corrected based on the temperature and the physical parameters;

[0037] A microfluidic sensor is installed at the end of the infusion drip tube;

[0038] The microfluidic sensor is used to detect the fluid dynamics parameters during droplet detachment;

[0039] The average volume estimate of a single droplet is calibrated based on the fluid dynamic parameters detected by the microfluidic sensor.

[0040] The droplet counter is weighted and corrected based on the average volume estimate of a single droplet after correction and calibration.

[0041] Preferably, the method further includes:

[0042] The inner wall of the infusion drip tube is periodically inspected using the visible light image data and the structured light image data, and the parameters of the inner wall of the drip tube in the droplet volume estimation model are adjusted according to the inspection results.

[0043] Preferably, the step of periodically inspecting the inner wall of the infusion drip tube using the visible light image data and the structured light image data, and adjusting the inner wall parameters of the drip tube in the droplet volume estimation model based on the inspection results, includes:

[0044] Analyze the visible light image data to extract the brightness, color distribution, and texture features of the inner wall of the infusion drip tube;

[0045] Analyze the structured light image data to extract the pattern distortion, contrast changes, and three-dimensional point cloud data of the inner wall of the infusion drip tube;

[0046] By fusing the visible light image data and the structured light image data, the response differences of different modal data in the defect region are compared;

[0047] Based on the differences in response, the wear, cracks, scratches, drug deposits, crystals, or biofilms on the inner wall can be distinguished.

[0048] Based on the differentiation results, adjust the surface roughness coefficient, effective inner diameter, and wetting angle in the droplet volume estimation model.

[0049] Preferably, the step of fusing the visible light image data and the structured light image data, and comparing the response differences of the different modal data in the defect region, includes:

[0050] The visible light image data and the structured light image data are time-stamped and spatially registered.

[0051] Based on the visible light image data after time-stamp synchronization and spatial registration, multi-frame sequence analysis is performed to obtain the visible light response sequence and the structured light response sequence;

[0052] Establish spatiotemporal correlation rules between the visible light response sequence and the structured light response sequence;

[0053] Based on the spatiotemporal correlation rules, the visible light response sequence and the structured light response sequence are compared to identify patterns of difference in time or space.

[0054] Secondly, a real-time monitoring module for intravenous infusion based on image recognition includes:

[0055] The acquisition unit is used to acquire visible light image data, non-contact distance data, and structured light image data;

[0056] The judgment unit is used to determine the transparency of the liquid inside the infusion bag based on the visible light image data;

[0057] A determining unit is configured to adaptively determine the liquid level height inside the infusion bag based on the transparency.

[0058] Specifically, when the liquid is transparent, the liquid level is determined based on the visible light image data and the structured light image data; when the liquid is turbid or opaque, the liquid level is determined based on the non-contact distance data.

[0059] The identification unit is used to adaptively identify and count droplets in an infusion tube based on ambient light conditions and droplet characteristics. Specifically, when the ambient light conditions meet a first threshold and the droplet characteristics meet a second threshold, the droplets are identified and counted based on the visible light image data; when they do not meet the threshold, the droplets are identified and counted based on the structured light image data.

[0060] The sensing parameters of the liquid level height are dynamically adjusted based on the visible light image data and the structured light image data.

[0061] Thirdly, a real-time monitoring system for intravenous infusion based on image recognition, characterized in that it includes:

[0062] The detection module is used to acquire visible light image data, non-contact distance data, and structured light image data;

[0063] The judgment module is used to determine the transparency of the liquid inside the infusion bag based on the visible light image data; and adaptively determine the liquid level height inside the infusion bag based on the transparency.

[0064] Specifically, when the liquid is transparent, the liquid level is determined based on the visible light image data and the structured light image data; when the liquid is turbid or opaque, the liquid level is determined based on the non-contact distance data.

[0065] An adjustment module is used to adaptively identify and count droplets in an infusion tube based on ambient light conditions and droplet characteristics. Specifically, when the ambient light conditions meet a first threshold and the droplet characteristics meet a second threshold, the droplets are identified and counted based on visible light image data; when they do not meet the threshold, the droplets are identified and counted based on structured light image data; and the sensing parameters of the liquid level height are dynamically adjusted based on the visible light image data and the structured light image data.

[0066] This application discloses a real-time monitoring method and system for intravenous infusion based on image recognition. By acquiring visible light image data, non-contact distance data, and structured light image data, and adaptively selecting the method for determining the liquid level height based on the transparency of the liquid inside the infusion bag—that is, using visible light and structured light image data when the liquid is transparent, and using non-contact distance data when the liquid is turbid or opaque—this effectively solves the problem of inaccurate liquid level identification in traditional single-image recognition methods when dealing with infusion liquids of varying transparency. Simultaneously, this method also adaptively selects visible light or structured light image data to identify and count droplets within the infusion tube based on ambient lighting conditions and droplet characteristics, overcoming the limitation of traditional methods where droplet identification is easily interfered with under complex lighting conditions. Furthermore, by dynamically adjusting the sensing parameters of the liquid level height based on visible light and structured light image data, this application can continuously optimize monitoring accuracy, enhancing the robustness and adaptability of the system during long-term operation. In summary, the technical solution of this application can significantly improve the accuracy and reliability of intravenous infusion monitoring, and effectively solve the problems of inaccurate monitoring data, false alarms or missed alarms caused by differences in infusion devices and environmental complexity in the prior art, thereby ensuring the patient's life safety and treatment effect. Attached Figure Description

[0067] Figure 1 This is a flowchart of a method for real-time monitoring of intravenous infusion process based on image recognition, provided by an embodiment of the present invention.

[0068] Figure 2 This is a flowchart of a method for determining liquid level height based on transparency, provided by an embodiment of the present invention.

[0069] Figure 3 This is a schematic diagram of a real-time monitoring system for intravenous infusion based on image recognition, provided in an embodiment of the present invention. Detailed Implementation

[0070] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] Reference Figure 1 , Figure 1 This is a flowchart of a method for real-time monitoring of intravenous infusion process based on image recognition, provided by an embodiment of the present invention, including:

[0072] S1, acquire visible light image data, non-contact distance data, and structured light image data;

[0073] S2, Based on the visible light image data, determine the transparency of the liquid inside the infusion bag;

[0074] S3, Based on the transparency, adaptively determine the liquid level height inside the infusion bag;

[0075] Specifically, when the liquid is transparent, the liquid level is determined based on the visible light image data and the structured light image data; when the liquid is turbid or opaque, the liquid level is determined based on the non-contact distance data.

[0076] S4, based on ambient light conditions and droplet characteristics, adaptively identify and count droplets in the infusion tube, wherein when the ambient light conditions meet a first threshold and the droplet characteristics meet a second threshold, the droplets are identified and counted based on the visible light image data; when they do not meet the threshold, the droplets are identified and counted based on the structured light image data.

[0077] S5, dynamically adjust the sensing parameters of the liquid level height based on the visible light image data and the structured light image data.

[0078] This application, by comprehensively utilizing visible light image data, non-contact distance data, and structured light image data, and combining them with an adaptive judgment and adjustment mechanism, effectively overcomes many problems encountered by traditional monitoring methods in complex clinical environments, and significantly improves the accuracy, robustness, and intelligence level of infusion monitoring.

[0079] The "visible light image data" mentioned in this application refers to two-dimensional image information acquired within the visible spectrum using a standard camera. It is primarily used to capture the visual characteristics of infusion bags and drip tubes, such as color, brightness, and texture. "Non-contact distance data" is typically acquired by non-contact distance measurement devices such as ultrasonic sensors, laser rangefinders, or millimeter-wave radar. It provides precise distance information between the object and the sensor, and is particularly suitable for scenarios with poor optical conditions or where measurements need to penetrate certain media. "Structured light image data" involves projecting structured light with a known pattern (such as stripes or dot matrix) onto a target object and using a camera to capture its distortion on the object's surface, thereby reconstructing the three-dimensional shape and depth information of the target object. The combined use of these data modalities aims to provide a more comprehensive and reliable source of information for real-time monitoring of the infusion process.

[0080] In the specific implementation process, it is first necessary to acquire visible light image data, non-contact distance data, and structured light image data. Visible light image data can be acquired using a visible light camera installed near the infusion point. This camera is configured to clearly capture the overall view of the infusion bag and the local details of the infusion drip. Non-contact distance data can be acquired using distance sensors installed above or to the side of the infusion bag. For example, ultrasonic sensors or laser rangefinders can be used to measure the distance from the sensor to the liquid surface of the infusion bag in real time. Structured light image data can be acquired using a structured light system consisting of a structured light projector and a structured light camera. The structured light projector projects a preset structured light pattern onto the infusion bag and drip, and the structured light camera captures the deformation of these patterns on the object surface.

[0081] After obtaining the aforementioned multimodal data, the transparency of the liquid inside the infusion bag is determined based on the visible light image data. For example, the transparency of the liquid can be evaluated by analyzing the brightness uniformity, transmittance, and visibility of background objects inside the infusion bag in the visible light image data. When the liquid is transparent, background objects inside the infusion bag (such as infusion tubing, support, etc.) will be relatively clear in the visible light image, and the overall image brightness will be high; when the liquid is turbid or opaque, background objects inside the infusion bag will be blurry or even completely invisible, and the overall image brightness will be low.

[0082] Based on the determined transparency, the liquid level inside the infusion bag is adaptively determined. Specifically, when the liquid is transparent, the liquid level can be determined using both visible light and structured light image data. For example, the liquid surface edge features (such as brightness abrupt changes and color boundaries) in the visible light image data are used for initial positioning, and then the three-dimensional depth information provided by the structured light image data is combined for precise measurement of the liquid level. Structured light image data can effectively eliminate the influence of ambient light and infusion bag deformation on liquid level detection, providing more accurate three-dimensional coordinates of the liquid level. When the liquid is turbid or opaque, since visible light and structured light may have difficulty penetrating the liquid, the liquid level is determined based on non-contact distance data. For example, the distance value measured by an ultrasonic sensor or laser rangefinder is directly used, combined with the sensor's installation location and the geometric model of the infusion bag, to calculate the liquid level. This adaptive method of determining the liquid level effectively addresses the challenges posed by varying drug liquid transparency, ensuring the accuracy of liquid level monitoring.

[0083] Furthermore, droplets within the infusion tube are adaptively identified and counted based on ambient lighting conditions and droplet characteristics. Ambient lighting conditions can be evaluated using parameters such as brightness and contrast in images captured by a visible light camera. Droplet characteristics can be extracted by analyzing the shape, size, and trajectory of droplets in visible light or structured light image data. When ambient lighting conditions meet a first threshold (e.g., sufficient and uniform illumination) and droplet characteristics meet a second threshold (e.g., clear droplet outline with no significant distortion), droplets are identified and counted based on visible light image data. In this case, traditional image processing algorithms, such as edge detection, morphological operations, and target tracking, can be used to identify and count droplets in the visible light image. When ambient lighting conditions do not meet the first threshold or droplet characteristics do not meet the second threshold, droplets are identified and counted based on structured light image data. For example, in low light, backlight, or when droplet shapes are irregular, structured light image data can provide three-dimensional morphology and depth information of the droplets, thereby more accurately identifying and counting droplets and avoiding misjudgments.

[0084] Finally, the sensing parameters for liquid level height are dynamically adjusted based on visible light and structured light image data. For example, the threshold, Region of Interest (ROI) range, and filtering parameters in the liquid level detection algorithm can be adjusted in real time based on the deformation and reflectivity of the infusion bag in the visible light image data and the depth information of the liquid surface area in the structured light image data. This dynamic adjustment mechanism makes the liquid level height sensing process more robust and adaptable to dynamic factors such as changes in the shape of the infusion bag and reduction in the amount of medication during infusion.

[0085] The image recognition-based real-time monitoring method for intravenous infusion proposed in this application aims to address the insufficient accuracy and robustness of existing infusion monitoring systems in complex clinical environments. Traditional methods often rely on single-modal data or fixed parameters, making it difficult to cope with the diversity of infusion bags and medications, changes in ambient lighting, and potential problems with the inner wall of the drip chamber. For example, existing technologies may fail to accurately identify the liquid level when faced with medications of varying transparency; droplet identification and counting are prone to errors under poor lighting conditions; and they lack the ability to adaptively adjust system parameters.

[0086] This application significantly improves the performance of the monitoring system by introducing multimodal data fusion and adaptive adjustment mechanisms. First, by acquiring visible light image data, non-contact distance data, and structured light image data, a rich information source is provided for subsequent judgment and identification. Second, the transparency of the liquid inside the infusion bag is determined based on the visible light image data, and the method for determining the liquid level height is adaptively selected accordingly. That is, when the liquid is transparent, both visible light and structured light data are combined; when the liquid is turbid or opaque, non-contact distance data is relied upon. This strategy effectively solves the impact of different drug liquid transparency on liquid level monitoring. Third, based on ambient lighting conditions and droplet characteristics, visible light or structured light data is adaptively selected for droplet identification and counting, ensuring accuracy under various complex lighting and droplet morphologies. Finally, the sensing parameters for the liquid level height are dynamically adjusted based on visible light and structured light image data, enabling the system to adapt to dynamic changes during the infusion process in real time.

[0087] Compared to existing technologies, the advantages of this application lie in its high adaptability and robustness. For example, in traditional systems, if the medication in the infusion bag changes from transparent to cloudy (e.g., after the addition of certain drugs), the liquid level monitoring may fail; however, this application can automatically switch monitoring modes based on transparency, ensuring the continuity and accuracy of monitoring. Furthermore, in ward environments with drastically changing lighting conditions, traditional systems may cause droplet identification errors due to insufficient or overexposed light, while this application effectively avoids these problems by adaptively selecting visible light or structured light data. Therefore, this application not only improves the accuracy of monitoring data and reduces the risk of false alarms and missed alarms, but also reduces the burden on medical staff and improves the safety and efficiency of intravenous infusion procedures.

[0088] In some of the embodiments described above in this application, although it is proposed to determine the liquid level height by switching between different sensing methods based on the transparency of the liquid, in practical applications, the optical properties of liquids may not be simply transparent or turbid, but rather exist in a continuously changing intermediate state, and the optical properties of different liquids vary greatly. A simple binary judgment may lead to insufficient accuracy and robustness in determining the liquid level height.

[0089] In response, this application further proposes to optimize the above-mentioned step of adaptively determining the liquid level height based on transparency by quantifying the optical properties of the liquid and dynamically adjusting the sensing weights, so as to more accurately determine the liquid level height in the infusion bag.

[0090] Specifically, refer to Figure 2 , Figure 2 This is a flowchart of a method for determining liquid level height based on transparency, provided by an embodiment of the present invention.

[0091] S3 includes:

[0092] S31, Analyze the visible light image data and the structured light image data, quantify the optical properties of the liquid in the infusion bag, and construct the liquid optical property curve;

[0093] S32, dynamically adjust the confidence weights of optical sensing and non-contact distance sensing based on the liquid optical property curve;

[0094] S33, Based on the confidence weight, the liquid level height is obtained by fusing the optical liquid level detection results and the non-contact distance liquid level detection results.

[0095] Specifically, analyzing visible light and structured light image data involves using image processing algorithms to extract visible light features such as brightness, color, texture, transmittance, and reflectance of the liquid within the infusion bag, as well as structured light features such as distortion, contrast changes, and 3D point cloud data of the structured light pattern on the liquid surface. These features allow for the quantification of the liquid's optical properties, such as its absorption coefficient, scattering coefficient, refractive index, transparency, or turbidity. Constructing a liquid optical property curve involves mapping these quantified optical property parameters to a continuous numerical range or multidimensional space, forming a curve or model that characterizes the liquid's optical state. The aim is to provide a more refined description of the liquid's optical state than a simple binary classification.

[0096] The dynamic adjustment of confidence weights for optical sensing and non-contact distance sensing refers to adjusting the confidence weights for liquid surface detection based on visible light and structured light image data, as well as the confidence weights for non-contact distance sensing, in real time according to the liquid's optical state reflected by the liquid's optical property curves. For example, when the liquid's optical property curves indicate high transparency, the confidence weight for optical sensing is increased, while the confidence weight for non-contact distance sensing is correspondingly decreased; conversely, when the liquid is highly turbid or opaque, the confidence weight for non-contact distance sensing is increased. For liquids in intermediate states, appropriate intermediate weights are assigned to both methods based on their specific optical properties. The aim is to ensure that the most reliable sensing method is always prioritized under different liquid optical conditions, and to reasonably weight different sensing results.

[0097] In practical applications, fusing optical and non-contact distance-based liquid level detection results to obtain the liquid level height refers to comprehensively processing the two sets of liquid level detection results after confidence weight adjustment to obtain the final liquid level height. For example, weighted averaging, Kalman filtering, and sensor fusion algorithms can be used to fuse the optical and non-contact distance-based liquid level detection results. The purpose is to combine the advantages of different sensing modalities to compensate for the shortcomings of a single modality, thereby obtaining a more accurate and robust liquid level height measurement.

[0098] This application's solution, through in-depth analysis of visible light and structured light image data, quantifies the specific optical properties of the liquid within the infusion bag, thus overcoming the limitations of traditional binary judgment. By constructing liquid optical property curves, continuous quantitative values ​​of liquid transparency or turbidity can be obtained, rather than simple classification. Based on these continuous quantitative values, the system can dynamically adjust the confidence weights of optical sensing (based on visible light and structured light) and non-contact distance sensing (e.g., ultrasonic or laser ranging). This means that when liquid optical properties are favorable to optical sensing, the weight of optical sensing increases; conversely, the weight of non-contact distance sensing increases. Ultimately, by fusing the liquid surface detection results from two different modalities, a more accurate and robust liquid surface height can be obtained, effectively avoiding errors that may arise from a single sensing method when the liquid's optical properties are complex or in a critical state.

[0099] Through the above technical solution, this application overcomes the limitations of simple binary judgment in the prior art when dealing with complex or continuously changing liquid optical properties. By precisely quantifying the optical properties of the liquid and dynamically adjusting the confidence weights of different sensing modes, the determination of liquid level is no longer a simple mode switching, but an intelligent fusion based on the actual properties of the liquid. This significantly improves the accuracy and robustness of liquid level sensing within the infusion bag, making it particularly suitable for medications with various transparency, color, and viscosity, thus providing more reliable real-time monitoring data for intravenous infusion processes.

[0100] In some preferred embodiments, it is assumed that the infusion bag contains a liquid with moderate turbidity. First, the liquid is analyzed using visible light and structured light image data. For example, by analyzing the transmittance of the visible light image and the distortion of the structured light image, the scattering and absorption coefficients of the liquid are quantified. Based on these quantified parameters, a liquid optical property curve is constructed, which may indicate that the liquid is in a continuous range between transparent and opaque. For example, when the liquid has a high scattering coefficient but a low absorption coefficient, the system dynamically adjusts the confidence weight of optical sensing (based on visible light and structured light) to 0.6 and the confidence weight of non-contact distance sensing (e.g., distance data acquired by an ultrasonic sensor) to 0.4 according to a preset mapping relationship. Subsequently, the optical liquid level detection results (e.g., the liquid level position obtained through image processing) and the non-contact distance liquid level detection results (e.g., the liquid level distance obtained through ultrasonic ranging) are weighted and fused to obtain the final liquid level height. This dynamic weighted fusion method enables the acquisition of high-precision liquid level data even when the optical properties of the liquid are complex or changing, avoiding the errors that may be caused by simple binary switching.

[0101] In some embodiments described above, a method for adaptively identifying and counting droplets within an infusion drip chamber based on ambient lighting conditions and droplet characteristics is proposed. This method primarily relies on visible light image data and structured light image data for identification. However, in actual intravenous infusion monitoring scenarios, droplet counting methods based purely on image recognition may be affected by various factors, such as drastic changes in ambient lighting, reflections from the inner wall of the infusion drip chamber, changes in the color or transparency of the medication itself, and image blurring due to excessively rapid droplet detachment. These factors may lead to a decrease in the accuracy of droplet identification, thereby affecting the precise calculation of the infusion drip rate.

[0102] To this end, based on ambient lighting conditions and droplet characteristics, the system adaptively identifies and counts droplets within the infusion tube, including:

[0103] Install a piezoelectric sensor at the end of the infusion drip tube;

[0104] Acquire the electrical signal output by the piezoelectric sensor;

[0105] The electrical signal is filtered to obtain a filtered electrical signal;

[0106] Set the signal threshold;

[0107] When the amplitude of the filtered electrical signal exceeds the signal threshold, it is identified as a droplet shedding event;

[0108] Analyze the waveform characteristics of the droplet detachment event;

[0109] Based on the waveform characteristics, the droplet detachment event was confirmed;

[0110] After confirming the droplet detachment event, increment the droplet counter;

[0111] The infusion drip rate is calculated based on the changes in the droplet counter within a preset time window.

[0112] Specifically, installing a piezoelectric sensor at the end of an infusion drip refers to fixing one or more piezoelectric sensors to the lower end of the drip tube, near the point where the droplet is about to detach. This piezoelectric sensor can convert the minute vibrations or impacts generated when the droplet detaches into a measurable electrical signal, allowing for direct sensing of the droplet detachment event through physical contact.

[0113] The acquisition of the electrical signal output by the piezoelectric sensor can be understood as the real-time acquisition of the voltage or current signal generated by the piezoelectric sensor through the data acquisition module. These signals contain instantaneous information about the droplet falling off, and their purpose is to convert the physical event into processable electrical data.

[0114] In practical applications, the electrical signal is filtered to obtain a filtered electrical signal. For example, low-pass filtering, band-pass filtering, or digital filtering algorithms can be used to remove environmental noise, high-frequency interference, and signal components unrelated to droplet shedding. The purpose is to improve the signal-to-noise ratio of the signal and ensure the accuracy of subsequent identification.

[0115] Setting a signal threshold refers to determining a critical value for the amplitude of an electrical signal based on experimental data or experience. This value is used to distinguish droplet detachment events from background noise and aims to provide a basis for the preliminary identification of droplet detachment events.

[0116] When the amplitude of the filtered electrical signal exceeds the signal threshold, it is identified as a droplet detachment event. This means that once the signal strength reaches the preset threshold, the system will initially determine that a droplet has detached. Its purpose is to respond quickly and mark the potential droplet detachment moment.

[0117] Analyzing the waveform characteristics of the droplet detachment event can specifically include analyzing the rising edge, falling edge, pulse width, peak value, etc., with the aim of verifying the authenticity of the event from multiple dimensions and eliminating misjudgments.

[0118] Confirming the droplet detachment event based on the waveform characteristics means further verifying and confirming that the event is indeed a droplet detachment by comparing it with a preset droplet detachment waveform template or feature pattern. The purpose is to improve the robustness and accuracy of the identification.

[0119] After confirming the droplet detachment event, incrementing the droplet counter means adding one to the value of an accumulator counter to record the number of detached droplets, with the aim of providing accurate droplet count statistics.

[0120] The infusion rate is calculated based on the changes of the droplet counter within a preset time window. This means that within a certain time period, the increment of the droplet counter is counted and divided by the length of the time window to obtain the number of droplets per unit time. The purpose is to monitor the infusion rate in real time.

[0121] This application's solution converts the physical impact of droplet detachment into an electrical signal by installing a piezoelectric sensor at the end of the infusion drip line, thus avoiding the limitations of traditional image recognition methods under complex lighting or background conditions. The piezoelectric sensor can directly sense the minute vibrations generated at the moment of droplet detachment, and its signal has high instantaneousness and specificity. By filtering the acquired electrical signal, environmental noise can be effectively removed, resulting in a pure droplet detachment signal. Subsequently, by setting a signal threshold and analyzing the waveform characteristics of the droplet detachment event, each droplet detachment can be accurately identified and confirmed, avoiding the blurring, overlapping, or missed identification problems that may occur in image recognition. Therefore, by accumulating a droplet counter and calculating its changes within a preset time window, the infusion drip rate can be accurately calculated, providing reliable real-time monitoring data for the intravenous infusion process.

[0122] Through the above technical solution, this application effectively overcomes the problems that pure image recognition methods may have in droplet counting, such as sensitivity to ambient light, large background interference, and limited recognition accuracy. As a physical contact sensor, the piezoelectric sensor provides a more direct and stable perception of droplet detachment, and is less susceptible to external visual factors. This solution, through fine processing of electrical signals and waveform feature analysis, significantly improves the accuracy and reliability of droplet detachment event recognition, thereby ensuring the precision of infusion drip rate calculation. This provides a more stable and reliable monitoring method for clinical intravenous infusion management, effectively improving patient medication safety.

[0123] In some embodiments described above, this application proposes a scheme to calculate the infusion drip rate based on the changes in a droplet counter within a preset time window. However, in practical applications, the infusion drip tube may be affected by environmental vibrations, leading to inaccurate droplet counting. Simultaneously, the actual volume of the droplets may vary due to factors such as the nature of the medication and the condition of the inner wall of the drip tube, resulting in inherent errors in drip rate calculations based on simple counting, affecting the accuracy and reliability of monitoring. If these problems are not addressed, misjudgments of the infusion drip rate may occur, thereby affecting patient medication safety and treatment efficacy. Therefore, this application further proposes an optimized infusion drip rate calculation method that comprehensively considers the vibration of the infusion drip tube, droplet morphology, and volume changes to improve the accuracy and robustness of the drip rate calculation.

[0124] The above calculation of the infusion drip rate based on the changes in the droplet counter within a preset time window specifically includes:

[0125] Monitor the vibration frequency spectrum of the infusion drip to identify background vibration modes that are unrelated to droplet shedding;

[0126] The length of the preset time window used to calculate the infusion drip rate is dynamically adjusted based on the vibration frequency spectrum of the infusion drip tube.

[0127] Morphological analysis was performed on the droplets at the end of the infusion tube to estimate the average volume of a single droplet;

[0128] The droplet counter is weighted and corrected based on the average volume of the individual droplets;

[0129] The infusion drip rate is calculated based on the changes in the weighted and corrected droplet counter within the preset time window.

[0130] Specifically, monitoring the vibration frequency spectrum of an infusion drip tube involves continuously collecting vibration signals from the tube using sensors (such as piezoelectric sensors or miniature accelerometers) installed on it, and then performing spectral analysis, such as Fourier transform, on these signals to obtain their frequency distribution. By analyzing this frequency spectrum, characteristic frequencies associated with droplet detachment events can be identified and distinguished from background vibration modes unrelated to droplet detachment, such as environmental noise and equipment vibration. The purpose is to filter out interference and ensure the accuracy of the droplet counter.

[0131] The method involves dynamically adjusting the preset time window length for calculating the infusion drip rate based on the vibration frequency spectrum of the infusion drip tube. This can be understood as follows: when significant background vibration or irregular vibration is detected in the infusion drip tube, the preset time window length can be shortened to avoid misjudging or missing drips, thus improving the response speed to instantaneous drip rate changes; conversely, when the vibration is stable, the window length can be appropriately extended to obtain a more stable average drip rate. The aim is to make the drip rate calculation better adaptable to different environmental conditions and infusion states.

[0132] In practical applications, morphological analysis of droplets at the tip of an infusion drip to estimate the average volume of a single droplet involves high-resolution imaging of the droplet about to detach using visible light and / or structured light image data. Image processing algorithms (such as edge detection, region segmentation, and 3D reconstruction) are then used to extract the droplet's geometric features, such as diameter, curvature, and surface area, to estimate the instantaneous volume of a single droplet. The aim is to obtain more accurate droplet volume information to correct for drip rates based on simple counting.

[0133] Furthermore, weighted correction of the droplet counter based on the average volume of a single droplet involves comparing the estimated average volume of a single droplet with the volume of a standard droplet, and adjusting the number of droplets recorded by the droplet counter according to the ratio. For example, if the estimated average droplet volume is greater than the standard volume, the counter can be appropriately adjusted downwards; conversely, it can be adjusted upwards. The purpose is to compensate for the droplet rate calculation error caused by changes in droplet volume.

[0134] Therefore, calculating the infusion drip rate based on the weighted correction of the droplet counter within a preset time window means dividing the change of the corrected droplet counter within the dynamically adjusted preset time window by the length of the time window, thereby obtaining a more accurate and reliable infusion drip rate.

[0135] This application's solution effectively solves the accuracy problem in traditional drip rate calculation by introducing multimodal sensing and intelligent analysis. Through the above technical solution, this application can effectively eliminate the influence of environmental vibration on droplet counting, significantly improving the accuracy and anti-interference capability of droplet identification. Simultaneously, dynamically adjusting the preset time window length makes drip rate calculation more flexible and adaptable, better capturing instantaneous changes during infusion. More importantly, through precise estimation and weighted correction of individual droplet volume, this application significantly improves the accuracy of infusion drip rate calculation, avoiding errors caused by changes in droplet volume. This provides more accurate and reliable real-time monitoring data for clinical intravenous infusion, ensuring patient medication safety and improving the quality of medical care.

[0136] In some preferred embodiments, this application is implemented as follows:

[0137] Assuming a piezoelectric sensor is installed on the infusion drip tube, its output electrical signal is collected and subjected to spectrum analysis. When the analysis results show the presence of low-frequency background vibrations below 50Hz, the system identifies this as a background vibration mode unrelated to droplet detachment. At this point, the preset time window length used to calculate the infusion drip rate is dynamically adjusted, for example, shortened from the default 10 seconds to 5 seconds, to improve sensitivity to actual droplet detachment events.

[0138] Simultaneously, visible light image data is used to continuously photograph the droplets about to detach from the infusion tube. Image processing algorithms are employed, such as Canny edge detection followed by Hough transform to identify the circular or elliptical contours of the droplets and calculate their diameter and curvature. Based on these morphological parameters and a pre-defined droplet shape model, the average volume of a single droplet can be estimated. For example, if the estimated average volume is 0.045 ml, and the standard droplet volume is 0.04 ml, the actual liquid volume recorded by the droplet counter will be corrected to 0.045 ml for each droplet recorded.

[0139] Finally, the change in the weighted and corrected droplet counter within the dynamically adjusted 5-second time window is divided by 5 seconds to obtain the current, more accurate infusion drip rate. For example, if the corrected droplet counter increases by 10 droplets within 5 seconds, the drip rate is 10 droplets / 5 seconds = 2 droplets / second, or 2 droplets / second. 0.045 mL / droplet = 0.09 mL / second. This method allows for high-precision monitoring of infusion droplet rates even in the presence of environmental vibrations or changes in droplet volume.

[0140] In some embodiments described above, this application proposes performing morphological analysis on the droplets at the end of the infusion drip tube to estimate the average volume of a single droplet, and then weighting and correcting the droplet counter based on this average volume to calculate the infusion drip rate. However, in actual infusion processes, the physical properties of the medication (such as viscosity and surface tension) are affected by environmental factors such as temperature, causing changes in the actual volume of a single droplet. Without precise correction, this may affect the accuracy of the weighted correction of the droplet counter, and consequently, the accuracy of the infusion drip rate calculation. Therefore, this application further proposes a more refined correction and calibration of the average volume of a single droplet to improve the accuracy of droplet counting and drip rate calculation.

[0141] The above-mentioned weighted correction of the droplet counter based on the average volume of the individual droplets includes:

[0142] Real-time monitoring of the temperature of the medication solution inside the infusion bag;

[0143] Based on the temperature, dynamically query the physical parameters of the drug solution, such as viscosity and surface tension, as a function of temperature.

[0144] The average volume estimate of the individual droplet is corrected based on the temperature and the physical parameters;

[0145] A microfluidic sensor is installed at the end of the infusion drip tube;

[0146] The microfluidic sensor is used to detect the fluid dynamics parameters during droplet detachment;

[0147] The average volume estimate of a single droplet is calibrated based on the fluid dynamic parameters detected by the microfluidic sensor.

[0148] The droplet counter is weighted and corrected based on the average volume estimate of a single droplet after correction and calibration.

[0149] Specifically, real-time monitoring of the medication temperature inside the infusion bag refers to continuously acquiring real-time temperature data of the medication inside the infusion bag using temperature sensors (such as thermistors, infrared temperature sensors, etc.). This temperature data is a key input parameter for correcting the droplet volume.

[0150] The system dynamically queries the physical parameters of the liquid's viscosity and surface tension as a function of temperature. This can be understood as the system having a built-in database of physical parameters for various commonly used liquids. This database stores data on the viscosity, surface tension, and other physical properties of different liquids at different temperatures. Upon obtaining the real-time temperature, the system retrieves the corresponding physical parameters from the database based on the current liquid type and temperature. The purpose is to obtain the key physical properties that affect droplet volume.

[0151] In practical applications, the estimated average volume of a single droplet is corrected based on the temperature and physical parameters. Specifically, this involves using a fluid dynamics model, combined with real-time temperature and queried parameters such as viscosity and surface tension, to theoretically correct the average volume of a single droplet initially estimated through morphology analysis. For example, droplet volume is typically proportional to surface tension and related to factors such as viscosity, which can be calculated by establishing a corresponding physical model.

[0152] Furthermore, installing a microfluidic sensor at the end of the infusion tube refers to installing one or more miniature sensors, such as piezoresistive sensors, capacitive sensors, or optical sensors, in the area where the droplet is about to detach. These sensors are used to sense the microfluidic state of the droplet in real time at the moment of detachment. Specifically, using the microfluidic sensor to detect the fluid dynamics parameters during droplet detachment can be understood as the microfluidic sensor capturing data such as pressure changes, flow rate changes, or minute deformations during droplet formation and separation as the droplet detaches from the end of the tube. These parameters directly reflect the actual physical process during droplet detachment.

[0153] In practical applications, the average volume estimate of a single droplet is calibrated based on the fluid dynamic parameters detected by the microfluidic sensor. Specifically, these real-time detected fluid dynamic parameters are used to calibrate the droplet volume estimate after temperature and physical parameter correction at the actual measurement level.

[0154] For example, sensor data can be mapped to more precise droplet volumes using machine learning algorithms or preset calibration curves. The aim is to compensate for the shortcomings of theoretical corrections and provide volume data closer to reality. Therefore, the droplet counter is weighted and corrected based on the estimated average volume of individual droplets after correction and calibration. This means applying the average volume value of individual droplets obtained through the above two precise processing steps to the weighting algorithm of the droplet counter to ensure that each counted droplet accurately reflects its true contribution to the drug volume, thereby improving the accuracy of infusion rate calculation.

[0155] This application's solution addresses the accuracy issue of traditional droplet volume estimation in complex environments by introducing a multi-dimensional, real-time parameter correction and calibration mechanism. First, it monitors the temperature of the medication within the infusion bag in real time and dynamically queries physical parameters such as viscosity and surface tension. This allows the system to make preliminary theoretical corrections to the average volume of individual droplets based on the actual state of the medication, rather than fixed parameters. This is because the viscosity and surface tension of the medication are key factors affecting droplet formation and detachment volume, and these parameters are closely related to temperature. Temperature-driven parameter querying and correction effectively address the impact of different medication types and environmental temperature variations. Second, a microfluidic sensor is installed at the end of the infusion tube, and it detects the hydrodynamic parameters during droplet detachment, providing real-time, physical feedback for droplet volume estimation. These hydrodynamic parameters directly reflect the actual state of the droplet at the moment of detachment, compensating for the shortcomings of purely morphological analysis and theoretical corrections. By using these real-time detected parameters to calibrate the estimated average volume of individual droplets, the accuracy of volume estimation is further improved. Finally, the more accurate estimate of the average volume of a single droplet after temperature correction and microfluidic sensor calibration is applied to the weighted correction of the droplet counter, ensuring that the droplet counter can more accurately reflect the actual infusion volume, thereby making the calculation of the infusion drip rate more accurate and reliable.

[0156] Through the above technical solution, this application can significantly improve the accuracy and robustness of infusion drip rate monitoring. Compared with the basic approach of estimating droplet volume solely through morphological analysis, this application effectively solves the problem of inaccurate droplet volume estimation caused by factors such as changes in ambient temperature and differences in the physical properties of different medications by introducing real-time corrections for physical parameters such as drug temperature, viscosity, and surface tension, as well as real-time calibration of fluid dynamic parameters by microfluidic sensors. This allows the droplet counter to perform weighted corrections based on a more realistic single droplet volume, thereby ensuring that the calculated infusion drip rate remains highly accurate under various complex infusion scenarios, avoiding infusion volume deviations caused by estimation errors, and greatly improving the safety and effectiveness of intravenous infusion.

[0157] In some embodiments described above, this application proposes a real-time monitoring method for intravenous infusion based on image recognition. This method can adaptively identify and count droplets within the infusion tube. However, in practical applications, the condition of the inner wall of the infusion tube changes with usage time, the type of infusion medication, and other factors. For example, wear, drug deposits, crystals, or biofilms may appear. These changes can alter the physical properties of droplet formation and shedding, thus affecting the accuracy of the droplet volume estimation model and causing deviations in the calculation of the infusion drip rate. If these problems are not addressed, the accuracy of infusion monitoring may decrease, affecting patient medication safety. Therefore, this application further proposes an optimization scheme that periodically checks the condition of the inner wall of the infusion tube and dynamically adjusts relevant parameters in the droplet volume estimation model to ensure long-term accuracy and reliability of monitoring.

[0158] The method further includes: periodically inspecting the inner wall of the infusion drip tube using the visible light image data and the structured light image data, and adjusting the inner wall parameters of the drip tube in the droplet volume estimation model based on the inspection results.

[0159] Specifically, periodic inspection refers to inspecting the inner wall of the infusion drip tube at preset time intervals (e.g., every few hours or daily) or when a specific event is triggered (e.g., when abnormal droplet morphology or infusion rate fluctuations are detected).

[0160] The visible light image data is used to acquire macroscopic visual information about the inner wall of the dropper, such as color changes, visible deposits, or wear marks. The structured light image data is used to acquire precise three-dimensional surface morphology information about the inner wall of the dropper, such as surface roughness, the thickness of microcracks, or deposits. By fusing image data from these two modalities, a comprehensive and detailed assessment of the condition of the inner wall of the dropper can be achieved. The inspection results refer to defects or changes in the inner wall identified through image analysis, such as wear, cracks, scratches, drug deposits, crystals, or biofilms. Based on these inspection results, the parameters of the inner wall of the dropper in the droplet volume estimation model will be dynamically adjusted. These parameters typically include the surface roughness coefficient, effective inner diameter, and wetting angle, which directly affect the droplet formation and shedding process. For example, when drug deposits are detected, the effective inner diameter may be corrected to a smaller value, and the surface roughness coefficient and wetting angle will also be adjusted accordingly to more accurately reflect the interaction between the droplet and the droplet wall.

[0161] The proposed solution periodically performs detailed inspections of the inner wall of the infusion drip chamber using visible light and structured light image data, enabling real-time sensing and quantification of changes in these physical parameters. Based on the inspection results, the corresponding inner wall parameters in the droplet volume estimation model are dynamically adjusted, such as correcting the surface roughness coefficient, effective inner diameter, and wetting angle, thereby ensuring that the estimated droplet volume remains highly consistent with the actual situation. This dynamic adjustment mechanism effectively compensates for errors introduced by changes in the inner wall condition of the drip chamber, maintaining the long-term accuracy of infusion drip rate monitoring.

[0162] The above technical solution significantly improves the long-term accuracy and stability of infusion drip rate monitoring, avoiding measurement errors caused by aging or contamination of the drip chamber inner wall. Furthermore, this solution enhances the system's adaptability and robustness, enabling it to cope with challenges posed by different infusion environments and drug characteristics. Therefore, it provides clinicians with more reliable infusion data, helping to promptly detect and correct infusion abnormalities and ensure patient medication safety.

[0163] In some preferred embodiments, it is assumed that during a long-term intravenous infusion, the system automatically initiates an inspection of the infusion drip chamber's inner wall every 4 hours. During one inspection, analysis of visible light image data reveals a slight color change in the drip chamber's inner wall, while structured light image data reveals minute pattern distortions and contrast variations on the inner wall surface, identified as the initial formation of drug deposits. Based on these inspection results, the system automatically increases the surface roughness coefficient in the droplet volume estimation model by 0.05, fine-tunes the effective inner diameter by 0.02 mm, and adjusts the wetting angle parameter by 2 degrees to reflect the impact of deposits on droplet formation characteristics. In this way, even if the condition of the drip chamber's inner wall changes, the droplet volume estimation remains highly accurate, ensuring accurate real-time monitoring of the infusion drip rate and providing reliable reference data for healthcare professionals.

[0164] In some embodiments described above, a scheme is proposed to periodically inspect the inner wall of the infusion drip using visible light and structured light image data, and adjust the parameters of the inner wall in the droplet volume estimation model based on the inspection results. However, in practical applications, the inner wall of the infusion drip may have various types of defects, such as wear, cracks, scratches, drug deposits, crystals, or biofilms. These different types of defects have different mechanisms of influence on droplet formation, shedding, and related parameters in the droplet volume estimation model (such as surface roughness, effective inner diameter, and wetting angle). If only a general inspection and adjustment are performed, it may be impossible to accurately identify the type of defect and its specific impact on droplet morphology and shedding behavior, resulting in insufficient precision in the adjustment of the droplet volume estimation model and affecting the accuracy and reliability of infusion drip rate monitoring. In response, this application further proposes a specific method for periodically inspecting the inner wall of the infusion drip using visible light image data and structured light image data, and adjusting the inner wall parameters of the drip in the droplet volume estimation model based on the inspection results. The aim is to accurately identify the type of inner wall defects through multimodal image data fusion analysis, and thereby refine the relevant parameters in the droplet volume estimation model to improve the accuracy of monitoring.

[0165] The aforementioned periodic inspection of the inner wall of the infusion drip using visible light and structured light image data, and adjustment of the drip inner wall parameters in the droplet volume estimation model based on the inspection results, includes:

[0166] Analyze the visible light image data to extract the brightness, color distribution, and texture features of the inner wall of the infusion drip tube;

[0167] Analyze the structured light image data to extract the pattern distortion, contrast changes, and three-dimensional point cloud data of the inner wall of the infusion drip tube;

[0168] By fusing the visible light image data and the structured light image data, the response differences of the different modal data in the defect region are compared.

[0169] Based on the differences in response, the wear, cracks, scratches, drug deposits, crystals, or biofilms on the inner wall can be distinguished.

[0170] Based on the differentiation results, adjust the surface roughness coefficient, effective inner diameter, and wetting angle in the droplet volume estimation model.

[0171] Specifically, when inspecting the inner wall of an infusion drip chamber, the first step is to analyze visible light image data to extract the brightness, color distribution, and texture features of the inner wall. Brightness reflects the overall illumination and reflectivity of the inner wall; color distribution reveals the presence of foreign matter adhesion or discolored areas; and texture features characterize the surface roughness or microstructural changes of the inner wall. For example, worn areas may exhibit uneven brightness and blurred texture, while drug adhesions may present a specific color or a speckled distribution.

[0172] Simultaneously, structured light image data is analyzed to extract pattern distortion, contrast variations, and 3D point cloud data of the inner wall. When structured light is projected onto an object's surface, its pattern is distorted due to the object's shape and defects. Analyzing these distortions allows for the acquisition of 3D morphological information of the object's surface. Contrast variations further enhance the visual recognizability of defective areas. The 3D point cloud data directly provides high-precision 3D geometric information of the inner wall surface, enabling accurate measurement of the inner wall's unevenness, crack depth, or deposit thickness. For example, cracks or scratches can cause local interruptions or sharp bends in the structured light pattern, while drug deposits will appear as localized bulges in the 3D point cloud data.

[0173] Furthermore, visible light image data and structured light image data are fused, and the response differences of different modal data in the defect area are compared. This fusion leverages the advantages of visible light images in terms of color and texture, and the advantages of structured light images in terms of three-dimensional topography and depth information, to achieve a more comprehensive and accurate perception of internal wall defects. By comparing the response differences of different modal data in the same defect area—for example, if the visible light image shows abnormal color but the structured light image shows a smooth surface (possibly due to staining), or if the visible light image shows slight scratches but the structured light image shows significant depth (confirming physical damage)—the nature of the defect can be identified more accurately.

[0174] Therefore, based on the aforementioned response differences, specific defect types such as wear, cracks, scratches, drug deposits, crystals, or biofilms on the inner wall can be distinguished. For example, wear typically manifests as uneven brightness and blurred texture in visible light images, while structured light images show increased surface roughness; cracks and scratches may appear as linear dark lines in visible light images and as obvious pattern distortion and depth variations in structured light images; drug deposits or crystals may exhibit specific colors or crystalline reflections in visible light images and as localized bulges or irregular surfaces in structured light images; biofilms may appear as blurry, translucent deposits in visible light images and as subtle changes in surface morphology in structured light images.

[0175] Finally, based on the differentiation results, the surface roughness coefficient, effective inner diameter, and wetting angle in the droplet volume estimation model are adjusted. For example, when inner wall wear is identified, the surface roughness coefficient can be increased; when drug deposits or crystallization cause a decrease in the effective inner diameter, the effective inner diameter parameter can be decreased; when biofilms cause changes in surface hydrophilicity or hydrophobicity, the wetting angle can be adjusted. Precise adjustments to these parameters are crucial for accurately estimating droplet volume.

[0176] This application's solution combines visible light image data and structured light image data to perform multimodal, high-precision inspection of the inner wall of an infusion drip chamber. Through this technical solution, the application overcomes the limitations of traditional methods in identifying inner wall defects and adjusting parameters. Accurately distinguishing different types of inner wall defects allows for more targeted and refined adjustments to relevant parameters in the droplet volume estimation model (such as surface roughness coefficient, effective inner diameter, and wetting angle). This refined adjustment significantly improves the accuracy and adaptability of the droplet volume estimation model, ensuring that the monitoring of the infusion drip rate remains highly accurate and reliable even when the inner wall condition of the infusion drip chamber changes. This effectively avoids monitoring errors caused by inner wall defects and enhances the safety of intravenous infusion procedures.

[0177] In some preferred embodiments, a specific example is illustrated below. Suppose that during a periodic inspection, the system detects a localized yellowing and slightly blurred texture feature on a certain area of ​​the inner wall of an infusion drip using visible light image data. Simultaneously, structured light image data shows a slight distortion of the structured light pattern in this area, and 3D point cloud data indicates a localized bulge of approximately 0.1 mm in this area. By fusing the visible light and structured light image data and comparing their response differences, the system identifies that the features of this area conform to a typical pattern of "drug adhesion."

[0178] Specifically, the yellowish color and blurred texture are visual manifestations of drug residue, while the localized bulges detected by structured light further confirm the presence and thickness of the deposits. Based on this distinction, the system determines that the drug deposits reduce the effective inner diameter of the dropper and may alter the wetting properties of the inner wall. Therefore, the effective inner diameter parameter in the droplet volume estimation model is correspondingly reduced, while the wetting angle parameter is also adjusted to reflect the influence of the deposits on droplet formation and shedding. In this way, even with drug deposits on the inner wall of the dropper, the droplet volume estimation remains highly accurate, thus ensuring the precision of infusion drip rate monitoring.

[0179] Specifically, the above-mentioned fusion of visible light image data and structured light image data, and comparison of the response differences of different modal data in the defect region, may include the following steps:

[0180] Perform timestamp synchronization and spatial registration on visible light image data and structured light image data;

[0181] Timestamp synchronization ensures that visible light and structured light image data are acquired at the same time to reflect the state of the inner wall of the infusion drip tube at the same moment. Spatial registration aligns image data from two different modalities in a spatial coordinate system, ensuring that pixels at the same physical location correspond accurately in different images. This can be achieved by calibrating the camera's intrinsic and extrinsic parameters using a calibration board and applying a geometric transformation algorithm, with the aim of providing an accurate foundation for subsequent multimodal data fusion and difference comparison.

[0182] Based on the visible light image data after timestamp synchronization and spatial registration, multi-frame sequence analysis is performed to obtain the visible light response sequence and the structured light response sequence;

[0183] Specifically, multi-frame sequence analysis refers to processing continuously acquired visible light and structured light image data to capture the dynamic changes of internal wall defects in the temporal dimension or subtle features in the spatial dimension. By analyzing multiple frames, the defect signal can be enhanced and random noise suppressed, resulting in more stable and reliable visible light and structured light response sequences. The visible light response sequence can be understood as the set of brightness, color, texture, and other features of internal wall defects that change over time or space under visible light; the structured light response sequence is the set of features such as pattern distortion, contrast changes, and three-dimensional morphology caused by internal wall defects under structured light illumination.

[0184] Establish spatiotemporal correlation rules between visible light response sequences and structured light response sequences;

[0185] Spatiotemporal correlation rules refer to models or algorithms that describe the relationship between visible light response sequences and structured light response sequences in time and space. For example, a mapping relationship can be established to show that a certain type of internal wall defect manifests as a specific color change in a visible light image, while simultaneously manifesting as a specific three-dimensional morphological distortion in a structured light image. The purpose is to provide a logical basis for subsequent differential pattern recognition, ensuring that comparisons between different modal data have physical meaning.

[0186] Based on spatiotemporal correlation rules, visible light response sequences and structured light response sequences are compared to identify patterns of difference in time or space.

[0187] In practical applications, comparing visible light response sequences and structured light response sequences to identify temporal or spatial differences involves using established spatiotemporal correlation rules to compare the response sequences of the two modes point-by-point or regionally. For example, if a region appears as an abnormal brightness in a visible light image but as a sudden change in three-dimensional shape in a structured light image, and this combination conforms to a preset spatiotemporal correlation rule, then this can be identified as a specific defect pattern. Identifying this difference pattern helps to more accurately locate and classify internal wall defects, such as distinguishing between different types of defects like wear, cracks, and drug deposits.

[0188] This application's solution ensures precise temporal and spatial alignment of different modal data by rigorously synchronizing and spatially registering visible light and structured light image data, laying the foundation for subsequent fusion and comparison. Through this technical solution, this application achieves high-precision fusion and comparison of visible light and structured light image data, overcoming the limitations of single-modal data in detecting defects on the inner wall of infusion drip tubes. Timestamp synchronization and spatial registration guarantee data consistency, multi-frame sequence analysis improves detection robustness, and the establishment of spatiotemporal correlation rules makes defect identification more intelligent and accurate. Therefore, it can more accurately and comprehensively identify and distinguish various defect types on the inner wall of infusion drip tubes, such as wear, cracks, scratches, drug deposits, crystals, or biofilms, providing more reliable input for subsequent adjustments to the droplet volume estimation model, thereby improving the overall accuracy and safety of infusion monitoring.

[0189] In modern medical settings, accurate and real-time monitoring of intravenous infusion processes is crucial for ensuring patient safety and treatment effectiveness. Traditional image recognition-based automated monitoring systems face numerous complex and unpredictable challenges in real-world hospital ward environments, including variations in infusion devices, physical displacement of monitoring equipment, performance degradation of optical components, information gaps within the system, and external physical interference. These factors combined can lead to inaccurate monitoring data, frequent false alarms or missed alarms, thereby affecting system reliability and the trust of healthcare professionals.

[0190] In response, this application proposes a real-time monitoring module for intravenous infusion based on image recognition, comprising:

[0191] The acquisition unit is used to acquire visible light image data, non-contact distance data, and structured light image data;

[0192] The judgment unit is used to determine the transparency of the liquid inside the infusion bag based on the visible light image data;

[0193] A determining unit is configured to adaptively determine the liquid level height inside the infusion bag based on the transparency; wherein, when the liquid is transparent, the liquid level height is determined based on the visible light image data and the structured light image data; when the liquid is turbid or opaque, the liquid level height is determined based on the non-contact distance data.

[0194] The identification unit is used to adaptively identify and count droplets in an infusion tube based on ambient light conditions and droplet characteristics. Specifically, when the ambient light conditions meet a first threshold and the droplet characteristics meet a second threshold, the droplets are identified and counted based on visible light image data; when they do not meet the threshold, the droplets are identified and counted based on structured light image data. The unit also dynamically adjusts the sensing parameters of the liquid level height based on the visible light image data and the structured light image data.

[0195] This application presents an image recognition-based real-time monitoring module for intravenous infusion, which integrates acquisition, judgment, determination, and recognition units. By comprehensively utilizing multimodal data and incorporating adaptive judgment and adjustment mechanisms, it effectively overcomes many problems encountered by traditional monitoring systems in complex clinical environments, significantly improving the accuracy, robustness, and intelligence of infusion monitoring. The collaborative work of each unit ensures precise sensing and dynamic adjustment of fluid level and droplet count under different infusion conditions and environmental changes.

[0196] In some embodiments of this application, the specific processes for acquiring visible light image data, non-contact distance data, and structured light image data have already been described in the above embodiments, and will not be repeated here. It should be emphasized that the acquisition unit can be configured to include one or more visible light cameras, one or more non-contact distance sensors (e.g., ultrasonic sensors or laser rangefinders), and a structured light system (including a structured light projector and a structured light camera). These hardware components are integrated or work together to acquire the required multimodal data in real time. For example, the acquisition unit can be a composite sensor module integrating multiple sensors, which is connected to the main processing unit via wired or wireless means and is responsible for the initial acquisition and transmission of data.

[0197] Furthermore, the specific process of determining the transparency of the liquid inside the infusion bag based on visible light image data has already been described in the above embodiments, and will not be repeated here. It is important to emphasize that the determination unit can be implemented as an image processing module, which contains an image analysis algorithm to evaluate the brightness uniformity, transmittance, and background visibility of the visible light image. This determination unit can be an independent microcontroller or embedded processor, specifically responsible for performing the transparency analysis task and outputting the determination result to the subsequent determination unit.

[0198] Furthermore, the specific process of adaptively determining the liquid level height inside the infusion bag based on transparency has already been described in the above embodiments, and will not be repeated here. It is important to emphasize that the determining unit can be implemented as a data fusion and calculation module, which receives the transparency information output by the judgment unit and selectively processes visible light image data, structured light image data, or non-contact distance data from the acquisition unit based on this information. For example, the determining unit can include multiple liquid level detection algorithms, selecting the appropriate algorithm to calculate the liquid level height based on the transparency. This determining unit can be a high-performance digital signal processor (DSP) or graphics processing unit (GPU) to support complex image processing and depth information calculations.

[0199] Furthermore, the specific process of adaptively identifying and counting droplets within the infusion tube based on ambient lighting conditions and droplet characteristics has already been described in the above embodiments, and will not be repeated here. It is important to emphasize that the identification unit can be implemented as an intelligent vision processing module, integrating an ambient lighting assessment module and a droplet feature analysis module. This identification unit can dynamically switch between visible light image data processing algorithms and structured light image data processing algorithms to identify and count droplets based on the assessment results. For example, the identification unit can employ a deep learning-based image recognition model, trained to adapt to different lighting conditions and droplet morphologies, thereby improving the accuracy and robustness of the identification.

[0200] Finally, regarding the sensing parameters for dynamically adjusting the liquid level height based on visible light and structured light image data, this function can be implemented by the aforementioned determining unit or a separate parameter adjustment module. This module can analyze multimodal data in real time, such as the deformation of the infusion bag, reflectivity, and depth information of the liquid surface area, and adjust the threshold, region of interest (ROI) range, and filtering parameters in the liquid level detection algorithm accordingly to ensure the accuracy and adaptability of the liquid level height sensing.

[0201] The image recognition-based real-time monitoring module for intravenous infusion proposed in this application aims to address the insufficient accuracy and robustness of existing infusion monitoring systems in complex clinical environments. Traditional monitoring modules often rely on single-modal data or fixed parameters, making it difficult to cope with the diversity of infusion bags and medications, changes in ambient lighting, and potential problems with the inner wall of the drip chamber. For example, existing modules may fail to accurately identify the liquid level when faced with medications of varying transparency; droplet identification and counting are prone to errors under poor lighting conditions; and they lack the ability to adaptively adjust system parameters.

[0202] This application achieves multimodal data fusion and adaptive adjustment mechanisms through the collaborative work of its acquisition unit, judgment unit, determination unit, and recognition unit, significantly improving the performance of the monitoring module. First, the acquisition unit provides visible light image data, non-contact distance data, and structured light image data, offering rich information sources for subsequent judgment and recognition. Second, the judgment unit determines the transparency of the liquid inside the infusion bag based on the visible light image data, and the determination unit adaptively selects the method for determining the liquid level height accordingly: when the liquid is transparent, it combines visible light and structured light data; when the liquid is turbid or opaque, it relies on non-contact distance data. This strategy effectively addresses the impact of varying drug transparency on liquid level monitoring. Third, the recognition unit adaptively selects visible light or structured light data for droplet identification and counting based on ambient lighting conditions and droplet characteristics, ensuring accuracy under various complex lighting and droplet morphologies. Finally, the entire module can dynamically adjust the sensing parameters of the liquid level height based on visible light and structured light image data, enabling the system to adapt to dynamic changes during the infusion process in real time.

[0203] Compared to existing technologies, the advantages of this application's module lie in its high adaptability and robustness. For example, in traditional modules, if the medication in the infusion bag changes from transparent to cloudy, the liquid level monitoring may fail; however, this application's module can automatically switch monitoring modes based on transparency, ensuring the continuity and accuracy of monitoring. Furthermore, in ward environments with drastically changing lighting conditions, traditional modules may cause droplet identification errors due to insufficient or overexposed light, while this application's module effectively avoids these problems by adaptively selecting visible light or structured light data. Therefore, this application's module not only improves the accuracy of monitoring data and reduces the risk of false alarms and missed alarms, but also reduces the burden on medical staff and improves the safety and efficiency of intravenous infusion procedures.

[0204] refer to Figure 3 , Figure 3 This application provides a schematic diagram of a real-time monitoring system for intravenous infusion based on image recognition, comprising:

[0205] The detection module is used to acquire visible light image data, non-contact distance data, and structured light image data;

[0206] The judgment module is used to determine the transparency of the liquid inside the infusion bag based on the visible light image data; and adaptively determine the liquid level height inside the infusion bag based on the transparency.

[0207] Specifically, when the liquid is transparent, the liquid level is determined based on the visible light image data and the structured light image data; when the liquid is turbid or opaque, the liquid level is determined based on the non-contact distance data.

[0208] An adjustment module is used to adaptively identify and count droplets in an infusion tube based on ambient light conditions and droplet characteristics. Specifically, when the ambient light conditions meet a first threshold and the droplet characteristics meet a second threshold, the droplets are identified and counted based on visible light image data; when they do not meet the threshold, the droplets are identified and counted based on structured light image data; and the sensing parameters of the liquid level height are dynamically adjusted based on the visible light image data and the structured light image data.

[0209] This system aims to address the problems of inaccurate monitoring data and poor system robustness caused by traditional intravenous infusion monitoring in complex clinical environments, due to factors such as differences in infusion devices, variations in ambient lighting, and diverse drug properties. By dividing the monitoring function into detection, judgment, and adjustment modules, this system can work collaboratively to achieve intelligent acquisition of multimodal data, adaptive judgment of fluid level, and intelligent adjustment of droplet count, thereby significantly improving the accuracy and reliability of monitoring.

[0210] The specific steps and principles of the image recognition-based real-time monitoring method for intravenous infusion have been described in the above embodiments and will not be repeated here. It should be emphasized that the image recognition-based real-time monitoring system for intravenous infusion proposed in this application, through its modular design, specifically implements the above method steps into operable hardware and / or software units.

[0211] Specifically, the detection module can be understood as a hardware unit integrating a series of sensors and data acquisition interfaces, with the aim of comprehensively and accurately acquiring multimodal data during the infusion process. For example, the detection module may include a visible light camera for acquiring visible light image data; a non-contact distance sensor (such as an ultrasonic sensor or a laser rangefinder) for acquiring non-contact distance data; and a structured light system consisting of a structured light projector and a structured light camera for acquiring structured light image data. These sensors can be integrated into a compact monitoring device or installed in a distributed manner near the infusion point, transmitting data to the subsequent processing unit via wired or wireless means.

[0212] The judgment module can be understood as a computing unit containing a processor, memory, and preset algorithms. Its purpose is to intelligently determine the liquid transparency and adaptively determine the liquid level height based on the data acquired by the detection module. For example, this judgment module can be a microcontroller or embedded system running image processing algorithms and decision logic. In one implementation, the judgment module can receive visible light image data and evaluate the liquid transparency by analyzing features such as brightness, contrast, and transmittance. Based on the evaluation results, the judgment module can call different liquid level detection algorithms: when the liquid is transparent, it fuses visible light image data and structured light image data to accurately calculate the liquid level height; when the liquid is turbid or opaque, it prioritizes non-contact distance data for liquid level height estimation.

[0213] The adjustment module can be understood as another computational unit, or software logic integrated with the judgment module. Its purpose is to adaptively identify and count droplets based on environmental conditions and droplet characteristics, and dynamically adjust sensing parameters. For example, the adjustment module can monitor data from the ambient light sensor in real time and analyze the morphological characteristics of droplets in visible light images. When the lighting conditions are good and the droplet characteristics are clear, the adjustment module can activate the droplet recognition algorithm based on visible light images; when the lighting conditions are poor or the droplet morphology is blurred, it switches to the droplet recognition algorithm based on structured light images, utilizing its three-dimensional information for more robust counting. In addition, the adjustment module can also dynamically adjust the parameters in the liquid level sensing algorithm based on factors such as the deformation of the infusion bag and changes in the amount of medication, such as adjusting the image processing threshold, the range of the region of interest (ROI), or the intensity of the filter, to ensure the accuracy and stability of monitoring throughout the infusion process.

[0214] Traditional intravenous infusion monitoring systems often suffer from insufficient monitoring accuracy, poor robustness, and a lack of adaptability when facing complex and ever-changing clinical environments. For example, single image recognition methods struggle to cope with drug solutions of varying transparency, changing ambient lighting, and differences in infusion devices, easily leading to inaccurate estimation of fluid level and errors in droplet counting.

[0215] To address this issue, the proposed image recognition-based real-time monitoring system for intravenous infusion significantly improves the intelligence and reliability of monitoring by introducing multimodal data acquisition, modular processing, and adaptive adjustment mechanisms. The system acquires visible light image data, non-contact distance data, and structured light image data through a detection module, providing rich and complementary information sources for subsequent intelligent judgment and adjustment. The judgment module adaptively selects the liquid level determination method based on the liquid's transparency, effectively solving the challenges posed by different drug characteristics. The adjustment module intelligently switches between droplet recognition and counting strategies and dynamically adjusts sensing parameters based on ambient lighting conditions and droplet characteristics, ensuring accuracy under various complex conditions. Therefore, this system effectively overcomes many problems encountered by traditional monitoring systems in complex clinical environments, reduces the risk of false alarms and missed alarms, alleviates the burden on medical staff, and thus improves the safety and efficiency of intravenous infusion.

[0216] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this invention.

Claims

1. A method for real-time monitoring of intravenous infusion process based on image recognition, characterized in that, include: Acquire visible light image data, non-contact distance data, and structured light image data; The transparency of the liquid inside the infusion bag is determined based on the visible light image data. The liquid level inside the infusion bag is adaptively determined based on the transparency. Specifically, when the liquid is transparent, the liquid level is determined based on the visible light image data and the structured light image data; when the liquid is turbid or opaque, the liquid level is determined based on the non-contact distance data. Based on ambient lighting conditions and droplet characteristics, droplets within the infusion tube are adaptively identified and counted. Specifically, when the ambient lighting conditions meet a first threshold and the droplet characteristics meet a second threshold, the droplets are identified and counted based on the visible light image data; when they do not meet the threshold, the droplets are identified and counted based on the structured light image data. The sensing parameters of the liquid level height are dynamically adjusted based on the visible light image data and the structured light image data. The step of adaptively determining the liquid level in the infusion bag based on the transparency includes: The visible light image data and the structured light image data are analyzed to quantify the optical properties of the liquid inside the infusion bag and construct the liquid optical property curve. Based on the liquid optical property curve, the confidence weights of optical sensing and non-contact distance sensing are dynamically adjusted. Based on the confidence weight, the liquid level height is obtained by fusing the optical liquid level detection results and the non-contact distance liquid level detection results. The adaptive identification and counting of droplets within the infusion tube based on ambient lighting conditions and droplet characteristics includes: Install a piezoelectric sensor at the end of the infusion drip tube; Acquire the electrical signal output by the piezoelectric sensor; The electrical signal is filtered to obtain a filtered electrical signal; Set the signal threshold; When the amplitude of the filtered electrical signal exceeds the signal threshold, it is identified as a droplet shedding event; Analyze the waveform characteristics of the droplet detachment event; Based on the waveform characteristics, the droplet detachment event was confirmed; After confirming the droplet detachment event, increment the droplet counter; The infusion drip rate is calculated based on the changes of the droplet counter within a preset time window; The step of calculating the infusion drip rate based on the change of the droplet counter within a preset time window includes: Monitor the vibration frequency spectrum of the infusion drip to identify background vibration modes that are unrelated to droplet shedding; The length of the preset time window used to calculate the infusion drip rate is dynamically adjusted based on the vibration frequency spectrum of the infusion drip tube. Morphological analysis was performed on the droplets at the end of the infusion tube to estimate the average volume of a single droplet; The droplet counter is weighted and corrected based on the average volume of the individual droplets; The infusion drip rate is calculated based on the changes of the weighted and corrected droplet counter within the preset time window; The step of weighting and correcting the droplet counter based on the average volume of the individual droplets includes: Real-time monitoring of the temperature of the medication solution inside the infusion bag; Based on the temperature, dynamically query the physical parameters of the drug solution, such as viscosity and surface tension, as a function of temperature. The average volume estimate of the individual droplet is corrected based on the temperature and the physical parameters; A microfluidic sensor is installed at the end of the infusion drip tube; The microfluidic sensor is used to detect the fluid dynamics parameters during droplet detachment; The average volume estimate of a single droplet is calibrated based on the fluid dynamic parameters detected by the microfluidic sensor. The droplet counter is weighted and corrected based on the average volume estimate of a single droplet after correction and calibration. The method further includes: The inner wall of the infusion drip tube is periodically inspected using the visible light image data and the structured light image data, and the parameters of the inner wall of the drip tube in the droplet volume estimation model are adjusted according to the inspection results.

2. The method for real-time monitoring of intravenous infusion process based on image recognition according to claim 1, characterized in that, The periodic inspection of the inner wall of the infusion drip using the visible light image data and the structured light image data, and the adjustment of the drip inner wall parameters in the droplet volume estimation model based on the inspection results, includes: Analyze the visible light image data to extract the brightness, color distribution, and texture features of the inner wall of the infusion drip tube; Analyze the structured light image data to extract the pattern distortion, contrast changes, and three-dimensional point cloud data of the inner wall of the infusion drip tube; By fusing the visible light image data and the structured light image data, the response differences of different modal data in the defect region are compared; Based on the differences in response, the wear, cracks, scratches, drug deposits, crystals, or biofilms on the inner wall can be distinguished. Based on the differentiation results, adjust the surface roughness coefficient, effective inner diameter, and wetting angle in the droplet volume estimation model.

3. The method for real-time monitoring of intravenous infusion process based on image recognition according to claim 2, characterized in that, The process of fusing the visible light image data and the structured light image data, and comparing the response differences of the different modal data in the defect region, includes: The visible light image data and the structured light image data are time-stamped and spatially registered. Based on the visible light image data after time-stamp synchronization and spatial registration, multi-frame sequence analysis is performed to obtain the visible light response sequence and the structured light response sequence; Establish spatiotemporal correlation rules between the visible light response sequence and the structured light response sequence; Based on the spatiotemporal correlation rules, the visible light response sequence and the structured light response sequence are compared to identify patterns of difference in time or space.

4. A real-time monitoring module for intravenous infusion process based on image recognition, characterized in that, include: The acquisition unit is used to acquire visible light image data, non-contact distance data, and structured light image data; The judgment unit is used to determine the transparency of the liquid inside the infusion bag based on the visible light image data; A determining unit is configured to adaptively determine the liquid level height inside the infusion bag based on the transparency. Specifically, when the liquid is transparent, the liquid level is determined based on the visible light image data and the structured light image data; when the liquid is turbid or opaque, the liquid level is determined based on the non-contact distance data. The identification unit is used to adaptively identify and count droplets in an infusion tube based on ambient light conditions and droplet characteristics. Specifically, when the ambient light conditions meet a first threshold and the droplet characteristics meet a second threshold, the droplets are identified and counted based on the visible light image data; when they do not meet the threshold, the droplets are identified and counted based on the structured light image data. The sensing parameters of the liquid level height are dynamically adjusted based on the visible light image data and the structured light image data. The step of adaptively determining the liquid level in the infusion bag based on the transparency includes: The visible light image data and the structured light image data are analyzed to quantify the optical properties of the liquid inside the infusion bag and construct the liquid optical property curve. Based on the liquid optical property curve, the confidence weights of optical sensing and non-contact distance sensing are dynamically adjusted. Based on the confidence weight, the liquid level height is obtained by fusing the optical liquid level detection results and the non-contact distance liquid level detection results. The adaptive identification and counting of droplets within the infusion tube based on ambient lighting conditions and droplet characteristics includes: Install a piezoelectric sensor at the end of the infusion drip tube; Acquire the electrical signal output by the piezoelectric sensor; The electrical signal is filtered to obtain a filtered electrical signal; Set the signal threshold; When the amplitude of the filtered electrical signal exceeds the signal threshold, it is identified as a droplet shedding event; Analyze the waveform characteristics of the droplet detachment event; Based on the waveform characteristics, the droplet detachment event was confirmed; After confirming the droplet detachment event, increment the droplet counter; The infusion drip rate is calculated based on the changes of the droplet counter within a preset time window; The step of calculating the infusion drip rate based on the change of the droplet counter within a preset time window includes: Monitor the vibration frequency spectrum of the infusion drip to identify background vibration modes that are unrelated to droplet shedding; The length of the preset time window used to calculate the infusion drip rate is dynamically adjusted based on the vibration frequency spectrum of the infusion drip tube. Morphological analysis was performed on the droplets at the end of the infusion tube to estimate the average volume of a single droplet; The droplet counter is weighted and corrected based on the average volume of the individual droplets; The infusion drip rate is calculated based on the changes of the weighted and corrected droplet counter within the preset time window; The step of weighting and correcting the droplet counter based on the average volume of the individual droplets includes: Real-time monitoring of the temperature of the medication solution inside the infusion bag; Based on the temperature, dynamically query the physical parameters of the drug solution, such as viscosity and surface tension, as a function of temperature. The average volume estimate of the individual droplet is corrected based on the temperature and the physical parameters; A microfluidic sensor is installed at the end of the infusion drip tube; The microfluidic sensor is used to detect the fluid dynamics parameters during droplet detachment; The average volume estimate of a single droplet is calibrated based on the fluid dynamic parameters detected by the microfluidic sensor. The droplet counter is weighted and corrected based on the average volume estimate of a single droplet after correction and calibration. Also includes: The inner wall of the infusion drip tube is periodically inspected using the visible light image data and the structured light image data, and the parameters of the inner wall of the drip tube in the droplet volume estimation model are adjusted according to the inspection results.

5. A real-time monitoring system for intravenous infusion based on image recognition, characterized in that, include: The detection module is used to acquire visible light image data, non-contact distance data, and structured light image data; The judgment module is used to determine the transparency of the liquid inside the infusion bag based on the visible light image data; and adaptively determine the liquid level height inside the infusion bag based on the transparency. Specifically, when the liquid is transparent, the liquid level is determined based on the visible light image data and the structured light image data; when the liquid is turbid or opaque, the liquid level is determined based on the non-contact distance data. An adjustment module is used to adaptively identify and count droplets in an infusion tube based on ambient light conditions and droplet characteristics. Specifically, when the ambient light conditions meet a first threshold and the droplet characteristics meet a second threshold, the droplets are identified and counted based on visible light image data; when they do not meet the threshold, the droplets are identified and counted based on structured light image data; and the sensing parameters for the liquid level height are dynamically adjusted based on the visible light image data and the structured light image data. The step of adaptively determining the liquid level in the infusion bag based on the transparency includes: The visible light image data and the structured light image data are analyzed to quantify the optical properties of the liquid inside the infusion bag and construct the liquid optical property curve. Based on the liquid optical property curve, the confidence weights of optical sensing and non-contact distance sensing are dynamically adjusted. Based on the confidence weight, the liquid level height is obtained by fusing the optical liquid level detection results and the non-contact distance liquid level detection results. The adaptive identification and counting of droplets within the infusion tube based on ambient lighting conditions and droplet characteristics includes: Install a piezoelectric sensor at the end of the infusion drip tube; Acquire the electrical signal output by the piezoelectric sensor; The electrical signal is filtered to obtain a filtered electrical signal; Set the signal threshold; When the amplitude of the filtered electrical signal exceeds the signal threshold, it is identified as a droplet shedding event; Analyze the waveform characteristics of the droplet detachment event; Based on the waveform characteristics, the droplet detachment event was confirmed; After confirming the droplet detachment event, increment the droplet counter; The infusion drip rate is calculated based on the changes of the droplet counter within a preset time window; The step of calculating the infusion drip rate based on the change of the droplet counter within a preset time window includes: Monitor the vibration frequency spectrum of the infusion drip to identify background vibration modes that are unrelated to droplet shedding; The length of the preset time window used to calculate the infusion drip rate is dynamically adjusted based on the vibration frequency spectrum of the infusion drip tube. Morphological analysis was performed on the droplets at the end of the infusion tube to estimate the average volume of a single droplet; The droplet counter is weighted and corrected based on the average volume of the individual droplets; The infusion drip rate is calculated based on the changes of the weighted and corrected droplet counter within the preset time window; The step of weighting and correcting the droplet counter based on the average volume of the individual droplets includes: Real-time monitoring of the temperature of the medication solution inside the infusion bag; Based on the temperature, dynamically query the physical parameters of the drug solution, such as viscosity and surface tension, as a function of temperature. The average volume estimate of the individual droplet is corrected based on the temperature and the physical parameters; A microfluidic sensor is installed at the end of the infusion drip tube; The microfluidic sensor is used to detect the fluid dynamics parameters during droplet detachment; The average volume estimate of a single droplet is calibrated based on the fluid dynamic parameters detected by the microfluidic sensor. The droplet counter is weighted and corrected based on the average volume estimate of a single droplet after correction and calibration. Also includes: The inner wall of the infusion drip tube is periodically inspected using the visible light image data and the structured light image data, and the parameters of the inner wall of the drip tube in the droplet volume estimation model are adjusted according to the inspection results.

Images