Infusion bottle liquid level monitoring method and system based on cloud and edge computing

Abstract

The invention discloses an infusion bottle liquid level monitoring method and system based on cloud and edge computing, which overcome the problems of complicated installation, easy interference, high cost and the like of the traditional system through computer vision and intelligent analysis technologies, provide a more convenient and reliable liquid level monitoring means, and improve infusion management efficiency and medical service quality. Meanwhile, the technical innovation also provides reference for intelligent monitoring and management in other fields, and promotes technical progress and intelligent development of the medical industry; moreover, the accuracy and reliability of liquid level monitoring during infusion are improved, the medical safety level can be improved, the risk of misoperation is reduced, and the credibility and speciality of medical services are enhanced.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision technology and artificial intelligence technology, and in particular to a method and system for monitoring the liquid level of an infusion bottle based on cloud and edge computing. Background Art

[0002] With the advancement of medical technology, people's requirements for real-time monitoring and safety of medical equipment are increasing. In the medical field, infusion is one of the common treatment methods, especially during surgery, emergency rescue and long-term treatment. However, traditional infusion bottle level monitoring systems usually rely on sensor technology. Although this method is effective, it also brings a series of problems. First, the installation process of the sensor is relatively cumbersome and complicated, requiring professional personnel to operate, which increases the investment cost and time cost of the equipment. Secondly, the sensor is easily disturbed by the external environment, such as temperature changes or electromagnetic interference from other equipment, which may lead to false alarms or monitoring failures. In addition, although the cost of the sensor itself is not high, its long-term use may lead to a decrease in accuracy and require regular maintenance and replacement, which increases the maintenance cost and difficulty. These problems limit the widespread application of sensor technology in medical infusion level monitoring, prompting people to seek more reliable and efficient alternatives.

[0003] In recent years, with the rapid development of computer vision, Internet of Things and embedded technology, liquid level monitoring systems based on image recognition and intelligent analysis have gradually attracted attention. Therefore, if a liquid level monitoring system for infusion bottles based on cloud and edge computing can be designed, it will inevitably overcome the problems of cumbersome installation, susceptibility to interference, and high cost of traditional systems, and provide a more convenient and reliable liquid level monitoring method to improve infusion management efficiency and medical service quality. Summary of the invention

[0004] In view of this, one of the purposes of the present invention is to use computer vision technology and artificial intelligence algorithms to propose a cloud and edge computing-based infusion bottle liquid level monitoring method, which can solve the accuracy and cumbersome operation problems of traditional liquid level monitoring methods. The second purpose of the present invention is to propose an infusion bottle liquid level monitoring system based on cloud and edge computing.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] The method for monitoring the liquid level of an infusion bottle based on cloud and edge computing includes:

[0007] Step S1: Acquire an infusion scene image, and annotate and preprocess the infusion bottle and liquid level target in the image to prepare a complete data set;

[0008] Step S2: Divide the acquired data set into a training set, a validation set, and a test set in proportion, and perform model training;

[0009] Step S3: using the model file trained by the deep learning network, using the deep learning network to monitor in real time whether there is an infusion bottle or an infusion bottle filled with liquid medicine in the video stream collected by the image acquisition device, if the target is not detected, the monitoring is continued in a loop; if it is detected, the target is framed with a detection frame, the height of the infusion bottle and the liquid medicine is calculated by the coordinate information of the detection frame, and the remaining proportion of the liquid medicine in each infusion bottle is further calculated and displayed in real time;

[0010] Step S4: When it is detected that the remaining amount of the liquid medicine is lower than the threshold value, a control signal is sent, and an external device is used to control the liquid dripping to stop so as to proceed to the next step of processing.

[0011] Furthermore, the processed data is uploaded to the cloud platform of the Internet of Things through the protocol, and the external device obtains relevant data from the cloud platform of the Internet of Things through wired or wireless communication. The data includes but is not limited to the remaining liquid level, alarm liquid level and online status of the device.

[0012] Further, in step S1, image data of the infusion bottle is captured from multiple angles using an image acquisition tool, the images including infusion bottles of different quantities, different liquid levels, containing different colors of liquid medicine, and with or without label obstructions, and the captured scenes include natural light environments and night lighting environments.

[0013] Furthermore, the labeling refers to labeling the collected images using labeling tools, and the infusion bottle and the liquid in the infusion bottle in each image are accurately labeled with labels and rectangular detection frames; the preprocessing refers to data processing of the collected data, including but not limited to screening data, removing noise, and adjusting image size and brightness to ensure data quality and consistency.

[0014] Furthermore, in step S2, the data is divided into a ratio of 7:2:1, where 70% is used for training, 20% is used for validation, and 10% is used for testing. The YOLOv5 deep learning network is used for model training, validation, and testing; and data enhancement processing is performed on the training set.

[0015] Further, in step S3, when the system detects multiple targets, it will identify and use a rectangular detection frame to select multiple infusion bottles and the liquid medicine in the bottles, and the steps include:

[0016] First, extract the six parameters of each detection box: the horizontal coordinate of the upper left corner, the vertical coordinate of the upper left corner, the horizontal coordinate of the lower right corner, the vertical coordinate of the lower right corner, the confidence and the label. By default, the detection boxes are sorted from high to low according to the confidence and output;

[0017] When it is difficult to quickly determine which frame corresponds to which infusion bottle or liquid medicine using these parameters directly, all detection frames are sorted from small to large according to the horizontal coordinate of the upper left corner point, and the horizontal coordinate difference of the upper left corner points of adjacent detection frames is compared. If the difference is less than 20 pixels, it is considered that the two detection frames belong to the same infusion bottle and the liquid medicine therein; otherwise, it is considered that the two detection frames do not belong to the same infusion bottle and the liquid medicine therein;

[0018] By comparing the differences in the horizontal coordinates of the upper left corner points of all adjacent detection frames in turn, all infusion bottles can be matched with their drug liquid detection frames and the remaining proportion of the liquid can be calculated, and finally the liquid level information arranged in order from left to right can be obtained, so as to clearly obtain the remaining amount of liquid in each infusion bottle containing drug liquid.

[0019] Furthermore, the improved YOLOv5 network is used to train the model;

[0020] The improved YOLOv5s network includes an input terminal Input, a backbone network Backbone, and a prediction layer Prediction. The input terminal Input uses Mosaic data enhancement to enhance the diversity of training data; the backbone network Backbone includes a Conv module and a C3 module; the bottleneck Neck adopts an FPN+PAN structure.

[0021] Furthermore, in step S3, the remaining proportion of the medicine in each infusion bottle is calculated and displayed in real time, and a time interval is set for voice reminder; in step S4, when it is detected that the liquid level ratio is lower than the set threshold, a voice reminder is set and the dripping is controlled to stop.

[0022] The second object of the present invention is to provide an infusion bottle liquid level monitoring system based on cloud and edge computing, which is used to implement the method as described above. The system includes

[0023] The liquid level monitoring unit includes an image acquisition device and an edge computing device for real-time acquisition of infusion bottle liquid level video stream data. The edge computing device uses a deep learning network to monitor in real time whether there is an infusion bottle or an infusion bottle filled with liquid medicine in the video stream acquired by the image acquisition device. If the target is not detected, the monitoring is continuously performed in a loop; if detected, the target is framed with a detection frame, and the height of the infusion bottle and the liquid medicine is calculated through the coordinate information of the detection frame, and the remaining proportion of the liquid medicine in each infusion bottle is further calculated. Finally, the remaining proportion data of the liquid medicine and the set alarm liquid level data are transmitted to the cloud platform of the Internet of Things;

[0024] The liquid level control unit includes a control chip and a drip rate regulator. The control chip communicates with a cloud platform via wireless communication to obtain the remaining amount and threshold of the liquid medicine. When the control chip detects that the remaining amount of the liquid medicine is lower than the threshold, the control motor is controlled to work and then the drip rate regulator is controlled to stop dripping.

[0025] The display unit is used to receive real-time data and images sent by the cloud platform and display them in real time.

[0026] Furthermore, the display unit includes a small program module for a mobile device, a GUI module for local monitoring of real-time images, and a GUI module for cloud monitoring of real-time images.

[0027] The beneficial effects of the present invention are:

[0028] (1) This system uses a camera to capture the liquid level status of the infusion bottle in real time and transmits the image data to the central processing unit for analysis. Through advanced computer vision algorithms and intelligent analysis technology, the system can accurately detect liquid level changes and issue an alarm. Combined with Internet of Things technology, this data can be uploaded to the cloud in real time, and medical staff can remotely monitor the patient's infusion status through mobile phones or computers, thereby reducing the burden of manual monitoring and improving work efficiency;

[0029] (2) With the support of embedded technology, the liquid level monitoring system of the present invention is more efficient and energy-saving, and can be applied to various medical environments; the introduction of edge computing allows the system to complete data processing on devices close to the data generation end, reducing latency and improving the real-time performance and reliability of the system; cloud computing provides powerful data storage and analysis capabilities, supporting more complex algorithm analysis and large-scale data processing;

[0030] (3) The present invention not only solves the limitations of traditional sensor monitoring, but also provides a more convenient and reliable solution for medical infusion. The system simplifies the installation and configuration process, avoids the impact of changes in liquid properties on sensors, and enhances the stability and reliability of monitoring. At the same time, the advancement of image recognition technology also improves the accuracy and sensitivity of the system;

[0031] (4) The liquid level monitoring system based on image recognition and intelligent analysis of the present invention has great application potential in the medical field. It can improve the safety and efficiency of the infusion process, reduce the workload of medical staff, and improve the medical experience of patients;

[0032] (5) The present invention aims to explore and construct a liquid level monitoring system based on computer vision, the Internet of Things, embedded technology, cloud computing and edge computing, which can overcome the limitations of traditional sensor technology, provide efficient and reliable monitoring solutions for the medical infusion process, promote the automated management and intelligent development of the medical industry, and lay the foundation for future smart medical care.

[0033] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description and the preceding claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0035] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0036] Figure 2 This is a schematic diagram of the collected infusion bottles;

[0037] Figure 3 This is the improved YOLOv5 structure diagram;

[0038] Figure 4 It is the structure diagram of C3 and SPPF of Example 2;

[0039] Figure 5 This is a flow chart of the overall solution of liquid level monitoring in Example 4;

[0040] Figure 6 This is a schematic diagram of the liquid level monitoring system architecture of Example 5;

[0041] Figure 7 Schematic diagram of the liquid level monitoring applet interface;

[0042] Figure 8 This is the flow chart of real-time liquid level monitoring on the GUI interface;

[0043] Fig. 9 Real-time monitoring images of single to five infusion bottles;

[0044] Fig.10 Evaluation diagram for target detection training metrics and monitoring system. DETAILED DESCRIPTION

[0045] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the preferred embodiments are only for illustrating the present invention, rather than for limiting the protection scope of the present invention.

[0046] The infusion bottle liquid level monitoring method based on cloud and edge computing of the present invention comprises:

[0047] Step S1: Acquire an infusion scene image, and annotate and preprocess the infusion bottles and liquid level targets in the image to prepare a complete data set; annotation refers to annotating the acquired images using annotation tools, and the infusion bottles and liquids in the infusion bottles in each image are accurately labeled with labels and rectangular detection frames; preprocessing refers to processing the acquired data, including but not limited to filtering data, removing noise, and adjusting image size and brightness, to ensure data quality and consistency.

[0048] Step S2: Divide the acquired data set into a training set, a validation set, and a test set in proportion, and perform model training; in step S2 of this embodiment, the data is divided according to a ratio of 7:2:1, of which 70% is used for training, 20% is used for validation, and 10% is used for testing, and the YOLOv5 deep learning network is used for model training, validation, and testing; and the training set is subjected to data enhancement processing.

[0049] Step S3: using the model file trained by the deep learning network, the deep learning network is used to monitor in real time whether there is an infusion bottle or an infusion bottle filled with medicine in the video stream collected by the image acquisition device. If the target is not detected, the monitoring is continued in a loop; if detected, the target is framed with a detection frame, and the height of the infusion bottle and the medicine is calculated through the coordinate information of the detection frame, and the remaining proportion of the medicine in each infusion bottle is further calculated and displayed in real time; the processed data is uploaded to the cloud platform of the Internet of Things through the protocol, and the external device obtains relevant data from the cloud platform through wired or wireless communication methods, and the data includes but is not limited to the remaining liquid level, alarm liquid level and online status of the device.

[0050] In step S3, when the system detects multiple targets, it will identify and use a rectangular detection frame to select multiple infusion bottles and the liquid medicine in the bottles. The steps include:

[0051] First, extract the six parameters of each detection box: the horizontal coordinate of the upper left corner, the vertical coordinate of the upper left corner, the horizontal coordinate of the lower right corner, the vertical coordinate of the lower right corner, the confidence and the label. By default, the detection boxes are sorted from high to low according to the confidence and output;

[0052] When it is difficult to quickly determine which frame corresponds to which infusion bottle or liquid medicine using these parameters directly, all detection frames are sorted from small to large according to the horizontal coordinate of the upper left corner point, and the horizontal coordinate difference of the upper left corner points of adjacent detection frames is compared. If the difference is less than 20 pixels, it is considered that the two detection frames belong to the same infusion bottle and the liquid medicine therein; otherwise, it is considered that the two detection frames do not belong to the same infusion bottle and the liquid medicine therein;

[0053] By comparing the difference of the horizontal coordinates of the upper left corner points of all adjacent detection frames in sequence, all infusion bottles can be matched with their drug liquid detection frames and the remaining proportion of the liquid can be calculated, and finally the information of each liquid level arranged in order from left to right can be obtained, so as to clearly obtain the remaining amount of liquid in each infusion bottle containing drug liquid;

[0054] When making a data set, it is necessary to draw a detection frame for each target in the image; when framing the liquid level of the infusion bottle, if the liquid level is a horizontal line, accurately set the upper boundary of the detection frame to coincide with this horizontal line; if the liquid surface of the drug solution is elliptical, the upper boundary of the detection frame should coincide with the major axis of the ellipse, otherwise, errors will occur.

[0055] Step S4: When it is detected that the remaining amount of the liquid medicine is lower than the threshold value, a control signal is sent, and an external device is used to control the liquid dripping to stop so as to proceed to the next step of processing.

[0056] Embodiment 1

[0057] In this embodiment, the specific process of establishing the data set in step S1 is described in detail, including:

[0058] (1) Data collection: In real-world scenarios, a high-definition camera model DS-2FA1202-B was used to capture image data of infusion bottles from multiple angles. These images cover a variety of situations, including infusion bottles with different quantities, different liquid levels, different colors of liquid, and with or without labels. The captured scenes include natural light environments and night lighting environments.

[0059] (2) Data labeling: Use the Labelimg labeling tool to label the collected images. The infusion bottle and the liquid in the infusion bottle in each image are accurately labeled with labels and rectangular detection frames. This labeling information is crucial for the subsequent training of the supervised learning model.

[0060] (3) Preprocessing: The collected data were preprocessed, including data screening, noise removal, and image size and brightness adjustment to ensure data quality and consistency.

[0061] (4) Data partitioning: A dataset containing 58,026 image samples was established. When using the YOLOv5 deep learning network for model training, validation, and testing, the data was divided in a ratio of 7:2:1, with 70% for training, 20% for validation, and 10% for testing. This was used to train and evaluate the performance of the liquid level monitoring model.

[0062] (5) Data enhancement: The training set was enhanced by performing data enhancement operations, including random brightness changes, cropping, translation, left-right flipping, and adding salt and pepper noise, to increase data diversity and improve robustness, making images more recognizable during monitoring.

[0063] This design mainly monitors the liquid level of the following infusion bottles in real time and determines the remaining amount of liquid medicine, including: empty infusion bottles, infusion bottles containing liquid, single infusion bottles, multiple infusion bottles, infusion bottles with different liquid levels, infusion bottles containing different colors of liquid medicine, and infusion bottles with labels and other obstructions to the liquid medicine as the collected data set, with a total of 58,026 images. The collected infusion bottles are as follows: Figure 2 shown.

[0064] Embodiment 2

[0065] In this embodiment, the improved YOLOv5 network used in step S2 is further described: YOLO is a one-stage target detection algorithm that can complete the recognition of object categories and bounding boxes in an image in one forward propagation. The YOLOv5 network model is divided into four versions, YOLOv5s, YOLOv5m, YOLOv5l and YOLOv5x, according to the network width and depth. In order to achieve the lightweight of the network model, the present invention takes the YOLOv5s with the smallest network width and depth as the improvement object.

[0066] The YOLOv5s network consists of four parts: Input, Backbone, Neck, and Prediction.

[0067] 1) Input: YOLOv5 uses Mosaic data enhancement, which is splicing through random scaling, random cropping, and random arrangement to enhance the diversity of training data.

[0068] 2) Backbone:

[0069] a) The Conv module encapsulates convolution (Conv2d), BatchNorm2d (BN), and SiLU activation functions to extract image features.

[0070] b) The C3 module contains three Conv convolution modules and N Bottleneck modules, which are used to learn residual features. Its structure has two branches, one of which is a stacked Conv convolution module and N Bottleneck modules, and the other is a branch that only uses Conv modules. Finally, the two branches are concat-operated and output through a Conv module.

[0071] c) SPP (Spatial Pyramid Pooling), first through a Conv module to halve the input channels, then perform Maxpooling with kernel-size 5, 9, 13, then concat the results of the three Maxpooling with the data without Maxpooling, and finally pass through a Conv module to output.

[0072] d) SPPF uses a kernel-size of 5×5 instead of 5, 9, and 13 in SPP to reduce the amount of calculation and improve the speed.

[0073] 3) Neck: YOLOv5 adopts FPN+PAN structure.

[0074] a) The FPN (Feature Pyramid Network) structure performs upsampling from top to bottom, so that the underlying feature maps contain richer semantic information.

[0075] b) Two PAN (Pyramid Attention Network) structures are added after the FPN structure. The PAN structure downsamples from bottom to top so that the top-level features contain more position information.

[0076] Through the collaborative work of the above modules, YOLOv5s has significantly reduced the amount of calculation while maintaining high performance, and achieved a lightweight network model. Figure 3 shown. Figure 4 This is the structure diagram of C3 and SPPF.

[0077] Embodiment 3

[0078] This embodiment is aimed at the abnormal monitoring in step S3, specifically refers to printing the remaining proportion of the liquid medicine in the infusion bottle on the display interface after the remaining amount of the liquid medicine is monitored, so that the user can monitor it in real time. However, the data will jump continuously, and occasionally an abnormal value will appear, affecting the monitoring. For this situation, this embodiment adopts the median filtering method to deal with it.

[0079] Median filtering smoothes data by taking the median of a set of values, thereby effectively reducing noise, stabilizing data and eliminating gross errors. The design of the present invention adopts median filtering every 60 liquid level ratio data, which can effectively reduce short-term fluctuations and provide a smoother curve, which is very important for continuous system monitoring.

[0080] (1) Stable data: When displaying the remaining amount of liquid, the raw data may fluctuate greatly, which will cause the displayed data to jump quickly, making it difficult to read and observe. By using median filtering, the data can be effectively smoothed, making the liquid remaining amount display process more stable and easy to monitor and record;

[0081] (2) Eliminate gross errors: The data may contain some outliers (i.e., gross errors), which will significantly affect statistics such as the mean. Median filtering replaces the mean with the median, which is not affected by outliers and can better reflect the true trend of the data and eliminate errors caused by outliers.

[0082] Embodiment 4

[0083] In this embodiment, liquid level monitoring mainly includes two main parts: real-time detection of infusion bottles and the medicinal liquid therein, and judgment of the remaining proportion of the medicinal liquid. The specific process is as follows: Use the YOLOv5 network to monitor in real time whether there is an infusion bottle or an infusion bottle filled with medicinal liquid in the video stream collected by the camera. If the target is not detected, the monitoring is continued in a loop; if detected, the target is framed with a detection frame, and the height of the infusion bottle and the medicinal liquid is calculated through the coordinate information of the frame, and the remaining proportion of the medicinal liquid in each infusion bottle is further calculated and displayed in real time, and a voice reminder is set every 5 minutes. When it is detected that the liquid level ratio is lower than the set threshold, a voice reminder is given every three seconds, and the motor is controlled to push the gear in the drip regulator on the infusion tube to control the dripping to stop. The overall solution flow chart of liquid level monitoring is shown below. Figure 5 shown.

[0084] Embodiment 5

[0085] Based on the design concept of the aforementioned infusion bottle liquid level monitoring method based on cloud and edge computing, this embodiment provides an infusion bottle liquid level monitoring system based on cloud and edge computing, such as Figure 6 As shown, the system specifically includes:

[0086] (1) Liquid level monitoring unit: including an image acquisition device (a high-definition camera in this embodiment) for real-time acquisition of video stream data of the liquid level of the infusion bottle, and using a data preprocessing module to preprocess the video stream data and then transmit it to a cloud platform, the cloud platform including a model file trained by a deep learning network (the training method and procedure have been disclosed in the aforementioned method), detecting whether there is an infusion bottle or an infusion bottle filled with liquid medicine in the video stream, if the target is not detected, continuous loop monitoring; if detected, the target is framed with a detection frame, the height of the infusion bottle and the liquid medicine is calculated by the coordinate information of the detection frame, and the remaining proportion of the liquid medicine in each infusion bottle is further calculated;

[0087] The hardware used by the cloud platform involved in this embodiment mainly includes an artificial intelligence computer, specifically using the Jetson Xavier NX in the NVIDIA series development board. The Jetson development board is a small and powerful artificial intelligence computer with a built-in SOC chip, which can be started with an SSD equipped with a system image. It supports a variety of neural network frameworks, including TensorFlow, PyTorch, Keras, Caffe / Caffe2, and MXNET. The models used in this system are all written in PyTorch, so it is the best choice to select the Jetson development board as the hardware carrier of the monitoring network. In the Jetson series, the GPU of Jetson Xavier NX has 384 cores, which is 1.5 times that of Jetson TX2, and is equipped with a deep learning accelerator and a visual accelerator. In addition, the CPU has been upgraded from 4 cores to 6 cores, the speed has been doubled, and the memory has also been correspondingly improved. These performance improvements are very important for the operation of AI algorithms.

[0088] As the hardware carrier for real-time monitoring of the remaining liquid in infusion bottles, the NVIDIA Jetson series development boards provide powerful AI processing capabilities and flexibility, and are suitable for the development and deployment of deep learning models.

[0089] (2) Liquid level control unit: including a control chip and a drip rate control regulator, the drip rate control regulator is provided with a control motor, the control chip maintains communication connection with the cloud platform through wireless communication (in this embodiment, through the OneNET mobile Internet of Things platform), when the cloud platform detects that the remaining amount of the liquid medicine is lower than the threshold value, the control chip receives a stop signal from the cloud platform and uses the control motor to stop the drip rate control regulator from dripping;

[0090] In this embodiment, the control chip uses ESP32, which integrates traditional Bluetooth, low-power Bluetooth and Wi-Fi, and can achieve wireless communication between the network and the device without an external wireless module. ESP32 adopts a dual-core processor architecture, achieving a good balance between high performance and low power consumption processors.

[0091] ESP32 has built-in watchdog timer, general timer, special function register, 520KB RAM and 4MB ROM, supports multiple USART interfaces, and adapts to various communication protocols and peripheral connection requirements. Because this system needs to obtain real-time data from the OneNET mobile Internet of Things platform, the wireless communication module integrated in ESP32 makes it easier to connect to the OneNET platform. In addition, ESP32 is inexpensive and has a good cost-effectiveness.

[0092] This system needs to use a chip to control a mechanical device to push the gear in the drip rate regulator after the liquid level is less than the set threshold. Since pushing the drip rate regulator can be approximated as linear motion, this system uses the Yingpengfei 28K-2R-63.5 stepper electric cylinder as a mechanical device to push the drip rate regulator. The stepper electric cylinder is a two-phase four-wire screw fixed axis stepper electric cylinder. The thread lead of the electric cylinder is 5.08mm, which can meet the needs of this system to push the drip rate regulator. This stepper electric cylinder has the characteristics of no step loss, low heat, high efficiency, smooth and precise, large torque, and high-speed response, so that it can stably push the drip rate regulator.

[0093] In this embodiment, ESP32 is used as the main control microcontroller to communicate with the OneNET mobile IoT platform and control the stepper cylinder. It integrates Wi-Fi and low-power Bluetooth modules, and can achieve wireless communication without an external wireless module. It is inexpensive and cost-effective.

[0094] (3) A display unit, which is used to receive and display real-time data sent from the cloud platform. In this embodiment, the display unit includes a small program module for a mobile device and a GUI module for monitoring a host.

[0095] The applet can monitor the remaining amount of liquid in the infusion bottle in real time. When no liquid is detected, the interface displays the liquid level as 99%; when the remaining liquid is detected, the real-time remaining liquid amount is displayed, and the percentage of the alarm liquid level and the device status information are displayed at the same time. Figure 7 shown.

[0096] The GUI interface provides real-time monitoring of the remaining liquid in the infusion bottle for infusion patients. The interface contains four buttons: start monitoring, end monitoring, voice broadcast, and end broadcast. When the liquid level in the infusion bottle changes, the historical liquid level information will be updated. The infusion bottle is marked with a blue frame, and the liquid medicine is marked with a red frame. The ratio of the height of the red frame to the height of the blue frame can be used to obtain the remaining liquid ratio, which can be updated and displayed in real time. The GUI interface liquid level real-time monitoring flow chart is as follows Figure 8 shown.

[0097] The monitoring system of this embodiment is used to conduct effect tests. Fig. 9 The real-time monitoring images of single infusion bottle to five infusion bottles are obtained, and the target detection training indicators and monitoring system evaluation are as follows: Fig.10 The performance evaluation of the real-time monitoring system for the liquid level of the infusion bottle is shown in the following table:

[0098]

[0099]

[0100] It can be seen from the chart that the response speed and recognition accuracy of the monitoring system of the present invention are better than the existing technology, and as the number of samples in the data set increases, the response speed and recognition accuracy can be more significantly improved.

[0101] The system has the following innovative features:

[0102] (1) Intelligent liquid level monitoring: Compared with traditional liquid level monitoring methods, this invention realizes automatic recognition of infusion bottles and liquid medicine and accurate liquid level monitoring by combining computer vision technology and deep learning network. The coordinate information of the YOLOv5 rectangular detection frame can be used to accurately calculate the remaining percentage of liquid, thereby improving the accuracy and stability of liquid level monitoring.

[0103] (2) Drip control: When the remaining liquid volume is lower than the set threshold, the system will remind the medical staff through voice and control the motor to push the gear in the drip regulator on the infusion tube to stop dripping. This intelligent adjustment system can automatically adjust the drip speed to ensure that the remaining liquid volume is within a safe range, thereby improving the efficiency and safety of liquid management in medical infusion;

[0104] (3) Network structure improvement: By using the improved deep learning network, the system achieved a liquid level detection recognition rate of more than 97% in real-time scenarios, effectively solving the problem of inaccurate monitoring in traditional liquid level monitoring methods.

[0105] (4) Hardware-software-platform integration: In order to solve the current technical bottleneck of liquid level monitoring in my country, this system has developed an integrated "hardware + software + platform" liquid level monitoring safety system; through deep integration and organic linkage, it has achieved comprehensive optimization of data collection, aggregation, analysis and processing, and improved the efficiency and accuracy of infusion safety management.

[0106] (5) Combination of multiple monitoring methods: The innovation of the intelligent liquid level monitoring system based on cloud and edge computing proposed in this design is reflected in the use of multiple monitoring methods; monitoring personnel can monitor the liquid level in real time through the indoor system interface, while infusion patients can view the liquid level information at any time through the indoor web interface or applet. This diversified monitoring method not only improves the accuracy, precision and safety of monitoring, but also enhances the practicality and operability of the system; monitoring personnel and patients can choose the most appropriate monitoring method according to actual needs and scenarios, conveniently obtain liquid level information, and take necessary measures in a timely manner.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for monitoring the liquid level of an infusion bottle based on cloud and edge computing, characterized in that: The method comprises Step S1: Acquire an infusion scene image, and annotate and preprocess the infusion bottle and liquid level target in the image to prepare a complete data set; Step S2: Divide the acquired data set into a training set, a validation set, and a test set in proportion, and perform model training; Step S3: using the model file trained by the deep learning network, using the deep learning network to monitor in real time whether there is an infusion bottle or an infusion bottle filled with liquid medicine in the video stream collected by the image acquisition device, if the target is not detected, the monitoring is continued in a loop; if it is detected, the target is framed with a detection frame, the height of the infusion bottle and the liquid medicine is calculated by the coordinate information of the detection frame, and the remaining proportion of the liquid medicine in each infusion bottle is further calculated and displayed in real time; Step S4: When it is detected that the remaining amount of the liquid medicine is lower than the threshold value, a control signal is sent, and an external device is used to control the liquid dripping to stop so as to proceed to the next step of processing.

2. The method for monitoring the liquid level of an infusion bottle based on cloud and edge computing according to claim 1, characterized in that: The processed data is uploaded to the cloud platform of the Internet of Things through the protocol, and the external device obtains relevant data from the cloud platform of the Internet of Things through wired or wireless communication. The data includes but is not limited to the remaining liquid level, alarm liquid level and online status of the device.

3. The method for monitoring the liquid level of an infusion bottle based on cloud and edge computing according to claim 1, characterized in that: In step S1, an image acquisition tool is used to capture image data of the infusion bottle from multiple angles, wherein the images include infusion bottles of different quantities, different liquid levels, different colors of liquid medicine, with or without labels blocking them, and the captured scenes include natural light environments and night lighting environments.

4. The method for monitoring the liquid level of an infusion bottle based on cloud and edge computing according to claim 3 is characterized in that: The labeling refers to labeling the collected images using labeling tools, and the infusion bottle and the liquid in the infusion bottle in each image are accurately labeled with labels and rectangular detection frames; the preprocessing refers to data processing of the collected data, including but not limited to screening data, removing noise, and adjusting image size and brightness to ensure data quality and consistency.

5. The method for monitoring the liquid level of an infusion bottle based on cloud and edge computing according to claim 1, characterized in that: In step S2, the data is divided into a ratio of 7:2:1, where 70% is used for training, 20% is used for validation, and 10% is used for testing. The YOLOv5 deep learning network is used for model training, validation, and testing; and the training set is data enhanced.

6. The method for monitoring liquid level of an infusion bottle based on cloud and edge computing according to claim 1, characterized in that: In step S3, when the system detects multiple targets, it will identify and use a rectangular detection frame to select multiple infusion bottles and the liquid medicine in the bottles, and the steps include: First, extract the six parameters of each detection box: the horizontal coordinate of the upper left corner, the vertical coordinate of the upper left corner, the horizontal coordinate of the lower right corner, the vertical coordinate of the lower right corner, the confidence and the label. By default, the detection boxes are sorted from high to low according to the confidence and output; When it is difficult to quickly determine which frame corresponds to which infusion bottle or liquid medicine using these parameters directly, all detection frames are sorted from small to large according to the horizontal coordinate of the upper left corner point, and the horizontal coordinate difference of the upper left corner points of adjacent detection frames is compared. If the difference is less than 20 pixels, it is considered that the two detection frames belong to the same infusion bottle and the liquid medicine therein; otherwise, it is considered that the two detection frames do not belong to the same infusion bottle and the liquid medicine therein; By comparing the differences in the horizontal coordinates of the upper left corner points of all adjacent detection frames in turn, all infusion bottles can be matched with their drug liquid detection frames and the remaining proportion of the liquid can be calculated, and finally the liquid level information arranged in order from left to right can be obtained, so as to clearly obtain the remaining amount of liquid in each infusion bottle containing drug liquid.

7. The method for monitoring liquid level of an infusion bottle based on cloud and edge computing according to claim 1, characterized in that: Use the improved YOLOv5 network to train the model; The improved YOLOv5s network includes an input terminal Input, a backbone network Backbone, and a prediction layer Prediction. The input terminal Input uses Mosaic data enhancement to enhance the diversity of training data; the backbone network Backbone includes a Conv module and a C3 module; the bottleneck Neck adopts an FPN+PAN structure.

8. The method for monitoring liquid level of an infusion bottle based on cloud and edge computing according to claim 1, characterized in that: In step S3, the remaining proportion of the medicine in each infusion bottle is calculated and displayed in real time, and a voice reminder is set at a time interval; in step S4, when it is detected that the liquid level ratio is lower than the set threshold, a voice reminder is set and the dripping is controlled to stop.

9. An infusion bottle liquid level monitoring system based on cloud and edge computing, used to implement the method according to any one of claims 1 to 7, characterized in that: The system comprises The liquid level monitoring unit includes an image acquisition device and an edge computing device for real-time acquisition of infusion bottle liquid level video stream data. The edge computing device uses a deep learning network to monitor in real time whether there is an infusion bottle or an infusion bottle filled with liquid medicine in the video stream acquired by the image acquisition device. If the target is not detected, the monitoring is continuously performed in a loop; if detected, the target is framed with a detection frame, and the height of the infusion bottle and the liquid medicine is calculated through the coordinate information of the detection frame, and the remaining proportion of the liquid medicine in each infusion bottle is further calculated. Finally, the remaining proportion data of the liquid medicine and the set alarm liquid level data are transmitted to the cloud platform of the Internet of Things; The liquid level control unit includes a control chip and a drip rate regulator. The control chip communicates with a cloud platform via wireless communication to obtain the remaining amount and threshold of the liquid medicine. When the control chip detects that the remaining amount of the liquid medicine is lower than the threshold, the control motor is controlled to work and then the drip rate regulator is controlled to stop dripping. The display unit is used to receive real-time data and images sent by the cloud platform and display them in real time.

10. The method for monitoring the liquid level of an infusion bottle based on cloud and edge computing according to claim 9, characterized in that: The display unit includes a small program module for mobile devices, a GUI module for local monitoring of real-time images, and a GUI module for cloud monitoring of real-time images.

Images