Medical infusion monitoring method and system based on improved YOLOv7 network
Through the improved YOLOv7 network, combined with the dynamic snake convolution module, deformable convolution module and partial convolution module, the medical infusion monitoring model is optimized, and the problems of low detection accuracy and high computational complexity in the existing technology are solved, and high precision and real-time medical infusion monitoring are achieved.
Patent Information
- Application Number
- CN202510198602.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-23
- Publication Date
- 2025-06-10
AI Technical Summary
The prior art has problems in medical infusion monitoring with low detection accuracy, high computational complexity and unsuitable for real-time monitoring of embedded devices, especially in complex medical environments.
Using the improved YOLOv7 network, by introducing dynamic snake-shaped convolution modules, deformable convolution modules and partial convolution modules, the medical infusion monitoring model is optimized, the detection ability of slender liquid level lines is improved, the calculation complexity is reduced, and real-time reasoning is realized on embedded devices.
It realizes high-precision and real-time medical infusion monitoring, which is suitable for complex medical environments, reduces monitoring costs, and improves the monitoring efficiency of medical infusion.
Smart Images
Figure CN120125537A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical fields of computer vision and medical monitoring, and particularly relates to a medical infusion monitoring method and system based on an improved YOLOv7 network. Background Art
[0002] Medical infusion is one of the most common operations in hospital treatment. Especially during inpatient treatment, the vast majority of patients need to receive medications through intravenous infusion. However, traditional infusion monitoring mainly relies on the naked-eye observation of medical staff or family members, which has problems such as heavy workload and low monitoring efficiency.
[0003] In recent years, with the rapid development of artificial intelligence and deep learning technologies, the application of computer vision technology in the medical field has gradually increased. Existing research has explored methods based on image processing to monitor the liquid level of infusion bottles, but these methods still have the following deficiencies:
[0004] (1) Deep learning-based object detection algorithms (such as SSD and Faster R-CNN), although having certain detection capabilities, are not suitable for the real-time monitoring requirements of embedded devices due to their relatively high model complexity.
[0005] (2) The currently widely used YOLO series algorithms still have bottlenecks in the detection of small objects and the recognition of slender structures (such as liquid level lines), especially in complex medical environments.
[0006] Therefore, in view of the above problems, there is an urgent need for an infusion bottle liquid level monitoring method with high precision, high real-time performance and suitable for embedded devices to improve medical monitoring efficiency and reduce the burden on medical staff and patients. Summary of the Invention
[0007] In view of the above deficiencies in the prior art, the present invention provides a medical infusion monitoring method and system based on an improved YOLOv7 network.
[0008] To achieve the above invention objective, the technical solution adopted by the present invention is as follows:
[0009] A medical infusion monitoring method based on an improved YOLOv7 network, comprising the following steps:
[0010] S1. Set an embedded camera to obtain infusion bottle image data;
[0011] S2. Perform standardization processing on the infusion bottle image data, including resolution adjustment, denoising, data augmentation, and manual annotation, to obtain standardized infusion bottle image data;
[0012] S3. Build a medical infusion monitoring model based on the improved YOLOv7 network, and obtain the boundary coordinates and liquid level coordinates of the infusion bottle according to the medical infusion monitoring model and the standard image data of the infusion bottle;
[0013] S4. Calculate the liquid proportion of the infusion bottle based on the boundary coordinates and liquid level coordinates of the infusion bottle, and monitor the medical infusion according to the liquid proportion of the infusion bottle and the set threshold method.
[0014] Further, in step S1, set an embedded camera, specifically: fix the embedded camera at a position 0.3 to 0.5 meters away from the infusion bottle on the support rod of the infusion stand, and adjust the shooting angle of the embedded camera to a lateral shooting angle with respect to the infusion bottle.
[0015] Further, in step S3, the medical infusion monitoring model includes a backbone network improved based on the deformable convolution module and the partial convolution module, a neck improved based on the dynamic snake convolution module and the distribution shift convolution module, and a detection head connected in sequence;
[0016] The backbone network improved based on the deformable convolution module and the partial convolution module is used to extract multi-scale features of the standard image data of the infusion bottle and output the multi-scale features of the standard image data of the infusion bottle;
[0017] The neck improved based on the dynamic snake convolution module and the distribution shift convolution module is used to fuse the multi-scale features of the standard image data of the infusion bottle to obtain a fused multi-scale feature map;
[0018] The detection head is used to obtain the boundary coordinates and liquid level coordinates of the infusion bottle according to the fused multi-scale feature map.
[0019] Further, the backbone network improved based on the deformable convolution module and the partial convolution module includes a first-scale feature extraction branch, a third partial convolution module, and a deformable convolution module connected in sequence;
[0020] The first-scale feature extraction branch includes a first CBS module, a second CBS module, a first partial convolution module, and a second partial convolution module connected in sequence; the first-scale feature extraction module is used to extract the first-scale features of the standard image data of the infusion bottle and output the first-scale features of the standard image data of the infusion bottle;
[0021] The third partial convolution module is used to extract the second-scale features of the standard image data of the infusion bottle and output the second-scale features of the standard image data of the infusion bottle;
[0022] The deformable convolution module is used to extract the third-scale features of the standard image data of the infusion bottle and output the third-scale features of the standard image data of the infusion bottle.
[0023] Further, the formulas for the data processing flows of the first partial convolution module, the second partial convolution module, and the third partial convolution module are as follows:
[0024] F out = W concat ·Concat(F′ 1 , F′ 2 ) + W residual ·F in
[0025] Where: F out is the output feature of the partial convolution module, W concat is the weight for feature fusion, Concat is the feature concatenation operation, F′ 1 is the first specific region feature extracted by the partial convolution module, F′ 1 = PConv(F 1 , M 1 ), PConv is the partial convolution operation, F 1 is the first input feature of the partial convolution module, M 1 is the first branch mask matrix, F′ 2 = PConv(F 2 , M 2 ), F 2 is the second input feature of the partial convolution module, M 2 is the second branch mask matrix, W residual is the weight for residual connection, F in is the input feature of the partial convolution module.
[0026] Further, the formula for the data processing flow of the deformable convolution module is as follows:
[0027]
[0028] Where: F out (p 0 ) is the output feature of the deformable convolution module, i is the sampling point number of the dynamic convolution kernel, k is the number of sampling points of the dynamic convolution kernel, W i is the weight of the i-th sampling point in the dynamic convolution kernel, is the sampling value of the input feature of the deformable convolution module at the sampling point , Δm i is the amplitude adjustment factor of the i-th sampling point in the dynamic convolution kernel.
[0029] Further, the neck improved based on the dynamic snake-shaped convolution module and the distribution shift convolution module includes a first feature fusion branch, a second feature fusion branch, a third feature fusion branch, a first upsampling module, a first dynamic snake-shaped convolution layer, a second upsampling module, and a second dynamic snake-shaped convolution layer;
[0030] The first feature fusion branch includes a third CBS module, a first splicing layer, and a first dynamic snake convolution module; the input end of the third CBS module is connected to the output end of the first-scale feature extraction branch, the output end of the third CBS module is connected to the input end of the first splicing layer, the input end of the first splicing layer is further connected to the second feature fusion branch through a first upsampling module, the output end of the first splicing layer is connected to the input end of the first dynamic snake convolution module, the output end of the first dynamic snake convolution module is connected to the second feature fusion branch through a first dynamic snake convolution layer, and the output end of the first dynamic snake convolution module is further connected to the input end of the detection head;
[0031] The second feature fusion branch includes a fourth CBS module, a second splicing layer, a second dynamic snake convolution module, a third splicing layer, and a third dynamic snake convolution module; the input end of the fourth CBS module is connected to the output end of the third partial convolution module, the output end of the fourth CBS module is connected to the input end of the second splicing layer, the input end of the second splicing layer is further connected to the third feature fusion branch through a second upsampling module, the output end of the second splicing layer is connected to the input end of the second dynamic snake convolution module, the output end of the second dynamic snake convolution module is connected to the input end of the third splicing layer, the output end of the second dynamic snake convolution module is further connected to the input end of the first splicing layer through a first upsampling module, the input end of the third splicing layer is further connected to the output end of the first dynamic snake convolution module through a first dynamic snake convolution layer, the output end of the third splicing layer is connected to the input end of the third dynamic snake convolution module, the output end of the third dynamic snake convolution module is connected to the third feature fusion branch through a second dynamic snake convolution layer, and the output end of the first dynamic snake convolution module is further connected to the input end of the detection head;
[0032] The third feature fusion branch includes a fifth CBS module, a sixth CBS module, a seventh CBS module, a first MP layer, a second MP layer, a third MP layer, a fourth splicing layer, a fifth splicing layer, an eighth CBS module, a sixth splicing layer, and a fourth dynamic snake convolution module; the input end of the fifth CBS module is connected to the output end of the deformable convolution module, and the output end of the fifth CBS module is simultaneously connected to the input ends of the sixth CBS module and the seventh CBS module. The output end of the sixth CBS module is connected to the input end of the fourth splicing layer, and the output end of the sixth CBS module is also connected to the input end of the fourth splicing layer through the first MP layer, the second MP layer, and the third MP layer. The output ends of the seventh CBS module and the fourth splicing layer are both connected to the input end of the eighth CBS module. The output end of the eighth CBS module is connected to the input end of the second splicing layer through the second upsampling module, and the output end of the eighth CBS module is also connected to the input end of the sixth splicing layer. The input end of the sixth splicing layer is also connected to the output end of the third dynamic snake convolution module through the second dynamic snake convolution layer. The output end of the sixth splicing layer is connected to the input end of the fourth dynamic snake convolution module, and the output end of the fourth dynamic snake convolution module is connected to the input end of the detection head.
[0033] Further, in step S4, the formula for calculating the liquid ratio of the infusion bottle based on the boundary coordinates and liquid level coordinates of the infusion bottle is:
[0034]
[0035] where: P is the liquid ratio of the infusion bottle, y t is the liquid level coordinate of the infusion bottle, p b is the lower boundary coordinate of the bounding box of the infusion bottle, p t is the upper boundary coordinate of the bounding box of the infusion bottle.
[0036] A medical infusion monitoring system based on an improved YOLOv7 network for the above method includes an image acquisition module, a data processing module, a deep learning monitoring module, and a data analysis module;
[0037] The image acquisition module is used to obtain infusion bottle image data according to an embedded camera;
[0038] The data processing module is used to perform standardization processing on the infusion bottle image data, including resolution adjustment, denoising, data augmentation, and manual annotation, to obtain standardized infusion bottle image data;
[0039] The deep learning detection module is used to construct a medical infusion monitoring model according to the improved YOLOv7 network, and obtain the boundary coordinates and liquid level coordinates of the infusion bottle according to the medical infusion monitoring model and the standardized infusion bottle image data;
[0040] The data analysis module is used to calculate the liquid proportion of the infusion bottle according to the boundary coordinates and liquid level coordinates of the infusion bottle, and monitor medical infusion according to the liquid proportion of the infusion bottle and the set threshold method.
[0041] The present invention has the following beneficial effects:
[0042] (1) The present invention constructs a medical infusion monitoring model based on the improved YOLOv7 network. During the construction of the medical infusion monitoring model, by introducing a dynamic snake-shaped convolution module, a deformable convolution module and a partial convolution module, the detection ability of the medical infusion monitoring model for slender liquid level lines is optimized, the robustness is improved, the calculation complexity is reduced, and real-time inference on embedded devices is realized.
[0043] (2) The present invention sets an embedded camera to obtain infusion bottle image data. Through this non-contact way of obtaining infusion bottle image data and combining the liquid level bounding box coordinates to calculate the liquid proportion, not only can the accuracy of medical infusion monitoring be improved, but also the cost of medical infusion monitoring can be reduced, and it is applicable to complex medical environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 is a schematic flow chart of a medical infusion monitoring method based on an improved YOLOv7 network;
[0045] Figure 2 is a schematic structural diagram of a medical infusion monitoring model;
[0046] Figure 3 is a schematic structural diagram of a partial convolution module;
[0047] Figure 4 is a structural diagram of a partial convolution module;
[0048] Figure 5 is a structural diagram of a partial convolution module;
[0049] Figure 6 is a schematic structural diagram of a medical infusion monitoring system based on an improved YOLOv7 network. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0050] The following describes the specific embodiments of the present invention to facilitate those skilled in the art of the present technology to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art of the present technology, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions and creations using the concept of the present invention are within the scope of protection.
[0051] Such as Figure 1As shown, a medical infusion monitoring method based on an improved YOLOv7 network includes steps S1 - S4, which are as follows:
[0052] S1. Set up an embedded camera to obtain infusion bottle image data.
[0053] In an optional embodiment of the present invention, an image acquisition module is constructed, and in the image acquisition module, an embedded camera is used to obtain infusion bottle image data. The embedded camera is set up as follows: The embedded camera is fixed at a position 0.3 to 0.5 meters away from the infusion bottle on the support rod of the infusion stand, and the shooting angle of the embedded camera is adjusted to a lateral shooting angle with respect to the infusion bottle, so that the obtained infusion bottle image data can cover the entire area of the infusion bottle and improve the clarity of the liquid level line.
[0054] S2. Perform standardization processing on the infusion bottle image data, including resolution adjustment, denoising, data augmentation, and manual annotation, to obtain standard infusion bottle image data.
[0055] In an optional embodiment of the present invention, a data processing module is constructed, and in the data processing module, the infusion bottle image data is standardized, including resolution adjustment, denoising, data augmentation, and manual annotation, to obtain standard infusion bottle image data.
[0056] S3. Construct a medical infusion monitoring model based on the improved YOLOv7 network, and obtain the boundary coordinates and liquid level coordinates of the infusion bottle according to the medical infusion monitoring model and the standard infusion bottle image data.
[0057] In an optional embodiment of the present invention, a deep learning monitoring module is constructed, and in the deep learning monitoring module, a medical infusion monitoring model is constructed based on the improved YOLOv7 network, and the boundary coordinates and liquid level coordinates of the infusion bottle are obtained according to the medical infusion monitoring model and the standard infusion bottle image data. The medical infusion monitoring model includes a backbone network improved based on a deformable convolution module and a partial convolution module, a neck improved based on a dynamic snake convolution module and a distribution shift convolution module, and a detection head connected in sequence. As Figure 2 shown, backbone is the backbone network improved based on the deformable convolution module and the partial convolution module, neck is the neck improved based on the dynamic snake convolution module and the distribution shift convolution module, and head is the detection head.
[0058] The backbone network improved based on the deformable convolution module and the partial convolution module is used to extract multi - scale features of the standard infusion bottle image data and output the multi - scale features of the standard infusion bottle image data.
[0059] The backbone network improved based on the deformable convolution module and the partial convolution module includes a first-scale feature extraction branch, a third partial convolution module, and a deformable convolution module connected in sequence.
[0060] The first-scale feature extraction branch includes a first CBS module, a second CBS module, a first partial convolution module, and a second partial convolution module connected in sequence; the first-scale feature extraction module is used to extract the first-scale features of the standard infusion bottle image data and output the first-scale features of the standard infusion bottle image data. In the present invention, the CBS module includes a convolution layer, a normalization layer, and a Silu function connected in sequence.
[0061] The third partial convolution module is used to extract the second-scale features of the standard infusion bottle image data and output the second-scale features of the standard infusion bottle image data.
[0062] The deformable convolution module is used to extract the third-scale features of the standard infusion bottle image data and output the third-scale features of the standard infusion bottle image data.
[0063] As Figure 3 shown, the present invention provides a schematic structural diagram of the partial convolution module. The formulas for the data processing flow of the first partial convolution module, the second partial convolution module, and the third partial convolution module are:
[0064] F out =W concat ·Concat(F′ 1 ,F′ 2 )+W residual ·F in
[0065] where: F out is the output feature of the partial convolution module, W concat is the weight of feature fusion, Concat is the feature concatenation operation, F′ 1 is the first specific region feature extracted by the partial convolution module, F′ 1 =PConv(F 1 ,M 1 ), PConv is the partial convolution operation, F 1 is the first input feature of the partial convolution module, M 1 is the first branch mask matrix, F′ 2 =PConv(F 2 ,M 2 ), F 2 is the second input feature of the partial convolution module, M 2 is the second branch mask matrix, W residual is the weight of the residual connection, F inIs the input feature of the partial convolution module.
[0066] The present invention introduces a partial convolution module. By combining partial convolution (PConv) and a multi-branch feature aggregation strategy (ELAN), the network focuses on the area where the infusion bottle and the liquid level line are located, ignores the irrelevant background, and significantly reduces the computational complexity. Its multi-branch structure (ELAN) divides the input features into multiple branches, extracts the features of different regions respectively, and completes feature fusion through feature splicing operations and weighted residual connections. This design not only improves the detection efficiency of slender targets (liquid level lines), but also enhances the real-time performance and robustness of the medical infusion monitoring model, and is particularly suitable for the scenario of embedded devices.
[0067] Such as Figure 4 As shown, the present invention provides a schematic structural diagram of a deformable convolution module. The formula for the data processing flow of the deformable convolution module is:
[0068]
[0069] Where: F out (p 0 ) is the output feature of the deformable convolution module, i is the sampling point number of the dynamic convolution kernel, k is the number of sampling points of the dynamic convolution kernel, W i Is the weight of the i-th sampling point in the dynamic convolution kernel, Is the sampling value of the input feature of the deformable convolution module at the sampling point , Δm i Is the amplitude adjustment factor of the i-th sampling point in the dynamic convolution kernel.
[0070] The present invention introduces a deformable convolution module, combines deformable convolution with standard convolution, and enables the convolution kernel to adapt to the geometry of the target by learning dynamic offsets, so as to adapt to the different shapes of infusion bottles and the deformation characteristics of liquid level lines, enhance the feature extraction ability for deformed targets, and improve the adaptability to complex targets.
[0071] The neck improved based on the dynamic snake-shaped convolution module and the distributed shift convolution module is used to fuse the multi-scale features of the standard image data of the infusion bottle to obtain a fused multi-scale feature map.
[0072] The neck improved based on the dynamic snake-shaped convolution module and the distributed shift convolution module includes a first feature fusion branch, a second feature fusion branch, a third feature fusion branch, a first upsampling module, a first dynamic snake-shaped convolution layer, a second upsampling module, and a second dynamic snake-shaped convolution layer. The first upsampling module and the second upsampling module in the present invention include a CBS module and an upsampling operation layer connected in sequence, Figure 2 Where Upsample in
[0073] The first feature fusion branch includes a third CBS module, a first concatenation layer, and a first dynamic serpentine convolution module; the input end of the third CBS module is connected to the output end of the first-scale feature extraction branch, the output end of the third CBS module is connected to the input end of the first concatenation layer, the input end of the first concatenation layer is further connected to the second feature fusion branch through a first upsampling module, the output end of the first concatenation layer is connected to the input end of the first dynamic serpentine convolution module, the output end of the first dynamic serpentine convolution module is connected to the second feature fusion branch through a first dynamic serpentine convolution layer, and the output end of the first dynamic serpentine convolution module is further connected to the input end of the detection head.
[0074] The second feature fusion branch includes a fourth CBS module, a second concatenation layer, a second dynamic serpentine convolution module, a third concatenation layer, and a third dynamic serpentine convolution module; the input end of the fourth CBS module is connected to the output end of the third partial convolution module, the output end of the fourth CBS module is connected to the input end of the second concatenation layer, the input end of the second concatenation layer is further connected to the third feature fusion branch through a second upsampling module, the output end of the second concatenation layer is connected to the input end of the second dynamic serpentine convolution module, the output end of the second dynamic serpentine convolution module is connected to the input end of the third concatenation layer, the output end of the second dynamic serpentine convolution module is further connected to the input end of the first concatenation layer through a first upsampling module, the input end of the third concatenation layer is further connected to the output end of the first dynamic serpentine convolution module through a first dynamic serpentine convolution layer, the output end of the third concatenation layer is connected to the input end of the third dynamic serpentine convolution module, the output end of the third dynamic serpentine convolution module is connected to the third feature fusion branch through a second dynamic serpentine convolution layer, and the output end of the first dynamic serpentine convolution module is further connected to the input end of the detection head.
[0075] The third feature fusion branch includes a fifth CBS module, a sixth CBS module, a seventh CBS module, a first MP layer, a second MP layer, a third MP layer, a fourth splicing layer, a fifth splicing layer, an eighth CBS module, a sixth splicing layer, and a fourth dynamic snake convolution module; the input end of the fifth CBS module is connected to the output end of the deformable convolution module, the output end of the fifth CBS module is simultaneously connected to the input ends of the sixth CBS module and the seventh CBS module, the output end of the sixth CBS module is connected to the input end of the fourth splicing layer, and the output end of the sixth CBS module is also connected to the input end of the fourth splicing layer through the first MP layer, the second MP layer, and the third MP layer; the output ends of the seventh CBS module and the fourth splicing layer are both connected to the input end of the eighth CBS module, the output end of the eighth CBS module is connected to the input end of the second splicing layer through the second upsampling module, the output end of the eighth CBS module is also connected to the input end of the sixth splicing layer, the input end of the sixth splicing layer is also connected to the output end of the third dynamic snake convolution module through the second dynamic snake convolution layer, the output end of the sixth splicing layer is connected to the input end of the fourth dynamic snake convolution module, and the output end of the fourth dynamic snake convolution module is connected to the input end of the detection head.
[0076] As Figure 5 shown, the present invention provides a structural schematic diagram of the dynamic snake convolution module. The present invention introduces the dynamic snake convolution module, realizes dynamic sampling of slender structures (liquid level lines) through geometric constraints, enhances the ability to detect liquid level edges, then realizes efficient feature learning through multi-branch feature extraction and feature aggregation, and finally improves the overall detection accuracy through feature fusion.
[0077] The detection head is used to obtain the boundary coordinates and liquid level coordinates of the infusion bottle based on the fused multi-scale feature map.
[0078] The detection head includes a ninth CBS module, a tenth CBS module, and an eleventh CBS module. The ninth CBS module, the tenth CBS module, and the eleventh CBS module are respectively used to receive the fused multi-scale feature maps output by the first feature fusion branch, the second feature fusion branch, and the third feature fusion branch, and process the fused multi-scale feature maps to obtain the boundary coordinates and liquid level coordinates of the infusion bottle.
[0079] S4. Calculate the liquid ratio of the infusion bottle based on the boundary coordinates and liquid level coordinates of the infusion bottle, and monitor the medical infusion according to the liquid ratio of the infusion bottle and the set threshold method.
[0080] In an optional embodiment of the present invention, the present invention constructs a data analysis module, and in the data analysis module, calculates the liquid ratio of the infusion bottle based on the boundary coordinates and liquid level coordinates of the infusion bottle, and monitors the medical infusion according to the liquid ratio of the infusion bottle and the set threshold method. The formula for calculating the liquid ratio of the infusion bottle based on the boundary coordinates and liquid level coordinates of the infusion bottle in the present invention is:
[0081]
[0082] Where: P is the liquid proportion of the infusion bottle, and y t is the liquid level coordinate of the infusion bottle, and p b is the lower boundary coordinate of the bounding box of the infusion bottle, and p t is the upper boundary coordinate of the bounding box of the infusion bottle.
[0083] In the present invention, the set threshold is determined to be 10%. If the liquid proportion of the infusion bottle is less than 10%, an alarm signal is triggered to prompt the medical staff to replace the infusion bottle in time for monitoring medical infusion.
[0084] Such as Figure 6 shown, a medical infusion monitoring system based on the improved YOLOv7 network applied to the above method includes an image acquisition module, a data processing module, a deep learning monitoring module, and a data analysis module.
[0085] In an alternative embodiment of the present invention, the image acquisition module is used to set an embedded camera to obtain infusion bottle image data.
[0086] In an alternative embodiment of the present invention, the data processing module is used to perform normalization processing on the infusion bottle image data, including resolution adjustment, denoising, data augmentation, and manual annotation, to obtain standard infusion bottle image data.
[0087] In an alternative embodiment of the present invention, the deep learning detection module is used to construct a medical infusion monitoring model based on the improved YOLOv7 network, and obtain the bounding coordinates and liquid level coordinates of the infusion bottle according to the medical infusion monitoring model and the standard infusion bottle image data.
[0088] In an alternative embodiment of the present invention, the data analysis module is used to calculate the liquid proportion of the infusion bottle according to the bounding coordinates and liquid level coordinates of the infusion bottle, and monitor medical infusion according to the liquid proportion of the infusion bottle and the set threshold method.
[0089] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, and the combination of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementing in the process Figure 1 one process or multiple processes and / or blocks Figure 1a device for the functions specified in one or more boxes.
[0090] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the functions specified in one Figure 1 process or more processes and / or boxes Figure 1 a box or more boxes.
[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are performed on the computer or other programmable device to produce a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 process or more processes and / or boxes Figure 1 a box or more boxes.
[0092] In the present invention, specific embodiments are used to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.
[0093] Those of ordinary skill in the art will realize that the embodiments described herein are for helping the reader understand the principles of the present invention, and it should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those of ordinary skill in the art can make various other specific deformations and combinations that do not depart from the essence of the present invention according to the technical revelations disclosed in the present invention, and these deformations and combinations are still within the protection scope of the present invention.
Claims
1. A medical infusion monitoring method based on an improved YOLOv7 network, characterized in that: The following steps are involved: S1. Setting an embedded camera to obtain image data of the infusion bottle; S2, standardizing the infusion bottle image data, including resolution adjustment, denoising, data enhancement, and manual annotation, to obtain standard image data of the infusion bottle; S3. Build a medical infusion monitoring model based on the improved YOLOv7 network, and obtain the boundary coordinates and liquid level coordinates of the infusion bottle according to the medical infusion monitoring model and the standard image data of the infusion bottle; S4. Calculate the liquid proportion of the infusion bottle based on the boundary coordinates and liquid level coordinates of the infusion bottle, and monitor the medical infusion according to the liquid proportion of the infusion bottle and the set threshold method.
2. The medical infusion monitoring method based on the improved YOLOv7 network according to claim 1, characterized in that: In step S1, an embedded camera is set, specifically: the embedded camera is fixed to the infusion stand support pole at a position 0.3 to 0.5 meters away from the infusion bottle, and the shooting angle of the embedded camera is adjusted to a sideways shooting angle with the infusion bottle.
3. The medical infusion monitoring method based on the improved YOLOv7 network according to claim 1, characterized in that: In step S3, the medical infusion monitoring model includes a backbone network based on an improved deformable convolution module and a partial convolution module, a neck based on an improved dynamic snake convolution module and a distributed shift convolution module, and a detection head, which are sequentially connected; The backbone network improved based on the deformable convolution module and the partial convolution module is used to extract the multi-scale features of the standard image data of the infusion bottle and output the multi-scale features of the standard image data of the infusion bottle; The neck improved based on the dynamic snake convolution module and the distributed shift convolution module is used to fuse the multi-scale features of the standard image data of the infusion bottle to obtain the fused multi-scale feature map; The detection head is used to obtain the boundary coordinates and liquid level coordinates of the infusion bottle based on the fused multi-scale feature map.
4. The medical infusion monitoring method based on the improved YOLOv7 network according to claim 3 is characterized in that: The backbone network improved based on the deformable convolution module and the partial convolution module includes a first scale feature extraction branch, a third partial convolution module and a deformable convolution module connected in sequence; The first scale feature extraction branch includes a first CBS module, a second CBS module, a first partial convolution module, and a second partial convolution module connected in sequence; the first scale feature extraction module is used to extract the first scale feature of the infusion bottle standard image data and output the first scale feature of the infusion bottle standard image data; The third part of the convolution module is used to extract the second scale feature of the standard image data of the infusion bottle and output the second scale feature of the standard image data of the infusion bottle; The deformable convolution module is used to extract the third scale features of the standard image data of the infusion bottle and output the third scale features of the standard image data of the infusion bottle.
5. The medical infusion monitoring method based on the improved YOLOv7 network according to claim 4, characterized in that: The formula for the data processing flow of the first part convolution module, the second part convolution module, and the third part convolution module is: F out =W concat ·Concat(F′1,F′2)+W residual ·F in Among them: F out is the output feature of the partial convolution module, W concat is the weight of feature fusion, Concat is the feature concatenation operation, F′1 is the first specific area feature extracted by the partial convolution module, F′1=PConv(F1,M1), PConv is the partial convolution operation, F1 is the first input feature of the partial convolution module, M1 is the first branch mask matrix, F′2=PConv(F2,M2), F2 is the second input feature of the partial convolution module, M2 is the second branch mask matrix, W residual is the weight of the residual connection, F in It is the input feature of some convolution modules.
6. The medical infusion monitoring method based on the improved YOLOv7 network according to claim 4, characterized in that: The formula for the data processing flow of the deformable convolution module is: Among them: F out (p0) is the output feature of the deformable convolution module, i is the sampling point number of the dynamic convolution kernel, k is the number of sampling points of the dynamic convolution kernel, W i is the weight of the i-th sampling point in the dynamic convolution kernel, The input features of the deformable convolution module at the sampling point The sampling value, Δm i is the amplitude adjustment factor of the i-th sampling point in the dynamic convolution kernel.
7. The medical infusion monitoring method based on the improved YOLOv7 network according to claim 3, characterized in that: The neck improved based on the dynamic snake convolution module and the distribution shift convolution module includes a first feature fusion branch, a second feature fusion branch, a third feature fusion branch, a first upsampling module, a first dynamic snake convolution layer, a second upsampling module, and a second dynamic snake convolution layer; The first feature fusion branch includes a third CBS module, a first splicing layer, and a first dynamic snake convolution module; the input end of the third CBS module is connected to the output end of the first scale feature extraction branch, the output end of the third CBS module is connected to the input end of the first splicing layer, the input end of the first splicing layer is also connected to the second feature fusion branch via the first upsampling module, the output end of the first splicing layer is connected to the input end of the first dynamic snake convolution module, the output end of the first dynamic snake convolution module is connected to the second feature fusion branch via the first dynamic snake convolution layer, and the output end of the first dynamic snake convolution module is also connected to the input end of the detection head; The second feature fusion branch includes a fourth CBS module, a second splicing layer, a second dynamic serpentine convolution module, a third splicing layer, and a third dynamic serpentine convolution module; the input end of the fourth CBS module is connected to the output end of the third partial convolution module, the output end of the fourth CBS module is connected to the input end of the second splicing layer, the input end of the second splicing layer is also connected to the third feature fusion branch via the second upsampling module, the output end of the second splicing layer is connected to the input end of the second dynamic serpentine convolution module, the output end of the second dynamic serpentine convolution module is connected to the input end of the third splicing layer, the output end of the second dynamic serpentine convolution module is also connected to the input end of the first splicing layer via the first upsampling module, the input end of the third splicing layer is also connected to the output end of the first dynamic serpentine convolution module via the first dynamic serpentine convolution layer, the output end of the third splicing layer is connected to the input end of the third dynamic serpentine convolution module, the output end of the third dynamic serpentine convolution module is connected to the third feature fusion branch via the second dynamic serpentine convolution layer, and the output end of the first dynamic serpentine convolution module is also connected to the input end of the detection head; The third feature fusion branch includes the fifth CBS module, the sixth CBS module, the seventh CBS module, the first MP layer, the second MP layer, the third MP layer, the fourth splicing layer, the fifth splicing layer, the eighth CBS module, the sixth splicing layer, and the fourth dynamic snake convolution module; the input end of the fifth CBS module is connected to the output end of the deformable convolution module, the output end of the fifth CBS module is simultaneously connected to the input end of the sixth CBS module and the input end of the seventh CBS module, the output end of the sixth CBS module is connected to the input end of the fourth splicing layer, and the output end of the sixth CBS module is also connected to the first MP layer, the second MP layer, the The third MP layer is connected to the input end of the fourth splicing layer, the output end of the seventh CBS module and the output end of the fourth splicing layer are both connected to the input end of the eighth CBS module, the output end of the eighth CBS module is connected to the input end of the second splicing layer via the second upsampling module, the output end of the eighth CBS module is also connected to the input end of the sixth splicing layer, the input end of the sixth splicing layer is also connected to the output end of the third dynamic serpentine convolution module via the second dynamic serpentine convolution layer, the output end of the sixth splicing layer is connected to the input end of the fourth dynamic serpentine convolution module, and the output end of the fourth dynamic serpentine convolution module is connected to the input end of the detection head.
8. The medical infusion monitoring method based on the improved YOLOv7 network according to claim 1, characterized in that: In step S4, the formula for calculating the liquid proportion of the infusion bottle based on the boundary coordinates and liquid level coordinates of the infusion bottle is: Where: P is the liquid proportion of the infusion bottle, y t is the liquid level coordinate of the infusion bottle, p b is the lower boundary coordinate of the infusion bottle’s bounding box, p t The upper boundary coordinates of the infusion bottle's bounding box.
9. A medical infusion monitoring system based on an improved YOLOv7 network applied to any of the methods described in claims 1-8, characterized in that: It includes image acquisition module, data processing module, deep learning monitoring module and data analysis module; An image acquisition module, used for acquiring image data of the infusion bottle according to an embedded camera; A data processing module is used to perform standard processing on the image data of the infusion bottle, including resolution adjustment, denoising, data enhancement and manual annotation, so as to obtain standard image data of the infusion bottle; A deep learning detection module is used to build a medical infusion monitoring model based on the improved YOLOv7 network, and obtain the boundary coordinates and liquid level coordinates of the infusion bottle based on the medical infusion monitoring model and the standard image data of the infusion bottle; The data analysis module is used to calculate the liquid proportion of the infusion bottle according to the boundary coordinates and liquid level coordinates of the infusion bottle, and monitor the medical infusion according to the liquid proportion of the infusion bottle and the set threshold method.