Infusion monitoring method and device based on visual model, equipment and medium

Abstract

The invention provides an infusion monitoring method and device based on a visual model, equipment and a medium, and relates to the technical field of artificial intelligence, in particular to the technical field of computer vision and image processing. According to the implementation scheme, a first image and a second image of a target infusion container at different moments are collected; inputting the first image and the second image into a visual model to obtain first capacity scale information and second capacity scale information output by the visual model; determining a predicted liquid outflow volume of the target infusion container based on the first image, the second image, the first volume scale information and the second volume scale information; determining a reference liquid outflow volume of the target infusion container based on the liquid dripping speed of the target infusion container; and in response to determining that the deviation between the predicted liquid outflow amount and the reference liquid outflow amount does not exceed the first deviation threshold value, determining the first liquid remaining amount of the target infusion container at the second moment based on the second volume scale information and the second image.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the technical field of computer vision and image processing, and more particularly to a visual model-based infusion monitoring method and device, electronic equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] Artificial intelligence is a discipline that studies enabling computers to simulate some thinking processes and intelligent behaviors of human beings (such as learning, reasoning, thinking, planning, etc.), which includes both hardware and software technologies. Artificial intelligence hardware technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing, etc.; artificial intelligence software technology mainly includes computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, knowledge graph technology, etc.

[0003] With the development of computer technology, computer vision technology based on artificial intelligence is widely used in various fields. In traditional medical scenarios, the monitoring and control of the infusion process usually rely on manual operation, which not only occupies human resources but also cannot effectively guarantee the safety of the infusion process.

[0004] The methods described in this section can not necessarily be the methods previously conceived or adopted. Unless otherwise indicated, nothing in this section should be assumed to be prior art merely because it is included in this section. Similarly, unless otherwise indicated, issues raised in this section should not be assumed to have been recognized in any prior art. SUMMARY

[0005] The present disclosure provides a visual model-based infusion monitoring method, device, electronic equipment, computer readable storage medium and computer program product.

[0006] According to an aspect of the present disclosure, a visual model-based infusion monitoring method is provided, including: collecting a first image of a target infusion container at a first time and a second image of the target infusion container at a second time; obtaining first volume scale information of the target infusion container and second volume scale information of the target infusion container output by a visual model by inputting the first image and the second image into the visual model, wherein the visual model is trained using sample images of sample liquid containers and reference volume scale information of the sample liquid containers; determining a predicted liquid outflow between the first time and the second time of the target infusion container based on the first image, the second image, the first volume scale information, and the second volume scale information; determining a liquid drop rate between the first time and the second time of the target infusion container; determining a reference liquid outflow between the first time and the second time of the target infusion container based on the liquid drop rate; and in response to determining that a deviation between the predicted liquid outflow and the reference liquid outflow does not exceed a first deviation threshold, determining a first liquid remaining amount of the target infusion container at the second time based on the second volume scale information and the second image.

[0007] According to another aspect of the present disclosure, a visual model-based infusion monitoring device is provided, including: a collection unit configured to collect a first image of a target infusion container at a first time and a second image of the target infusion container at a second time; an obtaining unit configured to obtain first volume scale information of the target infusion container and second volume scale information of the target infusion container output by a visual model by inputting the first image and the second image into the visual model, wherein the visual model is trained using sample images of sample liquid containers and reference volume scale information of the sample liquid containers; a first determination unit configured to determine a predicted liquid outflow between the first time and the second time of the target infusion container based on the first image, the second image, the first volume scale information, and the second volume scale information; a second determination unit configured to determine a liquid drop rate between the first time and the second time of the target infusion container; a third determination unit configured to determine a reference liquid outflow between the first time and the second time of the target infusion container based on the liquid drop rate; and a fourth determination unit configured to determine a first liquid remaining amount of the target infusion container at the second time based on the second volume scale information and the second image in response to determining that a deviation between the predicted liquid outflow and the reference liquid outflow does not exceed a first deviation threshold.

[0008] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the infusion monitoring method described above.

[0009] According to another aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to perform the infusion monitoring method described above.

[0010] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, can implement the infusion monitoring method described above.

[0011] According to one or more embodiments of the present disclosure, the monitoring accuracy of the liquid residual amount of an infusion container can be improved.

[0012] It should be understood that the contents described in this section are not intended to identify key or important features of the embodiments of the present disclosure, nor are they used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0013] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments and together with the description serve to explain exemplary implementations of the application. The illustrated embodiments are exemplary only and not limiting of the scope of the claims. In all the drawings, like reference numerals refer to like parts throughout the several views.

[0014] Figure 1 A schematic diagram of an exemplary system in which the various methods described herein can be implemented according to exemplary embodiments of the present disclosure is shown;

[0015] Figure 2 A flowchart of a visual model-based infusion monitoring method according to exemplary embodiments of the present disclosure is shown;

[0016] Figure 3 A structural schematic diagram of an infusion monitoring system according to exemplary embodiments of the present disclosure is shown;

[0017] Figure 4 A structural block diagram of a visual model-based infusion monitoring apparatus according to exemplary embodiments of the present disclosure is shown;

[0018] Figure 5 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0019] Exemplary embodiments of the present disclosure are described herein with reference to the accompanying drawings, in which various details of embodiments of the present disclosure are set forth to assist in the understanding of the present disclosure. It should be apparent to those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present disclosure. Also, descriptions of well-known functions and constructions can be omitted for clarity and conciseness.

[0020] In the present disclosure, the terms "first", "second", and the like are used to describe various elements only for the purpose of distinguishing one element from another, and the terms are not intended to limit the positions, sequence, or importance of the elements. In some examples, a first element and a second element can refer to the same instance of the element, and in some cases, they can refer to different instances of the element based on the context of the description.

[0021] The terms used in the description of various described examples in the present disclosure are only for the purpose of describing particular examples and are not intended to be limiting. Unless specifically defined otherwise, an element that is a singular can be plural and vice versa. Also, the term "and / or" used in the present disclosure encompasses any and all possible combinations of the listed items.

[0022] Embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0023] Figure 1 A schematic diagram of an example system 100 in which various methods and apparatus described herein can be implemented according to embodiments of the present disclosure is shown. Referring to Figure 1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 can be configured to execute one or more application programs.

[0024] In embodiments of the present disclosure, the server 120 can run one or more services or software applications that enable the execution of an infusion monitoring method.

[0025] In certain embodiments, the server 120 can also provide other services or software applications, which can include non-virtual and virtual environments. In certain embodiments, these services can be provided as web-based services or cloud services, for example, to users of the client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0026] In Figure 1 In the illustrated configuration, the server 120 can include one or more components that implement the functionality performed by the server 120. These components can include software components that are executable by one or more processors, hardware components, or combinations thereof. Users operating the client devices 101, 102, 103, 104, 105, and / or 106 can in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by the components. It should be understood that a wide variety of system configurations are possible, which can differ from system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0027] A user can use the client device 101, 102, 103, 104, 105, and / or 106 to transmit an image that includes an infusion container. The client device can provide an interface that enables the user of the client device to interact with the client device. The client device can also output information to the user via the interface. Although Figure 1 Only six client devices are depicted, but one of skill in the art will understand that the present disclosure can support any number of client devices.

[0028] Client devices 101, 102, 103, 104, 105, and / or 106 can include various categories of computer devices, such as portable handheld devices, general purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, smart screen devices, self-service kiosk devices, service robots, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, and the like. These computer devices can run various categories and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices can include cellular phones, smartphones, tablet computers, personal digital assistants (PDAs), and the like. Wearable devices can include head-mounted displays (such as smart glasses) and other devices. Gaming systems can include various handheld gaming devices, Internet-enabled gaming devices, and the like. Client devices are capable of executing a variety of different applications, such as various Internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and can use various communication protocols.

[0029] Network 110 can be any category of network in the art, including, but not limited to, a LAN, an Ethernet-based network, Token Ring, a WAN, the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0030] Server 120 can include one or more general purpose computers, special purpose server computers (e.g., PC (personal computer) servers, UNIX servers, midrange servers), blade servers, mainframe computers, server clusters, or any other appropriate arrangement and / or combination. Server 120 can include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization (such as one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for servers). In various embodiments, server 120 can run one or more services or software applications that provide the functionality described below.

[0031] The computing units in the server 120 can run one or more operating systems including any of the operating systems described above, as well as any commercially available server operating systems. Server 120 can also run any of a variety of additional server applications and / or mid-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0032] In some embodiments, the server 120 can include one or more applications to analyze and consolidate data feeds and / or event updates from users of the client devices 101, 102, 103, 104, 105, and 106. The server 120 can also include one or more applications to display the data feeds and / or real-time events via one or more display devices of the client devices 101, 102, 103, 104, 105, and 106.

[0033] In some embodiments, the server 120 can be a server of a distributed system, or a server that is combined with a blockchain. The server 120 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The cloud server is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and virtual private server (VPS, Virtual Private Server) services.

[0034] The system 100 can also include one or more databases 130. In certain embodiments, these databases can be used to store data and other information. For example, one or more of the databases 130 can be used to store information such as audio files and video files. The databases 130 can reside in a variety of locations. For example, databases used by the server 120 can reside locally to the server 120, or can be remote from the server 120 and can communicate with the server 120 via a network- or application-specific connection. The databases 130 can be of different categories. In certain embodiments, databases used by the server 120 can be, for example, relational databases. One or more of these databases can store, update, and retrieve data to and from the databases in response to commands.

[0035] In certain embodiments, one or more of the databases 130 can also be used by applications to store application data. Databases used by applications can be databases of different categories, such as key-value stores, object stores, or regular stores backed by file systems.

[0036] Figure 1The system 100 can be configured and operated in various ways to enable the application of various methods and apparatuses described in accordance with the present disclosure.

[0037] In traditional medical scenarios, the monitoring and control of infusion processes are usually dependent on manual operation, which not only occupies human resources but also cannot effectively guarantee the safety of the infusion process.

[0038] Based on this, the present disclosure provides a visual model-based infusion monitoring method, which generates capacity scale information by using a visual model after collecting an image of a target infusion container, calculates a reference liquid outflow amount in a time period of image collection of the infusion container according to a liquid drop rate, uses the deviation between the reference liquid outflow amount and a predicted liquid outflow amount determined based on the capacity scale information to indicate whether the capacity scale information is accurate, and then determines the liquid remaining amount based on the current capacity scale information in the case of small deviation, so as to ensure the accuracy of the liquid remaining amount result and realize more accurate infusion monitoring.

[0039] Figure 2 A flowchart of a visual model-based infusion monitoring method 200 according to an exemplary embodiment of the present disclosure is shown. As shown in Figure 2 The method 200 includes the following steps:

[0040] Step S201, collecting a first image of a target infusion container at a first time and a second image of the target infusion container at a second time;

[0041] Step S202, obtaining first capacity scale information of the target infusion container and second capacity scale information of the target infusion container output by a visual model by inputting the first image and the second image into the visual model, wherein the visual model is trained by using sample images of sample liquid containers and reference capacity scale information of the sample liquid containers;

[0042] Step S203, determining a predicted liquid outflow amount of the target infusion container between the first time and the second time based on the first image, the second image, the first capacity scale information, and the second capacity scale information;

[0043] Step S204, determining a liquid drop rate of the target infusion container between the first time and the second time;

[0044] Step S205, determining a reference liquid outflow amount of the target infusion container between the first time and the second time based on the liquid drop rate; and

[0045] In step S206, in response to determining that the deviation between the predicted liquid outflow amount and the reference liquid outflow amount does not exceed the first deviation threshold, a first liquid residual amount of the target infusion container at the second time is determined based on the second volume scale information and the second image.

[0046] By applying the method 200 described above, the volume scale information can be generated by using the visual model after the image of the target infusion container is collected, the reference liquid outflow amount in the time period of the image collection of the infusion container can be calculated according to the liquid drop rate, whether the volume scale information is accurate can be indicated by using the deviation between the reference liquid outflow amount and the predicted liquid outflow amount determined based on the volume scale information, and then the liquid residual amount can be determined based on the current volume scale information in the case of smaller deviation, so as to ensure the accuracy of the liquid residual amount result and realize more accurate infusion monitoring.

[0047] In some examples, the target infusion container can be various types of liquid containers, such as an infusion bottle or an infusion bag. In some examples, the target infusion container can be marked with a nominal capacity, in which case the visual model can read the marked information of the target infusion container by analyzing the first image or the second image, and generate more accurate volume scale information based thereon.

[0048] In some examples, the volume scale information output by the visual model can be a plurality of volume scale line marks added on the first image and the second image, such as a plurality of scale line marks of 100 milliliters, 200 milliliters, etc. added on the target infusion container. In some examples, the volume scale information output by the visual model can also be coordinate information of a plurality of volume scale lines, which indicates the scale positions of the target infusion container in the image based on the coordinate information. As long as the volume scale information can indicate the volume information corresponding to different positions of the target infusion container, the specific data form of the volume scale information is not limited in the present disclosure.

[0049] In some examples, the visual model can be a Transformer model pre-trained by using sample images of sample liquid containers and reference volume scale information of the sample liquid containers. In some examples, it can also be a neural network (such as a ResNet network) that extracts image feature information, and then generates volume scale information based on the image feature information by using a Transformer network. The training process of the visual model can be implemented based on a supervised training method, which will not be described herein.

[0050] According to some embodiments, the method 200 further comprises: in response to determining that the deviation between the predicted liquid outflow and the reference liquid outflow exceeds the first deviation threshold, correcting the second volume scale information based on the deviation; and determining the first liquid residual amount of the target infusion container at the second time based on the corrected second volume scale information and the second image. In this way, when the deviation between the reference liquid outflow and the predicted liquid outflow is large, the volume scale information can be corrected based on the deviation to improve the accuracy of the volume scale information, and a more accurate liquid residual amount result can be obtained based thereon.

[0051] According to some embodiments, the second volume scale information comprises a plurality of volume scale lines distributed along a vertical direction on the target infusion container, and the response to determining that the deviation between the predicted liquid outflow and the reference liquid outflow exceeds the first deviation threshold, correcting the second volume scale information based on the deviation comprises: determining the relative size relationship between the predicted liquid outflow and the reference liquid outflow; determining a correction direction of the volume scale lines based on the relative size relationship, wherein the correction direction comprises upward or downward; and moving at least one volume scale line below the liquid level position of the second image based on the correction direction. In this way, the volume scale information can be adjusted by moving part of the volume scale lines, and the specific moving direction can be determined based on the deviation between the reference liquid outflow and the predicted liquid outflow, so that the moved volume scale information is closer to the true information, and a more accurate liquid residual amount result can be obtained based thereon.

[0052] It can be understood that the method 200 described above can be executed multiple times during the infusion process. In some examples, the image acquisition position for the target infusion container can be fixed and unchanged, in which case the same volume scale information can be generated for the image sequence composed of multiple images of the target infusion container, and then the volume scale information can be corrected multiple times based on the liquid drop rate during the infusion process and the reference liquid outflow. For example, when the infusion process has not started or has just started, the initial liquid level position can be used as a reference volume scale line corresponding to the labeled volume of the target infusion container, and the predicted volume scale lines below the liquid level position can be corrected based on the reference volume scale line. By applying the above technical means to move the volume scale lines below the liquid level position during each scale correction process, the top-down scale line correction can be completed as the liquid level position continuously decreases during the infusion process, so that more accurate volume scale information can be obtained, and a more accurate liquid residual amount result can be obtained based thereon.

[0053] In some examples, the correction step for the volume scale information can also be implemented in other ways. For example, at least one of the relative size relationship between the predicted liquid outflow amount and the reference liquid outflow amount and the absolute value of the difference between the two can be used to select at least one volume scale line to be moved from all the volume scale lines according to a pre-set rule, and then a moving parameter is determined based on the deviation between the predicted liquid outflow amount and the reference liquid outflow amount, so that the volume scale information after moving is closer to the true information, and a more accurate liquid remaining amount result is obtained.

[0054] According to some embodiments, the method 200 further includes determining a correction distance of the volume scale line based on the deviation between the predicted liquid outflow amount and the reference liquid outflow amount, and wherein the moving at least one volume scale line below the second liquid level position based on the correction direction includes moving at least one volume scale line below the second liquid level position based on the correction direction and the correction distance. In this way, the correction distance of the volume scale line can be further determined based on the deviation between the reference liquid outflow amount and the predicted liquid outflow amount, so that the volume scale information after moving is closer to the true information.

[0055] According to some embodiments, the correcting the second volume scale information based on the deviation in response to determining that the deviation between the predicted liquid outflow amount and the reference liquid outflow amount exceeds the first deviation threshold includes correcting the second volume scale information based on the deviation in response to determining that the deviation between the predicted liquid outflow amount and the reference liquid outflow amount exceeds the first deviation threshold and does not exceed a second deviation threshold. It should be understood that when the initial deviation of the volume scale information is too large, the possibility of obtaining more accurate volume scale information based on a simple correction step is low. By setting a second deviation threshold for the step of correcting based on the deviation, the correction can be performed when the deviation does not exceed the larger second deviation threshold, improving the correction efficiency and avoiding waste of computing resources.

[0056] According to some embodiments, the method 200 further includes, in response to determining that the deviation between the predicted liquid outflow amount and the reference liquid outflow amount exceeds the first deviation threshold, acquiring a third image of the target infusion container at a third time; and determining a second liquid remaining amount of the target infusion container at the third time based on the third image. By applying the above technical means, when the deviation between the reference liquid outflow amount and the predicted liquid outflow amount is too large, a new image is directly acquired and a new step of determining the liquid remaining amount is performed, which can avoid the influence of inaccurate information on the monitoring accuracy.

[0057] It should be understood that the specific implementation of determining the second liquid remaining amount of the target infusion container at the third time based on the third image can be similar to the method 200 described above, and the present disclosure will not be repeated here.

[0058] In some examples, when the step of correcting based on the deviation is provided with the second deviation threshold as described in the foregoing, the step of directly capturing a new image of the target infusion image and re-determining the liquid residual volume can be performed when the deviation between the predicted liquid outflow and the reference liquid outflow exceeds the second deviation threshold, to achieve more efficient and accurate infusion monitoring.

[0059] According to some embodiments, the determining, in the step S203, of the predicted liquid outflow of the target infusion container between the first time and the second time based on the first image, the second image, the first volume scale information and the second volume scale information comprises: determining, based on the first image and the second image, first color value distribution information of the target infusion container at the first time and second color value distribution information of the target infusion container at the second time; and determining the predicted liquid outflow based on the first color value distribution information, the second color value distribution information, the first volume scale information and the second volume scale information. Thereby, the liquid volume in the container can be predicted based on color distribution information, improving the prediction accuracy.

[0060] In some examples, the first image and the first volume scale information can be input into a computer vision neural network model to obtain the predicted liquid residual volume of the target infusion container at the first time, or the predicted liquid residual volume of the target infusion container at the first time can also be directly obtained by using the visual model described in the foregoing. Correspondingly, the predicted liquid residual volume of the target infusion container at the second time can also be obtained in a similar manner, on the basis of which, the predicted liquid outflow can be obtained by subtracting the two sets of predicted liquid residual volumes of the target infusion container at the first time and the second time.

[0061] According to some embodiments, the determining, based on the first color value distribution information, the second color value distribution information, the first volume scale information and the second volume scale information, of the predicted liquid outflow comprises: determining a first liquid surface position in the first image based on the first color value distribution information; determining a second liquid surface position in the second image based on the second color value distribution information; and determining the predicted liquid outflow based on the first liquid surface position, the second liquid surface position, the first volume scale information and the second volume scale information. Thereby, the liquid volume in the container can be more simply predicted based on the liquid surface position information.

[0062] In some examples, the liquid drop speed in step S204 can be directly obtained based on the liquid sensor or the infusion controller. In some examples, the liquid drop speed in step S204 can also be obtained by collecting a drop video of the target infusion container between the first time and the second time and performing video content recognition. The video content recognition operation can be implemented by using various image frame processing algorithms, and the present disclosure does not limit the same.

[0063] After determining the liquid drop speed of the target infusion container, the number of liquid drops can be calculated based on the time length between the first time and the second time, and then the reference liquid outflow amount can be calculated. It should be understood that in actual application scenarios, different specifications of infusion devices correspond to different drop coefficients (for example, 20 drops per milliliter). In the case of determining the number of liquid drops based on the liquid drop speed and the time length, the reference liquid outflow amount can be calculated in combination with the drop coefficient of the infusion device, and the capacity scale information is corrected based on this to improve monitoring accuracy.

[0064] According to some embodiments, determining, in step S206, the first liquid residual amount of the target infusion container at the second time based on the second capacity scale information and the second image includes: generating an instruction for querying the first liquid residual amount to a large model based on the second capacity scale information and image feature information of the second image; and obtaining the first liquid residual amount output by the large model by inputting the instruction into the large model. In this way, the liquid residual amount can be obtained by using a large model to achieve efficient automatic monitoring.

[0065] In some examples, the image feature information can include at least one of second color value distribution information of the target infusion container at the second time and a second liquid surface position in the second image, so that the large model can infer the first liquid residual amount based on the liquid region information of the target infusion container at the second time and the capacity scale information. In some examples, step S206 can also be implemented in other ways, for example, after determining the second liquid surface position of the target infusion container at the second time, the first liquid residual amount can be determined based on the relative positional relationship between the second liquid surface position and the plurality of capacity scale lines to improve the prediction efficiency of the liquid residual amount.

[0066] According to some embodiments, the method 200 further includes: in response to determining that the first liquid residual amount does not exceed a liquid residual amount threshold, generating liquid residual amount alarm information. By alarming when the liquid residual amount is too low, the needs of actual application scenarios can be met, accidents can be avoided, and the safety of the infusion process can be ensured. In some examples, the liquid residual amount alarm information can be audio information, or it can also be text, pictures or videos for display on a display screen, and the present disclosure does not limit the specific form of the alarm information.

[0067] According to some embodiments, method 200 further includes: in response to determining that the first liquid remaining amount exceeds the liquid remaining amount threshold, determining a fourth moment based on the difference between the first liquid remaining amount and the liquid remaining amount threshold and the liquid dripping rate; capturing a fourth image of the target infusion container at the fourth moment; and determining a third liquid remaining amount of the target infusion container at the fourth moment based on the fourth image. By determining the timing for the next image capture and monitoring of the new liquid remaining amount based on the liquid remaining amount, i.e., eliminating the need for continuous monitoring of the liquid remaining amount, the safety of the infusion process can be ensured while saving hardware resources, thereby achieving more efficient infusion monitoring.

[0068] It should be understood that the specific implementation of determining the third liquid remaining amount of the target infusion container at the fourth moment based on the fourth image may be similar to the above-mentioned method 200, and will not be described in detail in this disclosure.

[0069] Figure 3 FIG. 1 shows a schematic diagram of the structure of an infusion monitoring system according to an exemplary embodiment of the present disclosure. Figure 3 As shown, the infusion monitoring system includes a camera 301, a visual model 302, a large model 303, and an alarm 304. In this example, the camera 301 is used to capture an image of the target infusion container and transmit it to the visual model 302, so that the visual model 302 can output volume scale information based on the image of the target infusion container. Furthermore, the technical means described above can be applied to verify and correct the volume scale information output by the visual model 302 based on the liquid dripping rate of the target infusion container. If the volume scale information is determined to be relatively accurate, an instruction can be generated to query the large model 303 for the liquid remaining in the target infusion container based on the volume scale information output by the visual model 302 and the image feature information of the target infusion container, so as to obtain the liquid remaining result returned by the large model 303. When the liquid remaining amount falls below a preset liquid remaining threshold, a liquid remaining alarm message can be generated and broadcasted using the alarm 304. By applying the above-mentioned infusion monitoring system to monitor and alarm during the infusion process, accurate and efficient automated infusion monitoring can be achieved, saving human resources and ensuring the safety of the infusion process.

[0070] According to one aspect of the present disclosure, a visual model-based infusion monitoring device is also provided. Figure 4 FIG. 4 shows a structural block diagram of an infusion monitoring device 400 according to an exemplary embodiment of the present disclosure. Figure 4 As shown, the apparatus 400 includes:

[0071] An acquisition unit 401 is configured to acquire a first image of a target infusion container at a first moment and a second image of the target infusion container at a second moment;

[0072] The acquisition unit 402 is configured to acquire first capacity scale information of the target infusion container and second capacity scale information of the target infusion container output by a visual model by inputting the first image and the second image into the visual model, wherein the visual model is trained by using sample images of sample liquid containers and reference capacity scale information of the sample liquid containers.

[0073] The first determination unit 403 is configured to determine a predicted liquid outflow amount of the target infusion container between the first time and the second time based on the first image, the second image, the first capacity scale information, and the second capacity scale information.

[0074] The second determination unit 404 is configured to determine a liquid drop rate of the target infusion container between the first time and the second time.

[0075] The third determination unit 405 is configured to determine a reference liquid outflow amount of the target infusion container between the first time and the second time based on the liquid drop rate; and

[0076] The fourth determination unit 406 is configured to determine a first liquid residual amount of the target infusion container at the second time based on the second capacity scale information and the second image in response to determining that a deviation between the predicted liquid outflow amount and the reference liquid outflow amount does not exceed a first deviation threshold.

[0077] According to some embodiments, the apparatus 400 further comprises a correction unit configured to correct the second capacity scale information based on the deviation in response to determining that the deviation between the predicted liquid outflow amount and the reference liquid outflow amount exceeds the first deviation threshold, wherein the fourth determination unit is further configured to determine the first liquid residual amount of the target infusion container at the second time based on the corrected second capacity scale information and the second image.

[0078] According to some embodiments, the second capacity scale information comprises a plurality of capacity scale lines distributed on the target infusion container along a vertical direction, the correction unit comprises: a first determination sub-unit configured to determine a relative size relationship of the predicted liquid outflow amount and the reference liquid outflow amount; a second determination sub-unit configured to determine a correction direction of a capacity scale line based on the relative size relationship, wherein the correction direction comprises upward or downward; and a moving sub-unit configured to move at least one capacity scale line below a liquid level position of the second image based on the correction direction.

[0079] According to some embodiments, the correction unit further includes a third determination subunit configured to determine a correction distance of the volume scale based on a deviation between the predicted liquid outflow amount and the reference liquid outflow amount, wherein the moving subunit is configured to move at least one volume scale located below the second liquid level position based on the correction direction and the correction distance.

[0080] According to some embodiments, the correction unit is configured to correct the second volume scale information based on the deviation in response to determining that the deviation between the predicted liquid outflow amount and the reference liquid outflow amount exceeds the first deviation threshold and does not exceed a second deviation threshold.

[0081] According to some embodiments, the acquisition unit is further configured to acquire a third image of the target infusion container at a third time in response to determining that the deviation between the predicted liquid outflow amount and the reference liquid outflow amount exceeds the first deviation threshold, and the device 400 is further configured to determine a second liquid remaining amount of the target infusion container at the third time based on the third image.

[0082] According to some embodiments, the first determination unit 403 includes a fourth determination subunit configured to determine first color value distribution information of the target infusion container at the first time and second color value distribution information of the target infusion container at the second time based on the first image and the second image, and a fifth determination subunit configured to determine the predicted liquid outflow amount based on the first color value distribution information, the second color value distribution information, the first volume scale information, and the second volume scale information.

[0083] According to some embodiments, the fifth determination subunit includes a first determination module configured to determine a first liquid level position in the first image based on the first color value distribution information, a second determination module configured to determine a second liquid level position in the second image based on the second color value distribution information, and a third determination module configured to determine the predicted liquid outflow amount based on the first liquid level position, the second liquid level position, the first volume scale information, and the second volume scale information.

[0084] According to some embodiments, the fourth determination unit includes a generation subunit configured to generate an instruction for querying the first liquid remaining amount from a large model based on the second volume scale information and image feature information of the second image, and an acquisition subunit configured to acquire the first liquid remaining amount output by the large model by inputting the instruction into the large model.

[0085] According to some embodiments, the apparatus 400 further comprises a generating unit configured to generate liquid volume alarm information in response to determining that the first liquid volume does not exceed the liquid volume threshold.

[0086] According to some embodiments, the apparatus 400 further comprises a fifth determining unit configured to determine a fourth time based on a difference between the first liquid volume and the liquid volume threshold and the liquid drop rate in response to determining that the first liquid volume exceeds the liquid volume threshold, and the collecting unit is further configured to collect a fourth image of the target infusion container at the fourth time, and the apparatus 400 is further configured to determine a third liquid volume of the target infusion container at the fourth time based on the fourth image.

[0087] According to another aspect of the present disclosure, an electronic device is also provided, comprising at least one processor, and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the infusion monitoring method described above.

[0088] According to another aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is also provided, wherein the computer instructions are used to enable the computer to perform the infusion monitoring method described above.

[0089] According to another aspect of the present disclosure, a computer program product comprising a computer program is also provided, wherein the computer program, when executed by a processor, implements the infusion monitoring method described above.

[0090] Reference Figure 5 A block diagram of an electronic device 500 that can be a server or a client of the present disclosure will now be described, which is an example of a hardware device that can be applied to aspects of the present disclosure. The electronic device is intended to represent a wide variety of digital electronic computing devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computing devices. The electronic device can also represent a wide variety of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections, and their functions, as well as their relationships to one another, are merely examples and are not meant to limit the implementations of the present disclosure described and / or claimed in this document.

[0091] As Figure 5As shown, the device 500 includes a computing unit 501 that can perform various appropriate actions and processes according to a computer program stored in a read only memory (ROM) 502 or a computer program loaded into a random access memory (RAM) 503 from a storage unit 508. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0092] A plurality of components in the device 500 are connected to the I / O interface 505, including an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. The input unit 506 can be any kind of device capable of inputting information to the device 500, can receive inputted digital or character information, and generate key signal inputs related to user settings and / or function controls of the electronic device, and can include, but is not limited to, a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 507 can be any kind of device capable of presenting information, and can include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 508 can include, but is not limited to, a magnetic disk, an optical disk. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0093] The computing unit 501 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs various methods and processes described above, such as the infusion monitoring method. For example, in some embodiments, the infusion monitoring method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded onto the RAM 503 and executed by the computing unit 501, one or more steps of the infusion monitoring method described above can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform the infusion monitoring method by any other appropriate means, such as by means of firmware.

[0094] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0095] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0096] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0097] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0098] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0099] The computer system can include clients and servers. This relationship can be remote or on-site. The servers can be cloud servers, servers of a distributed system, or servers combined with a blockchain.

[0100] It should be understood that the various forms of flow shown above can be used with reordering, additions, or deletions in the steps. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, which are not limited herein.

[0101] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-described methods, systems, and devices are merely exemplary embodiments or examples, and the scope of the present disclosure is not limited by these embodiments or examples, but is only limited by the granted claims and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, various elements in the embodiments or examples can be combined in various ways. It is important that many of the elements described herein can be replaced by equivalent elements that appear after the present disclosure as technology evolves.

Claims

1. A method for monitoring infusion based on a visual model, comprising: capturing a first image of a target infusion container at a first moment and a second image of the target infusion container at a second moment; inputting the first image and the second image into a visual model to obtain first volume scale information of the target infusion container and second volume scale information of the target infusion container output by the visual model, wherein the visual model is trained using a sample image including a sample liquid container and reference volume scale information of the sample liquid container; determining a predicted liquid outflow volume of the target infusion container between the first moment and the second moment based on the first image, the second image, the first volume scale information, and the second volume scale information; determining a liquid dripping rate of the target infusion container between the first moment and the second moment; determining a reference liquid outflow volume of the target infusion container between the first moment and the second moment based on the liquid dripping rate; and In response to determining that the deviation between the predicted liquid outflow amount and the reference liquid outflow amount does not exceed a first deviation threshold, a first liquid remaining amount of the target infusion container at the second moment is determined based on the second volume scale information and the second image.

2. The method of claim 1, further comprising: in response to determining that a deviation between the predicted liquid outflow amount and the reference liquid outflow amount exceeds the first deviation threshold, correcting the second volume scale information based on the deviation; as well as Based on the corrected second volume scale information and the second image, a first liquid remaining amount of the target infusion container at the second moment is determined.

3. The method according to claim 2, wherein: The second volume scale information includes a plurality of volume scale lines vertically distributed on the target infusion container, and in response to determining that a deviation between the predicted liquid outflow amount and the reference liquid outflow amount exceeds a first deviation threshold, correcting the second volume scale information based on the deviation includes: Determining a relative magnitude relationship between the predicted liquid outflow and the reference liquid outflow; Based on the relative size relationship, determining a correction direction of the capacity scale line, wherein the correction direction includes upward or downward; and At least one volume scale line located below the liquid level position in the second image is moved based on the correction direction.

4. The method of claim 3, further comprising: determining a correction distance of the capacity scale line based on a deviation between the predicted liquid outflow and the reference liquid outflow, Wherein, moving at least one capacity scale line located below the second liquid level position based on the correction direction includes: At least one capacity scale line located below the second liquid level position is moved based on the correction direction and the correction distance.

5. The method according to any one of claims 2 to 4, wherein In response to determining that a deviation between the predicted liquid outflow amount and the reference liquid outflow amount exceeds a first deviation threshold, correcting the second volume scale information based on the deviation includes: In response to determining that a deviation between the predicted liquid outflow amount and the reference liquid outflow amount exceeds the first deviation threshold and does not exceed a second deviation threshold, the second volume scale information is corrected based on the deviation.

6. The method according to any one of claims 1 to 4, further comprising: In response to determining that the deviation between the predicted liquid outflow amount and the reference liquid outflow amount exceeds the first deviation threshold, acquiring a third image of the target infusion container at a third moment; and A second remaining liquid volume of the target infusion fluid container at the third moment is determined based on the third image.

7. The method according to any one of claims 1 to 6, wherein The determining, based on the first image, the second image, the first volume scale information, and the second volume scale information, of the predicted liquid outflow of the target infusion container between the first moment and the second moment includes: determining, based on the first image and the second image, first color value distribution information of the target infusion container at the first moment and second color value distribution information of the target infusion container at the second moment; and The predicted liquid outflow amount is determined based on the first color value distribution information, the second color value distribution information, the first capacity scale information, and the second capacity scale information.

8. The method of claim 7, wherein: The determining the predicted liquid outflow amount based on the first color value distribution information, the second color value distribution information, the first capacity scale information, and the second capacity scale information includes: determining a first liquid level position in the first image based on the first color value distribution information; determining a second liquid level position in the second image based on the second color value distribution information; and The predicted liquid outflow amount is determined based on the first liquid level position, the second liquid level position, the first capacity scale information, and the second capacity scale information.

9. The method according to any one of claims 1 to 8, wherein The determining, based on the second volume scale information and the second image, a first liquid remaining amount of the target infusion container at the second moment includes: generating, based on the second volume scale information and the image feature information of the second image, an instruction for inquiring the large model about the first liquid remaining amount; and The first liquid residual amount output by the large model is obtained by inputting the instruction into the large model.

10. The method according to any one of claims 1 to 9, further comprising: In response to determining that the first liquid remaining amount does not exceed a liquid remaining amount threshold, liquid remaining amount warning information is generated.

11. The method of claim 10, further comprising: In response to determining that the first liquid remaining amount exceeds the liquid remaining amount threshold, determining a fourth time based on a difference between the first liquid remaining amount and the liquid remaining amount threshold and the liquid dripping rate; capturing a fourth image of the target infusion container at the fourth moment; and A third remaining liquid volume of the target infusion container at the fourth moment is determined based on the fourth image.

12. A visual model-based infusion monitoring device, comprising: an acquisition unit configured to acquire a first image of a target infusion container at a first moment and a second image of the target infusion container at a second moment; an acquisition unit configured to acquire first volume scale information and second volume scale information of the target infusion container output by the visual model by inputting the first image and the second image into a visual model, wherein the visual model is trained using a sample image including a sample liquid container and reference volume scale information of the sample liquid container; a first determining unit configured to determine a predicted liquid outflow volume of the target infusion container between the first moment and the second moment based on the first image, the second image, the first volume scale information, and the second volume scale information; a second determining unit configured to determine a liquid dripping rate of the target infusion container between the first moment and the second moment; a third determining unit configured to determine a reference liquid outflow volume of the target infusion container between the first moment and the second moment based on the liquid dripping rate; and A fourth determination unit is configured to determine, in response to determining that a deviation between the predicted liquid outflow amount and the reference liquid outflow amount does not exceed a first deviation threshold, a first liquid remaining amount of the target infusion container at the second moment based on the second volume scale information and the second image.

13. An electronic device comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 11.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable a computer to execute the method according to any one of claims 1-11.

15. A computer program product comprising a computer program, wherein The computer program implements the method according to any one of claims 1 to 11 when executed by a processor.

Images