Liquid capacity identification method and device based on visual model, equipment and medium

Abstract

The invention provides an infusion monitoring method and device based on a visual model, equipment and a medium, and relates to the technical field of artificial intelligence, in particular to the technical field of computer vision and image processing. According to the implementation scheme, a first image of a target liquid container is collected; inputting the first image into a first visual model to obtain a liquid level position line of a target liquid container and a container boundary line of the target liquid container output by the first visual model; determining at least one container area polygon corresponding to the target liquid container and at least one liquid area polygon capable of indicating distribution of liquid in the target liquid container based on the liquid level position line and the container boundary line; and determining a current liquid capacity of the target liquid container based on the at least one container region polygon, the at least one liquid region polygon, and a nominal capacity of the target liquid container.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the technical field of computer vision and image processing, and more particularly to a liquid capacity recognition method and device based on a visual model, an electronic device, a computer readable storage medium and a computer program product. BACKGROUND

[0002] Artificial intelligence is a discipline that studies enabling computers to simulate some thinking processes and intelligent behaviors of human beings (such as learning, reasoning, thinking, planning, etc.), which includes both hardware technology and software technology. Artificial intelligence hardware technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing, etc.; artificial intelligence software technology mainly includes computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, knowledge graph technology, etc.

[0003] With the development of computer technology, computer vision technology based on artificial intelligence is widely used in various fields. In traditional medical scenarios, the monitoring of the liquid residual amount of the infusion bag during the infusion process usually relies on manual work, which not only occupies human resources but also cannot effectively guarantee the safety of the infusion process.

[0004] The methods described in this section can not necessarily be the methods previously conceived or adopted. Unless otherwise indicated, nothing in this section should be assumed to be prior art merely because it is included in this section. Similarly, unless otherwise indicated, issues raised in this section should not be assumed to have been recognized in any prior art. SUMMARY

[0005] The present disclosure provides a liquid capacity recognition method and device based on a visual model, an electronic device, a computer readable storage medium and a computer program product.

[0006] According to one aspect of the present disclosure, a liquid capacity recognition method based on a visual model is provided, comprising: acquiring a first image of a target liquid container; obtaining a liquid level position line of the target liquid container and a container boundary line of the target liquid container output by the first visual model by inputting the first image into a first visual model, wherein the first visual model is trained using a sample image including a sample liquid container, a reference liquid level position line of the sample liquid container, and a reference container boundary line of the sample liquid container; determining at least one container area polygon corresponding to the target liquid container and at least one liquid area polygon capable of indicating the distribution of the liquid in the target liquid container based on the liquid level position line and the container boundary line; and determining the current liquid capacity of the target liquid container based on the at least one container area polygon, the at least one liquid area polygon, and the nominal capacity of the target liquid container.

[0007] According to another aspect of the present disclosure, a liquid capacity recognition device based on a visual model is provided, comprising: an acquisition unit configured to acquire a first image of a target liquid container; an acquisition unit configured to acquire the liquid level position line of the target liquid container and the container boundary line of the target liquid container output by the first visual model by inputting the first image into a first visual model, wherein the first visual model is trained using a sample image including a sample liquid container, a reference liquid level position line of the sample liquid container, and a reference container boundary line of the sample liquid container; a first determination unit configured to determine, based on the liquid level position line and the container boundary line, at least one container area polygon corresponding to the target liquid container and at least one liquid area polygon capable of indicating the distribution of the liquid in the target liquid container; and a second determination unit configured to determine the current liquid capacity of the target liquid container based on the at least one container area polygon, the at least one liquid area polygon, and the nominal capacity of the target liquid container.

[0008] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned visual model-based liquid capacity recognition method.

[0009] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above-mentioned liquid capacity recognition method based on the visual model.

[0010] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, can implement the above-mentioned liquid capacity recognition method based on a visual model.

[0011] According to one or more embodiments of the present disclosure, the amount of liquid in a liquid container can be identified more efficiently and accurately.

[0012] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings illustrate exemplary embodiments and constitute a part of the specification. Together with the description of the specification, they serve to explain exemplary implementation of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals designate similar, but not necessarily identical, elements.

[0014] Figure 1 A schematic diagram illustrating an exemplary system in which the various methods described herein may be implemented according to exemplary embodiments of the present disclosure;

[0015] Figure 2 A flow chart of a liquid capacity recognition method based on a visual model according to an exemplary embodiment of the present disclosure is shown;

[0016] Figure 3 A schematic structural diagram of a liquid remaining amount monitoring system according to an exemplary embodiment of the present disclosure is shown;

[0017] Figure 4 A structural block diagram of a liquid capacity recognition device based on a visual model according to an exemplary embodiment of the present disclosure is shown;

[0018] Figure 5 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0019] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0020] In the present disclosure, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, the timing relationship or the importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element can refer to the same instance of the element, and in some cases, based on the description of the context, they can also refer to different instances.

[0021] The terms used in the description of various described examples in the present disclosure are only for the purpose of describing the specific examples, and are not intended to be limiting. Unless the number of elements is specifically limited, the element can be one or more, if the number of elements is not specifically limited. In addition, the term "and / or" used in the present disclosure encompasses any one of the listed items and all possible combinations.

[0022] Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0023] Figure 1 A schematic diagram of an example system 100 in which various methods and apparatus described herein can be implemented according to embodiments of the present disclosure is shown. Referring to Figure 1 The system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 can be configured to execute one or more application programs.

[0024] In embodiments of the present disclosure, the server 120 can run one or more services or software applications that enable the execution of a visual model based liquid capacity recognition method.

[0025] In certain embodiments, the server 120 can also provide other services or software applications, which can include non-virtual environments and virtual environments. In certain embodiments, these services can be provided as web-based services or cloud services, for example, to users of the client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0026] In Figure 1In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may, in turn, utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0027] The user may use client devices 101, 102, 103, 104, 105, and / or 106 to send an image including a liquid container. The client device may provide an interface that enables the user of the client device to interact with the client device. The client device may also output information to the user via the interface. Figure 1 Only six client devices are depicted, but one skilled in the art will appreciate that the present disclosure can support any number of client devices.

[0028] Client devices 101, 102, 103, 104, 105, and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include cellular phones, smartphones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Gaming systems may include various handheld gaming devices, internet-enabled gaming devices, etc. The client device is capable of executing various different applications, such as various Internet-related applications, communication applications (eg, email applications), Short Message Service (SMS) applications, and may use various communication protocols.

[0029] The network 110 may be any type of network known to those skilled in the art that can support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0030] Server 120 may include one or more general-purpose computers, specialized server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0031] The computing units in the server 120 may run one or more operating systems including any of the operating systems described above as well as any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, and the like.

[0032] In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display the data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.

[0033] In some embodiments, server 120 may be a distributed system server or a server integrated with blockchain. Server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host equipped with artificial intelligence technology. A cloud server is a host product within the cloud computing service system that addresses the management difficulties and poor scalability of traditional physical hosts and virtual private servers (VPS) services.

[0034] The system 100 can also include one or more databases 130. In certain embodiments, these databases can be used to store data and other information. For example, one or more of the databases 130 can be used to store information such as audio files and video files. The databases 130 can reside in various locations. For example, databases used by the server 120 can be local to the server 120, or can be remote from the server 120 and can communicate with the server 120 via a network- based or dedicated connection. The databases 130 can be of different categories. In certain embodiments, databases used by the server 120 can be, for example, relational databases. One or more of these databases can store, update, and retrieve data to and from the databases in response to commands.

[0035] In certain embodiments, one or more of the databases 130 can also be used by applications to store application data. Databases used by applications can be databases of different categories, such as key-value stores, object stores, or regular stores supported by file systems.

[0036] Figure 1 The system 100 of FIG. 1 can be configured and operated in various ways to enable the various methods and apparatuses described in accordance with the present disclosure to be applied.

[0037] In traditional medical scenarios, the monitoring and control of infusion processes usually rely on manual operation, which not only occupies human resources but also cannot effectively guarantee the safety of the infusion process.

[0038] Based on this, the present disclosure provides a visual model-based infusion monitoring method, which identifies the container boundary line and the liquid surface position line by using a visual model after collecting the image of the target liquid container, performs geometric region segmentation on the container according to the container boundary line and the liquid surface position line to obtain the container region polygon and the liquid region polygon, and then determines the liquid remaining amount of the liquid container based on the polygon geometric region information and the nominal capacity of the container.

[0039] Figure 2 A flowchart of a visual model-based liquid capacity identification method 200 according to an exemplary embodiment of the present disclosure is shown. As shown in FIG. 2, Figure 2 The method 200 includes the following steps:

[0040] Step S201, collecting a first image of a target liquid container;

[0041] Step S202: inputting the first image into a first visual model to obtain a liquid level position line of the target liquid container and a container boundary line of the target liquid container output by the first visual model, wherein the first visual model is trained using a sample image including a sample liquid container, a reference liquid level position line of the sample liquid container, and a reference container boundary line of the sample liquid container;

[0042] Step S203: determining at least one container area polygon corresponding to the target liquid container and at least one liquid area polygon capable of indicating distribution of liquid in the target liquid container based on the liquid level position line and the container boundary line; and

[0043] Step S204: Determine the current liquid capacity of the target liquid container based on the at least one container area polygon, the at least one liquid area polygon, and the nominal capacity of the target liquid container.

[0044] According to the method 200 provided in the present disclosure, after capturing an image of a target liquid container, a visual model can be used to identify the container boundary line and liquid level position line, and the container can be geometrically segmented based on the container boundary line and liquid level position line to obtain container area polygons and liquid area polygons, thereby determining the current liquid capacity in the container based on the polygon geometric area information and the container's nominal capacity. By applying the above-mentioned liquid capacity recognition method based on a visual model, it is possible to automatically monitor the remaining liquid amount by capturing an image of the target liquid container, and efficiently and accurately determine the current liquid capacity based on the geometric feature information in the image of the target liquid container, thereby improving the convenience of liquid capacity recognition.

[0045] The target liquid container can be various types of liquid containers, such as rigid bottles or flexible bags. In some examples, the nominal capacity of the target liquid container can be directly obtained as known explicit information. In some examples, the target liquid container can be marked with the nominal capacity, and the marking information of the target liquid container can be read by parsing the first image. For example, the target detection method can be used to detect the label of the target liquid container, and the nominal capacity can be obtained by OCR (Optical Character Recognition) technology.

[0046] In some examples, the liquid level position line and container boundary line output by the first visual model can be lines added to the first image. The liquid level position line and container boundary line can be curves or straight lines, and this disclosure does not limit this. In some examples, the liquid level position line and container boundary line output by the first visual model can also be line position coordinate information, and the position of the liquid level position line and container boundary line in the image is indicated based on the line position coordinate information. As long as the liquid level position and container boundary of the target liquid container can be indicated, this disclosure does not limit the shape or specific data form of the liquid level position line and container boundary line.

[0047] In some examples, the first visual model can be a Transformer model pre-trained using a sample image including a sample liquid container, a reference liquid level position line, and a reference container boundary line. In some examples, other types of neural networks (such as a ResNet network) can be used to extract image feature information, and then a Transformer network is used to generate the liquid level position line and container boundary line of the liquid container based on the image feature information. The training process of the visual model can be implemented based on a supervised training method, which will not be described in detail in this disclosure.

[0048] In some examples, step S203 may involve extracting multiple key points based on the liquid surface line and the container boundary line, and then connecting the key points to determine the boundaries of multiple polygons, thereby obtaining at least one container region polygon and at least one liquid region polygon. In some examples, the step of extracting multiple key points is achieved by detecting the curvature of the liquid surface line and the container boundary line. By determining points on the liquid surface line and the container boundary line whose curvature exceeds a preset threshold as key points and determining the container region polygon and the liquid region polygon based on these key points, the container region polygon and the liquid region polygon can be made to more accurately fit the container region and the liquid region in the first image.

[0049] According to some embodiments, determining the current liquid capacity of the target liquid container based on the at least one container area polygon, the at least one liquid area polygon, and the nominal capacity of the target liquid container in step S204 includes: determining a liquid area ratio in the target liquid container based on the sum of the areas of the at least one container area polygon and the sum of the areas of the at least one liquid area polygon; and determining the current liquid capacity of the target liquid container based on the liquid area ratio and the nominal capacity. By calculating the area based on each polygonal geometric area and determining the liquid ratio in the container, the current liquid capacity can be easily determined based on the liquid ratio and the nominal capacity.

[0050] In some examples, other methods may be used to predict the current liquid capacity of the target liquid container in step S204. For example, when the target liquid container has different shapes, the accuracy of the current liquid capacity predicted based on the two-dimensional container region polygon information and liquid region information is limited. In this case, the current liquid capacity of the target liquid container may be predicted by further combining the shape information of the target liquid container (e.g., conical or square pyramidal) to improve the accuracy of liquid capacity recognition.

[0051] According to some embodiments, method 200 further includes: in response to determining that the target liquid container includes at least one capacity scale line, determining scale position information of the at least one capacity scale line in the first image, wherein determining the current liquid capacity of the target liquid container based on the liquid area ratio and the nominal capacity includes: determining the current liquid capacity of the target liquid container based on the liquid area ratio, the nominal capacity, the liquid level position line, and the scale position information. When the container has capacity scale lines, determining the current liquid capacity by combining the liquid level position with the position of the capacity scale lines can improve the accuracy of liquid capacity recognition.

[0052] According to some embodiments, in response to determining that the target liquid container includes at least one volume scale line, determining the scale position information of the at least one volume scale line in the first image includes: inputting the first image into a second visual model to obtain the scale position information output by the second visual model, wherein the second visual model is trained using a sample image including a sample liquid container and reference scale position information of the sample liquid container. Thus, the trained visual model can be used to more efficiently and accurately extract volume scale line information from the container image.

[0053] In some examples, the second visual model and the aforementioned first visual model can be the same visual model, which can be trained using multiple sample images corresponding to multiple different sample liquid containers. In this case, the scale position information of the capacity scale line can be used as an optional output of the visual model. That is, the scale position information is only output when the liquid container image input to the model has capacity scale lines. Then, the first image of the target liquid container can be directly input into the visual model to directly identify the current liquid capacity of the target liquid container based on the output information of the visual model, simplifying the processing steps and improving recognition efficiency.

[0054] According to some embodiments, determining the current liquid capacity of the target liquid container based on the liquid area ratio, the nominal capacity, the liquid level line, and the scale position information includes: generating an instruction for querying the large model about the current liquid capacity based on the liquid area ratio, the nominal capacity, the liquid level line, and the scale position information; and obtaining the current liquid capacity output by the large model by inputting the instruction into the large model. Thus, the large model can be used to obtain liquid remaining information to achieve efficient automatic monitoring.

[0055] In some examples, the large model can be a large language model pre-trained based on large-scale corpus, which can intelligently understand and perform reasoning tasks based on input information, and then predict the current liquid capacity in the target liquid container based on the image information of the target liquid container identified by the visual model, thereby realizing automatic monitoring of the target liquid container.

[0056] According to some embodiments, method 200 further includes: determining, based on the first image, whether the target liquid container is a flexible container, wherein determining, in step S203, at least one container area polygon corresponding to the target liquid container and at least one liquid area polygon capable of indicating the distribution of the liquid in the target liquid container based on the liquid level position line and the container boundary line includes: in response to determining that the target liquid container is a flexible container, determining, based on the liquid level position line and the container boundary line, at least one container area polygon corresponding to the target liquid container and at least one liquid area polygon capable of indicating the distribution of the liquid in the target liquid container. By first identifying whether the target liquid container is a flexible container, the curve fitting polygon step can be specifically performed for flexible containers whose capacity is difficult to identify, so as to more accurately determine the liquid remaining amount based on the irregular boundary and irregular liquid level position line of the flexible container. For non-flexible containers, there is no need to perform the above steps, thereby saving hardware resources.

[0057] In some examples, whether the target liquid container is a flexible container can be directly obtained as explicit information, for example, by directly obtaining the type of the target liquid container (various rigid bottles, cans, boxes, or flexible bags of different materials and shapes), based on which it is indicated whether the target liquid container is a flexible container. In some examples, container type classification can also be performed based on the first image of the target liquid container, and the classification result can be used to indicate whether the target liquid container is a flexible container. This step can be implemented using an image classification model, which can be trained using sample images of sample liquid containers and reference classification labels of the sample liquid containers (whether they are flexible containers).

[0058] According to some embodiments, in response to determining that the target liquid container is a flexible container, determining at least one container area polygon corresponding to the target liquid container and at least one liquid area polygon capable of indicating the distribution of liquid in the target liquid container based on the liquid level position line and the container boundary line includes: in response to determining that the target liquid container is a flexible container, determining multiple liquid level position line segments based on the liquid level position line; determining multiple container boundary line segments based on the container boundary line; and determining the at least one container area polygon and the at least one liquid area polygon based on the multiple liquid level position line segments and the multiple container boundary line segments. When the target liquid container is a flexible container, the container boundary line and the liquid level position line are irregular curves. By fitting the liquid level position line and the container boundary line of the flexible container into multiple line segments, the container area polygon and the liquid area polygon can be more simply and efficiently obtained based on the fitting results.

[0059] In some examples, an image region segmentation model can be used to fit the liquid level line and container boundary line of a flexible container into multiple line segments and, based on these segments, determine the container region polygon and the liquid region polygon. By inputting information including the liquid level line and the container boundary line (this information can be in the form of graphical data or line position coordinates) into the image region segmentation model, the model outputs the determination results of at least one container region polygon and at least one liquid region polygon in a simple and efficient manner. In this example, the image region segmentation model can be trained in a supervised manner using labeled sample data to ensure the accuracy of polygon region segmentation.

[0060] According to some embodiments, method 200 further includes: in response to determining that the target liquid container is not a flexible container, determining the container height of the target liquid container and the liquid height of the liquid in the target liquid container based on the liquid level position line and the container boundary line; and determining the current liquid capacity of the target liquid container based on the container height, the liquid height, and the nominal capacity of the target liquid container. By directly determining the remaining liquid amount based on the container height and the liquid height for non-flexible containers (such as rigid bottles and cans), the efficiency of liquid capacity recognition can be improved and hardware resources can be saved.

[0061] According to some embodiments, method 200 further includes: generating a liquid remaining alarm message in response to determining that the current liquid capacity does not exceed a liquid remaining threshold. By issuing an alarm when the liquid remaining is too low, the needs of actual application scenarios can be met. For example, when method 200 is used to monitor an infusion container in a medical scenario, by issuing an alarm message when the infusion liquid remaining is too low, infusion accidents can be avoided and the safety of the infusion process can be ensured. In some examples, the liquid remaining alarm message can be an audio message, or it can be a text, picture, or video for display on a display screen. The present disclosure does not limit the specific form of the alarm message.

[0062] In some examples, when method 200 is applied to an infusion scenario, that is, when the target liquid container is a flexible infusion bag, the following methods can be further used to improve the accuracy of liquid remaining monitoring:

[0063] In this example, after capturing a first image of the target liquid container at a first moment, a second image of the target liquid container at a second moment can be further captured. By inputting the first and second images into a capacity labeling model, the capacity labeling model can be used to obtain first capacity scale information of the target liquid container marked on the first image and second capacity scale information of the target liquid container marked on the second image. The capacity labeling model can be trained using a sample image of the sample liquid container and reference capacity scale information of the sample liquid container.

[0064] Based on the volume scale information of the target liquid container marked using the volume marking model, the predicted liquid outflow of the target liquid container between the first moment and the second moment can be determined based on the first image, the second image, the first volume scale information, and the second volume scale information. This step can be calculated based on the relative position of the target liquid container's liquid level position line in the first image and the first volume scale information, and the relative position of the target liquid container's liquid level position line in the second image and the second volume scale information. Simultaneously, by obtaining the infusion liquid drip rate of the target liquid container, a reference liquid outflow of the target infusion container between the first moment and the second moment can be determined based on the liquid drip rate.

[0065] When the deviation between the predicted liquid outflow and the reference liquid outflow does not exceed the first deviation threshold, it indicates that the capacity scale information output by the capacity marking model meets the accuracy requirement, and thus the liquid capacity of the target liquid container at the second moment can be determined based on the liquid level position line, the container boundary line, and the second capacity scale information in the second image. The specific implementation method of this step can be similar to the method described above. For example, based on the liquid level position line, the container boundary line, and the second capacity scale information in the second image (the second capacity scale information may include the nominal capacity of the target liquid container), an instruction for querying the large model for the current liquid capacity can be generated. Then, by inputting the instruction into the large model, the current liquid capacity output by the large model is obtained, that is, using the large model to achieve efficient automatic monitoring.

[0066] According to some embodiments, method 200 further includes: in response to determining that the current liquid volume exceeds the liquid remaining threshold, determining a monitoring time based on the difference between the current liquid volume and the liquid remaining threshold; capturing a monitoring image of the target liquid container at the monitoring time; and determining the liquid volume of the target liquid container at the monitoring time based on the monitoring image. By determining the next time to capture an image and monitor the new liquid remaining volume based on the liquid remaining volume, i.e., eliminating the need for continuous monitoring of the liquid remaining volume, it is possible to avoid excessively low liquid remaining volume, ensure safety, save hardware resources, and achieve more efficient liquid volume monitoring.

[0067] It should be understood that the specific implementation of determining the liquid remaining amount of the target liquid container at the monitoring time based on the monitoring image may be similar to the above-mentioned method 200, and the present disclosure will not elaborate on this.

[0068] Figure 3 FIG. 1 shows a schematic structural diagram of a liquid remaining amount monitoring system according to an exemplary embodiment of the present disclosure. Figure 3 As shown, the liquid level monitoring system includes a camera 301, a visual model 302, a large model 303, and an alarm 304. In this example, the camera 301 is used to capture an image of the target liquid container and transmit it to the visual model 302, so that the visual model 302 can output the container boundary line and liquid level position line based on the image of the target liquid container. Furthermore, based on the container boundary line and liquid level position line output by the visual model 302, an instruction is generated to query the large model 303 for the liquid level of the target liquid container, and the large model 303 returns the current liquid capacity result. When the liquid level falls below a preset liquid level threshold, a liquid level alarm message is generated and broadcasted by the alarm 304. This system can be used to monitor the liquid level of an infusion container during an infusion process. By applying the above-mentioned infusion monitoring system to monitor and alarm during the infusion process, accurate and efficient automated infusion monitoring can be achieved, saving human resources and ensuring the safety of the infusion process.

[0069] According to one aspect of the present disclosure, a liquid capacity recognition device based on a visual model is also provided. Figure 4 FIG. 4 shows a structural block diagram of a liquid capacity recognition device 400 based on a visual model according to an exemplary embodiment of the present disclosure. Figure 4 As shown, the apparatus 400 includes:

[0070] An acquisition unit 401 is configured to acquire a first image of a target liquid container;

[0071] an acquisition unit 402 configured to acquire the liquid level position line of the target liquid container and the container boundary line of the target liquid container output by the first visual model by inputting the first image into a first visual model, wherein the first visual model is trained using a sample image including a sample liquid container, a reference liquid level position line of the sample liquid container, and a reference container boundary line of the sample liquid container;

[0072] A first determining unit 403 is configured to determine, based on the liquid level position line and the container boundary line, at least one container area polygon corresponding to the target liquid container and at least one liquid area polygon capable of indicating distribution of the liquid in the target liquid container; and

[0073] The second determining unit 404 is configured to determine the current liquid capacity of the target liquid container based on the at least one container area polygon, the at least one liquid area polygon, and the nominal capacity of the target liquid container.

[0074] According to some embodiments, the second determination unit 404 includes: a first determination subunit, configured to determine the liquid area ratio in the target liquid container based on the sum of the areas of the at least one container area polygon and the sum of the areas of the at least one liquid area polygon; and a second determination subunit, configured to determine the current liquid capacity of the target liquid container based on the liquid area ratio and the nominal capacity.

[0075] According to some embodiments, the device 400 also includes: a third determination unit, configured to determine the scale position information of the at least one capacity scale line in the first image in response to determining that the target liquid container includes at least one capacity scale line, wherein the second determination subunit is configured to: determine the current liquid capacity of the target liquid container based on the liquid area proportion, the nominal capacity, the liquid level position line and the scale position information.

[0076] According to some embodiments, the third determination unit is configured to obtain the scale position information output by the second visual model by inputting the first image into a second visual model, wherein the second visual model is trained using a sample image including a sample liquid container and reference scale position information of the sample liquid container.

[0077] According to some embodiments, the second determination subunit includes: a first determination module, configured to generate an instruction for inquiring the large model about the current liquid capacity based on the liquid area ratio, the nominal capacity, the liquid level position line and the scale position information; and an input module, configured to obtain the current liquid capacity output by the large model by inputting the instruction into the large model.

[0078] According to some embodiments, the device 400 also includes: a fourth determination unit, configured to determine whether the target liquid container is a flexible container based on the first image, wherein the first determination unit 403 is configured to: in response to determining that the target liquid container is a flexible container, determine at least one container area polygon corresponding to the target liquid container and at least one liquid area polygon capable of indicating the distribution of the liquid in the target liquid container based on the liquid level position line and the container boundary line.

[0079] According to some embodiments, the first determination unit 403 includes: a second determination module, configured to determine a plurality of liquid surface position line segments based on the liquid surface position line in response to determining that the target liquid container is a flexible container; a third determination module, configured to determine a plurality of container boundary line segments based on the container boundary line; and a fourth determination module, configured to determine the at least one container area polygon and the at least one liquid area polygon based on the plurality of liquid surface position line segments and the plurality of container boundary line segments.

[0080] According to some embodiments, the device 400 also includes: a fifth determination unit, configured to determine the container height of the target liquid container and the liquid height of the liquid in the target liquid container based on the liquid surface position line and the container boundary line in response to determining that the target liquid container is not a flexible container; and a sixth determination unit, configured to determine the current liquid capacity of the target liquid container based on the container height, the liquid height and the nominal capacity of the target liquid container.

[0081] According to some embodiments, the apparatus 400 further includes a generating unit configured to generate liquid remaining amount alarm information in response to determining that the current liquid capacity does not exceed a liquid remaining amount threshold.

[0082] According to some embodiments, the device 400 also includes a seventh determination unit, which is configured to determine a monitoring time based on a difference between the current liquid capacity and the liquid remaining threshold in response to determining that the current liquid capacity exceeds the liquid remaining threshold, the acquisition unit is further configured to acquire a monitoring image of the target liquid container at the monitoring time, and the second determination unit 404 is further configured to determine the liquid capacity of the target liquid container at the monitoring time based on the monitoring image.

[0083] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0084] According to another aspect of the present disclosure, an electronic device is also provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above-mentioned visual model-based liquid capacity recognition method.

[0085] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is further provided, wherein the computer instructions are used to enable the computer to execute the above-mentioned liquid capacity recognition method based on the visual model.

[0086] According to another aspect of the present disclosure, a computer program product is further provided, comprising a computer program, wherein the computer program implements the above-mentioned liquid capacity recognition method based on a visual model when executed by a processor.

[0087] refer to Figure 5 , a block diagram of an electronic device 500 that can serve as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0088] like Figure 5As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the device 500 can also be stored in the RAM 503. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0089] Multiple components within device 500 are connected to I / O interface 505, including an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. Input unit 506 can be any device capable of inputting information into device 500. Input unit 506 can receive input numeric or character information and generate key signal input related to user settings and / or function control of the electronic device. It may include, but is not limited to, a mouse, keyboard, touch screen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 507 can be any device capable of presenting information and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. Storage unit 508 may include, but is not limited to, a magnetic disk or an optical disk. Communication unit 509 allows device 500 to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks. It may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and / or chipset, such as a Bluetooth™ device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0090] The computing unit 501 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as the liquid capacity recognition method based on the visual model. For example, in some embodiments, the liquid capacity recognition method based on the visual model can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the liquid capacity recognition method based on the visual model described above can be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute the liquid volume recognition method based on a visual model in any other appropriate manner (for example, by means of firmware).

[0091] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0092] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0093] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0094] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0095] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0096] The computer system can include clients and servers. This relationship can be remote or on-site. The servers can be cloud servers, servers of a distributed system, or servers combined with a blockchain.

[0097] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps described in the present disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, which are not limited herein.

[0098] Although embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-described methods, systems, and devices are merely exemplary embodiments or examples, and the scope of the present disclosure is not limited by these embodiments or examples, but only by the granted claims and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, various elements in the embodiments or examples can be combined in various ways. It is important that many of the elements described herein can be replaced by equivalent elements that appear after the present disclosure as technology evolves.

Claims

1. A liquid capacity recognition method based on a visual model, comprising: Acquire a first image of the target liquid container; inputting the first image into a first visual model to obtain a liquid level position line of the target liquid container and a container boundary line of the target liquid container output by the first visual model, wherein the first visual model is trained using a sample image including a sample liquid container, a reference liquid level position line of the sample liquid container, and a reference container boundary line of the sample liquid container; determining, based on the liquid level position line and the container boundary line, at least one container area polygon corresponding to the target liquid container and at least one liquid area polygon capable of indicating distribution of liquid in the target liquid container; and A current liquid capacity of the target liquid container is determined based on the at least one container area polygon, the at least one liquid area polygon, and a nominal capacity of the target liquid container.

2. The method according to claim 1, wherein The determining the current liquid capacity of the target liquid container based on the at least one container area polygon, the at least one liquid area polygon, and the nominal capacity of the target liquid container comprises: determining a liquid area ratio in the target liquid container based on the sum of the areas of the at least one container area polygon and the sum of the areas of the at least one liquid area polygon; and Based on the liquid area ratio and the nominal capacity, a current liquid capacity of the target liquid container is determined.

3. The method of claim 2, further comprising: In response to determining that the target liquid container includes at least one capacity scale line, determining scale position information of the at least one capacity scale line in the first image, Wherein, determining the current liquid capacity of the target liquid container based on the liquid area proportion and the nominal capacity includes: Based on the liquid area ratio, the nominal capacity, the liquid level position line and the scale position information, the current liquid capacity of the target liquid container is determined.

4. The method according to claim 3, wherein: In response to determining that the target liquid container includes at least one capacity scale line, determining scale position information of the at least one capacity scale line in the first image includes: The scale position information output by the second visual model is obtained by inputting the first image into the second visual model, wherein the second visual model is trained using a sample image including a sample liquid container and reference scale position information of the sample liquid container.

5. The method according to claim 3 or 4, wherein: The determining of the current liquid capacity of the target liquid container based on the liquid area proportion, the nominal capacity, the liquid level position line, and the scale position information includes: Based on the liquid area ratio, the nominal capacity, the liquid level position line, and the scale position information, generating an instruction for querying the large model about the current liquid capacity; and The instruction is input into the large model to obtain the current liquid capacity output by the large model.

6. The method according to any one of claims 1 to 5, further comprising: determining whether the target liquid container is a flexible container based on the first image, Wherein, determining at least one container area polygon corresponding to the target liquid container and at least one liquid area polygon capable of indicating the distribution of the liquid in the target liquid container based on the liquid level position line and the container boundary line includes: In response to determining that the target liquid container is a flexible container, at least one container area polygon corresponding to the target liquid container and at least one liquid area polygon capable of indicating distribution of liquid in the target liquid container are determined based on the liquid level position line and the container boundary line.

7. The method according to claim 6, wherein: In response to determining that the target liquid container is a flexible container, determining at least one container area polygon corresponding to the target liquid container and at least one liquid area polygon capable of indicating distribution of the liquid in the target liquid container based on the liquid level position line and the container boundary line includes: In response to determining that the target liquid container is a flexible container, determining a plurality of liquid level position line segments based on the liquid level position line; determining a plurality of container boundary line segments based on the container boundary line; and Based on the plurality of liquid level position line segments and the plurality of container boundary line segments, the at least one container area polygon and the at least one liquid area polygon are determined.

8. The method according to claim 6 or 7, further comprising: In response to determining that the target liquid container is not a flexible container, determining a container height of the target liquid container and a liquid height of the liquid in the target liquid container based on the liquid level position line and the container boundary line; and A current liquid capacity of the target liquid container is determined based on the container height, the liquid height, and the nominal capacity of the target liquid container.

9. The method according to any one of claims 1 to 8, further comprising: In response to determining that the current liquid capacity does not exceed the liquid remaining threshold, liquid remaining warning information is generated.

10. The method of claim 9, further comprising: In response to determining that the current liquid capacity exceeds the liquid remaining threshold, determining a monitoring time based on a difference between the current liquid capacity and the liquid remaining threshold; collecting a monitoring image of the target liquid container at the monitoring moment; as well as The liquid capacity of the target liquid container at the monitoring time is determined based on the monitoring image.

11. A liquid capacity recognition device based on a visual model, comprising: an acquisition unit configured to acquire a first image of the target liquid container; an acquisition unit configured to acquire the liquid level position line of the target liquid container and the container boundary line of the target liquid container output by the first visual model by inputting the first image into a first visual model, wherein the first visual model is trained using a sample image including a sample liquid container, a reference liquid level position line of the sample liquid container, and a reference container boundary line of the sample liquid container; a first determining unit configured to determine, based on the liquid level position line and the container boundary line, at least one container area polygon corresponding to the target liquid container and at least one liquid area polygon capable of indicating distribution of the liquid in the target liquid container; and A second determining unit is configured to determine a current liquid capacity of the target liquid container based on the at least one container area polygon, the at least one liquid area polygon, and a nominal capacity of the target liquid container.

12. An electronic device comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1-10.

14. A computer program product comprising a computer program, wherein The computer program implements the method according to any one of claims 1 to 10 when executed by a processor.

Images