Liquid level detection method and device, water dispenser, storage medium and program product

Abstract

This disclosure relates to a liquid level detection method, apparatus, water dispenser, storage medium, and program product. The method includes: acquiring a first image; determining a first area ratio between the upper surface of the liquid and the container opening based on the first image; and determining liquid level information within a first container based on the first area ratio. The first image is an image frame from a first video, which is a video taken during the process of injecting liquid into the first container at the container opening. Thus, using the first area ratio obtained from the first image to determine the liquid level information improves the accuracy and reliability of the liquid level information; furthermore, this method has the advantages of being non-contact and having wide applicability.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a liquid level detection method, apparatus, water dispenser, storage medium, and program product. Background Technology

[0002] Common liquid level detection technologies can be divided into two categories: contact liquid level detection and non-contact liquid level detection. Since the water in the glass is for drinking, a non-contact liquid level detection technology is needed to detect its level. However, current non-contact liquid level detection technologies struggle to achieve accurate level detection. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a liquid level detection method, device, water dispenser, storage medium, and program product to obtain liquid level information with high accuracy.

[0004] According to a first aspect of the present disclosure, a liquid level detection method is provided, comprising: Acquire a first image, which is an image frame of a first video, and the first video is a video of the opening of the first container captured during the process of injecting liquid into the first container. Based on the first image, determine a first area ratio between the upper surface of the liquid and the opening of the container; Based on the first area ratio, the liquid level information of the liquid in the first container is determined.

[0005] Optionally, acquiring the first image includes: Each image frame of the first video is taken as the first image; or The image frame located at a preset position in the first video is taken as the first image; Determining the liquid level information of the liquid in the first container based on the first area ratio includes: The liquid level information is determined based on the changing trend of the first area ratio.

[0006] Optionally, determining the liquid level information based on the changing trend of the first area ratio includes: Among multiple preset change trends, a first preset change trend that matches the change trend of the first area ratio is determined, wherein the preset change trend is used to characterize the change trend of the second area ratio of the upper surface of the liquid in the second container to the container opening of the second container with the liquid level information of the liquid in the second container, and different preset change trends correspond to different second containers. Based on the first preset change trend, liquid level information corresponding to the first area ratio determined based on the most recent frame of the first image is determined as the liquid level information of the liquid in the first container.

[0007] Optionally, the liquid level detection method further includes: Get the height of the first container; The first volume of the first container is determined based on the height of the first container and the first preset change trend; Based on the first volume, determine the height threshold; If the liquid level information reaches the height threshold, then the injection of liquid into the first container is stopped.

[0008] Optionally, determining the first volume of the first container based on the height of the first container and the first preset change trend includes: Acquire a second image, the second image including the bottom region of the first container; The second image is identified to obtain the bottom area of ​​the first container; The second volume of the first container is determined based on the height of the first container and the bottom area; Based on the predetermined relationship between the preset change trend, container height and volume correction value, a first volume correction value corresponding to the first preset change trend and the height of the first container is determined. The second volume is corrected based on the first volume correction value to obtain the first volume.

[0009] Optionally, determining the first volume of the first container based on the height of the first container and the first preset change trend includes: Based on the predetermined preset change trend and the correspondence between container height and container volume, the volume corresponding to the first preset change trend and the height of the first container is determined as the first volume.

[0010] Optionally, determining the ratio of the upper surface area of ​​the liquid to the first area of ​​the container opening based on the first image includes: The first image is processed using a semantic segmentation model to obtain the first area ratio. Optionally, the semantic segmentation model is trained in the following manner: The first neural network model is trained using the first sample dataset to obtain the teacher model. The first sample dataset includes a third image and a third area ratio of the upper surface of the liquid in the third container to the container opening of the third container at the acquisition time corresponding to the third image. The third image is an image frame of the second video, and the second video is a video of the container opening of the third container acquired during the process of injecting the liquid into the third container. The teacher model is used to train the student model using knowledge distillation, and the trained student model is then identified as the semantic segmentation model.

[0011] Optionally, the teacher model includes an encoder and a decoder, and the student model includes the encoder, convolutional layer, activation layer, downsampling layer, fusion layer and fully connected layer connected in sequence; The step of processing the first image using a semantic segmentation model to obtain the first area ratio includes: The first image is input into the encoder to obtain first feature maps at multiple scales; Each of the first feature maps is processed sequentially through the convolutional layer, the activation layer, and the downsampling layer to obtain a second feature map of a preset scale. The third feature map is obtained by connecting the various second feature maps using the fusion layer; The third feature map is input into the fully connected layer for processing to obtain the first area ratio.

[0012] Optionally, the step of using the teacher model to perform knowledge distillation training on the student model includes: Based on the third image and the third area ratio at the acquisition time corresponding to the third image, generate real training sample pairs; Acquire a fourth image, which is an image frame of the third video, which is a video of the opening of the fourth container captured during the process of injecting the liquid into the fourth container; The teacher model is used to process the fourth image to obtain the fourth area ratio of the upper surface of the liquid in the fourth container to the container opening of the fourth container at the acquisition time corresponding to the fourth image. Based on the fourth image and the fourth area ratio at the acquisition time corresponding to the fourth image, pseudo-training sample pairs are generated. The student model is trained using the real training sample pairs, the pseudo training sample pairs, and the teacher model through knowledge distillation.

[0013] According to a second aspect of the present disclosure, a liquid level detection device is provided, comprising: The first acquisition module is used to acquire a first image, wherein the first image is an image frame of a first video, and the first video is a video of the container opening of the first container captured during the process of injecting liquid into the first container. The first determining module is used to determine, based on the first image, a first area ratio between the upper surface of the liquid and the container opening; The second determining module is used to determine the liquid level information of the liquid in the first container based on the first area ratio.

[0014] According to a third aspect of the present disclosure, a liquid level detection device is provided, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions in the memory to implement the steps of the liquid level detection method provided in the first aspect of this disclosure.

[0015] According to a fourth aspect of the present disclosure, a water dispenser is provided, comprising: An image acquisition device is used to acquire video below the water outlet of the water dispenser, the video including a first video at the opening of the first container during the process of the water dispenser injecting liquid into a first container located below the water outlet. processor; Memory used to store processor-executable instructions; The processor is configured as follows: Acquire a first image, which is an image frame of a first video, and the first video is a video of the opening of the first container captured during the process of injecting liquid into the first container. Based on the first image, determine a first area ratio between the upper surface of the liquid and the opening of the container; Based on the first area ratio, the liquid level information of the liquid in the first container is determined.

[0016] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the liquid level detection method provided in the first aspect of the present disclosure.

[0017] According to a sixth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the liquid level detection method provided in the first aspect of the present disclosure.

[0018] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: A first image is acquired; based on the first image, a first area ratio between the upper surface of the liquid and the container opening is determined; based on the first area ratio, the liquid level information inside the first container is determined, wherein the first image is an image frame of a first video, and the first video is a video of the container opening of the first container captured during the process of injecting liquid into the first container. Thus, using the first area ratio obtained based on the first image to determine the liquid level information can improve the accuracy and reliability of the liquid level information; furthermore, this method also has the advantages of being non-contact and having strong applicability.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0021] Figure 1 This is a flowchart illustrating a liquid level detection method according to an exemplary embodiment.

[0022] Figure 2 This is a schematic diagram showing an image of the container opening taken during the process of injecting liquid into a container, according to an exemplary embodiment.

[0023] Figure 3 This is a flowchart illustrating a liquid level detection method according to an exemplary embodiment.

[0024] Figure 4 This is a flowchart illustrating a liquid level detection method according to an exemplary embodiment.

[0025] Figure 5 This is a schematic diagram illustrating a preset change trend of a second container according to an exemplary embodiment.

[0026] Figure 6 This is a flowchart illustrating a liquid level detection method according to an exemplary embodiment.

[0027] Figure 7 This is a schematic diagram of a second image according to an exemplary embodiment.

[0028] Figure 8 This is a schematic diagram illustrating a teacher model and a student model according to an exemplary embodiment.

[0029] Figure 9 This is a block diagram illustrating a liquid level detection device according to an exemplary embodiment.

[0030] Figure 10This is a block diagram illustrating a liquid level detection device according to an exemplary embodiment. Detailed Implementation

[0031] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0032] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0033] Figure 1 This is a flowchart illustrating a liquid level detection method according to an exemplary embodiment. This method can be used in water dispensers, for example, it can be applied to the MCU (Microcontroller Unit) of the water dispenser. Of course, the liquid level detection method disclosed herein can also be used in other electronic devices that require liquid level detection. Figure 1 As shown, the method may include steps S101 to S103.

[0034] In step S101, the first image is acquired.

[0035] The first image is an image frame of the first video, which is a video of the opening of the first container captured during the process of injecting liquid into the first container.

[0036] For example, a video below the water dispenser's spout can be captured using an image acquisition device pre-installed on the water dispenser. Thus, a first video can be captured by this image acquisition device. The video can be captured from the container opening to the bottom of the container. For instance, the video at the container opening can include the spout and / or the surrounding area. Each frame of the first video can be used as a separate first image, or frames from the first video spaced at preset intervals can be used as the first image; there is no limitation here.

[0037] In step S102, based on the first image, a first area ratio between the upper surface of the liquid and the container opening is determined.

[0038] For example, image segmentation techniques in image processing can be used to segment the upper surface of the liquid inside the first container and the opening of the first container in the first image. The segmentation result can be as follows: Figure 2 As shown, Figure 2In the segmentation, circles are used to represent the container opening, and irregular shapes are used to represent the liquid surface. The first area ratio can be determined based on the number of pixels in each part of the segmentation result; for example, it can be determined using the following formula. :

[0039] in, This represents the number of pixels on the upper surface of the liquid in the segmentation result of the current first image. This represents the number of pixels in the container opening in the segmentation result of the current first image.

[0040] In one embodiment, to improve the accuracy of the determined first area ratio, edge detection technology can also be used to perform edge distortion correction processing on the first image.

[0041] In step S103, the liquid level information of the liquid in the first container is determined based on the first area ratio.

[0042] For example, a correspondence between the liquid level information and the area ratio of the liquid in the first container can be preset, and a nonlinear regression model (Height Non-Linear Regression, HLR) can be constructed using this correspondence. In this way, the first area ratio can be processed using the nonlinear regression model to obtain the liquid level information corresponding to the current first area ratio.

[0043] In the above technical solution, a first image is acquired; based on the first image, a first area ratio between the upper surface of the liquid and the container opening is determined; based on the first area ratio, the liquid level information in the first container is determined. The first image is an image frame from a first video, which is a video taken during the process of injecting liquid into the first container at the container opening. Thus, using the first area ratio obtained from the first image to determine the liquid level information can improve the accuracy and reliability of the liquid level information; furthermore, this method also has the advantages of being non-contact and having strong applicability.

[0044] In an optional embodiment, in step S101, acquiring the first image includes: Each image frame in the first video is used as the first image; or an image frame located at a preset position in the first video is used as the first image.

[0045] By using each image frame in the first video as a first image, the continuity between the first images can be effectively improved, the dynamic process of injecting liquid into the first container can be captured in detail, and the accuracy of the information can be improved.

[0046] For example, the preset position can be pre-set based on actual needs. For instance, it can be set at equal intervals. After determining the first frame image in the first video as the first image, a first image frame can be determined at preset intervals of image frames. Redundant information may exist in consecutive image frames of the video. Using the image frame located at the preset position in the first video as the first image can remove redundant information, ensuring the overall quality and usability of the first image while reducing computational load and improving processing efficiency.

[0047] Figure 3 This is a flowchart illustrating a liquid level detection method according to an exemplary embodiment. Figure 3 As shown, step S103 may include step S1031.

[0048] In step S1031, the liquid level information is determined based on the changing trend of the first area ratio.

[0049] For example, a ratio sequence can be generated based on the ratio of the upper surface of the liquid in the first container to the first area of ​​the container opening at the acquisition time corresponding to each first image. The earlier the acquisition time corresponding to the first image, the earlier its corresponding first area ratio appears in the ratio sequence. As the length of the first video increases, the number of first images also increases, and the length of the ratio sequence also increases. Based on this ratio sequence, the trend of change in the first area ratio can be determined, for example, by using a function or a graph to represent this trend.

[0050] In one embodiment, it can be achieved through, as follows Figure 4 The steps S1032 and S1033 shown confirm the liquid level information based on the changing trend of the first area ratio.

[0051] In step S1032, a first preset change trend that matches the change trend of the first area ratio is determined from multiple preset change trends.

[0052] In step S1033, based on the first preset change trend, liquid level information corresponding to the first area ratio determined based on the most recent frame of the first image is determined as the liquid level information of the liquid in the first container.

[0053] The preset change trend is used to characterize the change trend of the ratio of the upper surface of the liquid in the second container to the second area of ​​the container opening as the liquid level information in the second container changes, and different preset change trends correspond to different second containers.

[0054] For example, several second containers of different shapes can be pre-selected. For each second container, liquid is poured into it at a uniform rate, and a corresponding video is recorded. For each video, images from the start of liquid pouring to the end of pouring are retained. Based on the ratio of the upper surface of the liquid in the second container to the container opening area at the acquisition time corresponding to each frame of the video, the preset trend of the second container's change in the video is generated using Occam's razor. Liquid level information can be represented as a percentage of liquid level, and the percentage of the liquid level in the video can be determined using the following formula. Liquid level percentage corresponding to the frame image :

[0055] in, For the first The number of frames in a frame image. The percentage of the maximum liquid level injected into this video. This represents the maximum number of frames in the video.

[0056] For each second container, multiple video clips of liquid being injected into it at a constant speed can be recorded, thereby generating a preset trend for that second container. This improves the reliability of the determined preset trend. Figure 5 This is a schematic diagram illustrating a predetermined variation trend of a second container according to an exemplary embodiment. Figure 5 Each point in the image can correspond to one frame. Figure 5 The straight line in the figure represents the preset trend of change of the second container.

[0057] For example, the similarity between trends can be determined by calculating Euclidean distance, cosine similarity, or Manhattan distance. The similarity between the trend of change of the first area ratio and each preset trend can be determined, and the preset trend with the highest similarity is identified as the first preset trend. Once the first preset trend is determined, the liquid level relationship corresponding to the first area ratio of the most recent frame of the first image can be determined using this first preset trend, and this relationship can be identified as the liquid level information in the first container. In this way, liquid level information can be determined simply, quickly, and accurately.

[0058] Figure 6 This is a flowchart illustrating a liquid level detection method according to an exemplary embodiment. Figure 6 As shown, the liquid level detection method provided in this disclosure may further include steps S201 to S204.

[0059] In step S201, the height of the first container is obtained.

[0060] For example, the height of the first container can be obtained by a height sensor pre-installed on the water dispenser. This height sensor can be an ultrasonic sensor, a laser sensor, or other similar sensors, and there are no restrictions here. Taking an ultrasonic sensor as an example, when the first container is placed on the container base of the water dispenser and before liquid is started being poured into the first container, the ultrasonic sensor can detect a first distance between the container and the bottom of the first container, and a second distance between the container and the opening of the first container. The difference between the first distance and the second distance can be determined as the height of the first container.

[0061] In step S202, the first volume of the first container is determined based on the height of the first container and the first preset change trend.

[0062] In one embodiment, the first volume of the first container can be determined in the following way: Based on a predetermined preset trend of change and the correspondence between container height and container volume, the volume corresponding to the first preset trend of change and the height of the first container is determined as the first volume.

[0063] As mentioned earlier, different preset trends correspond to different second containers, and the volume and height of the second container corresponding to each preset trend are fixed. Considering the existence of cylindrical containers with the same base area but different heights, to ensure the reliability of the obtained first volume, the correspondence between the preset trend, container height, and container volume can be preset. In this way, given the first preset trend and the height of the first container, the corresponding first volume can be obtained simply, quickly, and accurately.

[0064] In yet another embodiment, the first volume of the first container can be determined in the following manner: Obtain a second image, which includes the bottom region of the first container; The second image is identified to obtain the bottom area of ​​the first container; The second volume of the first container is determined based on its height and bottom area. Based on the predetermined preset change trend, the correspondence between container height and volume correction value, determine the first volume correction value corresponding to the first preset change trend and the height of the first container; The second volume is corrected based on the first volume correction value to obtain the first volume.

[0065] For example, the second image can be the first image frame in the first video that includes the bottom area of ​​the first container. If the first container is placed on the base of the water dispenser, the bottom area of ​​the first container can be identified by positioning marks set on the base. For example, the positioning marks can be standard-sized black and white alternating squares (e.g., each square is 5mm*5mm). When initializing the water dispenser, the image acquisition device can calculate its own intrinsic parameters by photographing the black and white alternating squares, thereby correcting image edge distortion. Figure 7 Taking the example of a circular bottom area of ​​the first container, the area of ​​the bottom area of ​​the first container can be determined based on the number of black and white squares, i.e., the bottom area of ​​the first container. Alternatively, the radius of the circle can be determined based on the number of white squares, and then the bottom area of ​​the first container can be determined.

[0066] For example, the product of the height and base area of ​​the first container can be used to determine the second volume of the first container. Considering that not all containers are cylindrical, to apply to containers of more shapes, a preset trend and a correspondence between container height and volume correction values ​​can be pre-set. Using this correspondence, based on the first preset trend and the height of the first container, the corresponding first volume correction value can be determined simply, quickly, and accurately. For example, the sum of the first volume correction value and the second volume can be used to determine the first volume. This yields a first volume with relatively high accuracy.

[0067] Turn back Figure 6 In step S203, the height threshold is determined based on the first volume.

[0068] For example, the height threshold corresponding to the first volume can be determined based on a predetermined correspondence between volume and height thresholds. Thus, determining the height threshold based on the volume of the first container allows the determined height threshold to be adapted to containers of different shapes, enabling more accurate control of the liquid injection process and preventing liquid overflow.

[0069] In step S204, if the liquid level information reaches the height threshold, the injection of liquid into the first container is stopped.

[0070] For example, if the liquid level information reaches the height threshold, it can be determined that continuing to inject liquid may cause the liquid to overflow. In this case, the injection of liquid into the first container can be stopped. Conversely, if the liquid level information does not reach the height threshold, liquid can continue to be injected into the first container and steps S101 to S103 can be repeated to achieve real-time monitoring of the liquid level information.

[0071] In this way, the liquid injection can be automatically stopped based on the first volume of the first container without the need for manual operation by the user. This improves the convenience of liquid access and avoids liquid overflow caused by untimely user operation, thus enhancing the intelligence of the device.

[0072] In an optional embodiment, before acquiring the first image, the liquid level detection method provided in this disclosure may further include: Obtain brightness information; if the brightness information is less than the brightness threshold, control the auxiliary light source to start.

[0073] For example, brightness information can be obtained through a photosensor. If the brightness information is less than a brightness threshold, it can be determined that the ambient light is insufficient. By controlling the activation of an auxiliary light source, the image acquisition unit can be assisted in acquiring a clear image. For instance, the brightness of the auxiliary light source can be controlled by a PWM (Pulse Width Modulation) square wave to keep the brightness information within a preset range, thereby assisting the image acquisition unit in acquiring a clear image while avoiding overexposure. If the brightness information is not less than the brightness threshold, step S101 can be executed.

[0074] In an optional embodiment, in step S102, determining the first area ratio of the upper surface of the liquid to the container opening based on the first image may include: The first image is processed using a semantic segmentation model to obtain the first area ratio.

[0075] The semantic segmentation model can be trained in the following way: The first neural network model is trained using the first sample dataset to obtain the teacher model; The teacher model is used to train the student model through knowledge distillation, and the trained student model is then identified as the semantic segmentation model.

[0076] The first sample dataset includes a third image and a third area ratio between the upper surface of the liquid in the third container and the opening of the third container at the acquisition time corresponding to the third image. The third image is an image frame from the second video, which is a video of the opening of the third container captured during the process of injecting liquid into the third container. The third image serves as the input to the first neural network model, and the corresponding third area ratio is the target output of the first neural network model. The teacher model is obtained by training the first neural network model to convergence using the first sample dataset. The third container can be one or more of the second containers mentioned above.

[0077] The teacher model possesses the ability to segment images semantically (both container and liquid surface). A student model can be built upon the backbone structure of the teacher model, and knowledge distillation can be performed to enable the student model to also acquire the ability to segment images semantically (both container and liquid surface). Thus, by using the teacher model to obtain the student model and defining the student model as a semantic model, it is possible to maintain high prediction accuracy while achieving model miniaturization and efficiency.

[0078] For example, the teacher model can be a Unet++ model, and its structure can be as follows: Figure 8 As shown in M1, the part within the dashed box is the encoder. The goodness of fit of the teacher model M1... The value is 0.653, indicating the goodness of fit of the student model M2. The value is 0.651, and the goodness of fit of the two is similar. However, the student model is simpler and has higher data processing efficiency. Therefore, the trained student model can be identified as the semantic segmentation model.

[0079] In one embodiment, model pruning and compression training methods can be used to further simplify the structure of the student model, thereby reducing the number of parameters and computational load.

[0080] In one embodiment, the teacher model includes an encoder and a decoder, and the student model includes an encoder, a convolutional layer, an activation layer, a downsampling layer, a fusion layer and a fully connected layer connected in sequence. The encoder in the student model is the same as the encoder in the teacher model.

[0081] The first area ratio can be obtained by processing the first image using a semantic segmentation model in the following way: The first image is input into the encoder to obtain first feature maps at multiple scales; Each first feature map is processed sequentially through a convolutional layer, an activation layer, and a downsampling layer to obtain a second feature map of a preset scale; The third feature map is obtained by connecting the various second feature maps using a fusion layer; The third feature map is input into the fully connected layer for processing to obtain the first area ratio.

[0082] For example, the upsampling layer in the decoder of the teacher model severely restricts inference speed. Therefore, the decoder of the teacher model can be omitted in the student model to improve data processing speed. In the student model, the encoder is used to extract features. The encoder can obtain multiple first feature maps of different scales. These first feature maps can be processed sequentially through convolutional layers, activation layers, and downsampling layers to obtain second feature maps with the same preset scale. Then, multiple second feature maps can be concatenated along the channel dimension in the fusion layer through the concat operation to obtain a third feature map. Finally, the third feature map can be input into the fully connected layer, and the fully connected layer outputs the first area ratio.

[0083] After processing the first image using a semantic segmentation model to obtain the first area ratio, this first area ratio can be input into a pre-constructed liquid level information determination model to obtain the liquid level information output by the model. The liquid level information determination model can be a nonlinear regression model built based on the mapping relationship between the first area ratio and the liquid level information. Thus, by sequentially using the semantic segmentation model and the liquid level information determination model, the liquid level information can be determined in real time and accurately, providing reliable data support for the control of equipment such as water dispensers.

[0084] In an optional embodiment, the student model can be trained using the teacher model through knowledge distillation in the following manner: Based on the third image and the third area ratio at the acquisition time corresponding to the third image, generate real training sample pairs; Acquire the fourth image, which is an image frame of the third video, which is a video of the opening of the fourth container captured during the process of injecting liquid into the fourth container; The teacher model is used to process the fourth image to obtain the fourth area ratio of the upper surface of the liquid in the fourth container to the container opening of the fourth container at the acquisition time corresponding to the fourth image. Based on the fourth image and the fourth area ratio at the acquisition time corresponding to the fourth image, pseudo training sample pairs are generated. The student model is trained using knowledge distillation with real training sample pairs, pseudo training sample pairs, and the teacher model.

[0085] For example, the fourth container can be one or more of the second containers described above. Typically, real training sample pairs are constructed manually, a time-consuming and labor-intensive process. To provide more training sample pairs for the student model, the trained teacher model can be used to process the fourth image to obtain the fourth area ratio at the corresponding acquisition time, thereby generating pseudo-training sample pairs. This allows for effective training of the student model using unlabeled data, thus improving the student model's performance.

[0086] When training the student model, cross-entropy loss and Dice loss can be used as loss functions. This can prevent poor model fitting due to imbalanced samples. The student model can calculate the gradient of the model parameters through backpropagation based on the loss function, and update the model parameters based on gradient descent algorithm with adaptive learning rate.

[0087] Figure 9 This is a block diagram illustrating a liquid level detection device according to an exemplary embodiment. (Refer to...) Figure 9 The device 300 includes an acquisition module 301, a first determination module 302, and a second determination module 303.

[0088] The first acquisition module 301 is used to acquire a first image, wherein the first image is an image frame of a first video, and the first video is a video of the container opening of the first container captured during the process of injecting liquid into the first container. The first determining module 302 is used to determine a first area ratio between the upper surface of the liquid and the container opening based on the first image. The second determining module 303 is used to determine the liquid level information of the liquid in the first container based on the first area ratio.

[0089] In the above technical solution, a first image is acquired; based on the first image, a first area ratio between the upper surface of the liquid and the container opening is determined; based on the first area ratio, the liquid level information in the first container is determined. The first image is an image frame from a first video, which is a video taken during the process of injecting liquid into the first container at the container opening. Thus, using the first area ratio obtained from the first image to determine the liquid level information can improve the accuracy and reliability of the liquid level information; furthermore, this method also has the advantages of being non-contact and having strong applicability.

[0090] Optionally, the acquisition module 301 is used to acquire the first image in the following manner: Each image frame in the first video is used as the first image; or an image frame located at a preset position in the first video is used as the first image. The second determining module 303 includes: The first determining submodule is used to determine the liquid level information based on the changing trend of the first area ratio.

[0091] Optionally, the first determining submodule includes: The second determining submodule is used to determine a first preset change trend that matches the change trend of the first area ratio among multiple preset change trends. The preset change trend is used to characterize the change trend of the second area ratio of the upper surface of the liquid in the second container to the container opening of the second container with the liquid level information of the liquid in the second container, and different preset change trends correspond to different second containers. The third determining submodule is used to determine, based on the first preset change trend, the liquid level information corresponding to the first area ratio determined based on the most recent frame of the first image, as the liquid level information of the liquid in the first container.

[0092] Optionally, the device 300 further includes: The second acquisition module is used to acquire the height of the first container; The third determining module is used to determine the first volume of the first container based on the height of the first container and the first preset change trend; The fourth determining module is used to determine a height threshold based on the first volume; The control module is configured to stop injecting the liquid into the first container if the liquid level information reaches the height threshold.

[0093] Optionally, the third determining module includes: The fourth determining submodule is used to acquire a second image, the second image including the bottom area of ​​the first container; to identify the second image to obtain the bottom area of ​​the first container; to determine the second volume of the first container based on the height of the first container and the bottom area; to determine a first volume correction value corresponding to the first preset change trend and the height of the first container based on the predetermined correspondence between the preset change trend and the container height and the volume correction value; and to correct the second volume based on the first volume correction value to obtain the first volume.

[0094] Optionally, the third determining module includes: The fifth determining submodule is used to determine the volume corresponding to the first preset change trend and the height of the first container, based on the predetermined correspondence between the preset change trend and the container height and the container volume, and use it as the first volume.

[0095] Optionally, the first determining module 302 includes: The sixth determining submodule is used to process the first image using a semantic segmentation model to obtain the first area ratio.

[0096] Optionally, the semantic segmentation model is trained in the following manner: The first neural network model is trained using the first sample dataset to obtain the teacher model. The first sample dataset includes a third image and a third area ratio of the upper surface of the liquid in the third container to the container opening of the third container at the acquisition time corresponding to the third image. The third image is an image frame of the second video, and the second video is a video of the container opening of the third container acquired during the process of injecting the liquid into the third container. The teacher model is used to train the student model using knowledge distillation, and the trained student model is then identified as the semantic segmentation model.

[0097] Optionally, the teacher model includes an encoder and a decoder, and the student model includes the encoder, convolutional layer, activation layer, downsampling layer, fusion layer and fully connected layer connected in sequence; The sixth determining submodule is used to obtain the first area ratio in the following manner: The first image is input into the encoder to obtain first feature maps at multiple scales; Each of the first feature maps is processed sequentially through the convolutional layer, the activation layer, and the downsampling layer to obtain a second feature map of a preset scale. The third feature map is obtained by connecting the various second feature maps using the fusion layer; The third feature map is input into the fully connected layer for processing to obtain the first area ratio.

[0098] Optionally, the student model is trained using the teacher model through knowledge distillation, including: Based on the third image and the third area ratio at the acquisition time corresponding to the third image, generate real training sample pairs; Acquire a fourth image, which is an image frame of the third video, which is a video of the opening of the fourth container captured during the process of injecting the liquid into the fourth container; The teacher model is used to process the fourth image to obtain the fourth area ratio of the upper surface of the liquid in the fourth container to the container opening of the fourth container at the acquisition time corresponding to the fourth image. Based on the fourth image and the fourth area ratio at the acquisition time corresponding to the fourth image, pseudo-training sample pairs are generated. The student model is trained using the real training sample pairs, the pseudo training sample pairs, and the teacher model through knowledge distillation.

[0099] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0100] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the liquid level detection method provided in this disclosure.

[0101] Figure 10 This is a block diagram illustrating a liquid level detection device 800 according to an exemplary embodiment. (Refer to...) Figure 8 The device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output interface 812, a sensor component 814, and a communication component 816.

[0102] Processing component 802 typically controls the overall operation of device 800, including operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the liquid level detection method described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0103] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of this data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0104] Power supply component 806 provides power to various components of device 800. Power supply component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to device 800.

[0105] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0106] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0107] Input / output interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0108] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0109] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0110] In an exemplary embodiment, the device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the liquid level detection method described above.

[0111] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to complete the liquid level detection method described above. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0112] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the liquid level detection method described above when executed by the programmable device.

[0113] In another exemplary embodiment, a water dispenser is also provided, comprising: An image acquisition device is used to acquire video below the water outlet of the water dispenser, the video including a first video at the opening of the first container during the process of the water dispenser injecting liquid into a first container located below the water outlet. processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions in the memory to implement the steps of the liquid level detection method provided in any of the embodiments above.

[0114] Optionally, the processor is further configured to stop injecting the liquid into the first container if the liquid level information reaches a height threshold.

[0115] Optionally, the water dispenser is also equipped with a height sensor to detect the height of the first container.

[0116] Optionally, the water dispenser includes a base for placing the first container, the base having a positioning mark for being identified by the processor to determine the bottom area of ​​the first container.

[0117] Those skilled in the art will also understand that the various illustrative logical blocks and steps listed in the embodiments of this application can be implemented by electronic hardware, computer software, or a combination of both. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functionality using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of this application.

[0118] In the above detailed description, reference has been made to the accompanying drawings, which illustrate specific aspects of how this disclosure can be practiced. In this regard, terms indicating direction or positional relationship, such as “up,” “down,” “front,” and “back,” can be used with reference to the orientation of the described figures. Since components of the described device can be positioned in several different orientations, these directional terms are for illustrative purposes and not for limitation. It should be understood that other aspects can be utilized and structural or logical changes can be made without departing from the concept of this disclosure. Therefore, the following detailed description should not be considered limiting.

[0119] It should be understood that, unless otherwise specifically indicated, features of various embodiments of this disclosure described herein can be combined with each other. As used herein, the term “and / or” includes any one of the relevant listed items and any combination of any two or more; similarly, “at least one of…” includes any one of the relevant listed items and any combination of any two or more.

[0120] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, parts, regions, layers, or sections, these components, parts, regions, layers, or sections are not limited to these terms. Rather, these terms are used only to distinguish one component, part, region, layer, or section from another. Therefore, without departing from the teachings of the examples described herein, the first component, part, region, layer, or section mentioned in the examples may also be referred to as the second component, part, region, layer, or section. Furthermore, the terms “first” and “second” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as “first” or “second” may explicitly or implicitly include at least one of that feature. In the description herein, “a plurality” means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0121] Furthermore, the term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term “exemplary” is intended to present the concept in a concrete manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from the context, “X applies A or B” is intended to mean any of the natural inclusive arrangements. That is, “X applies A or B” satisfies any of the foregoing instances if X applies A; X applies B; or both X applies A and B. Additionally, unless otherwise specified or clear from the context to refer to the singular form, the articles “a” and “an” as used in this application and the appended claims are generally understood to mean “one or more.”

[0122] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding this specification and the accompanying drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term “including.”

[0123] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

[0124] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A liquid level detection method, characterized in that, include: Acquire a first image, which is an image frame of a first video, and the first video is a video of the opening of the first container captured during the process of injecting liquid into the first container. Based on the first image, determine a first area ratio between the upper surface of the liquid and the opening of the container; Based on the first area ratio, the liquid level information of the liquid in the first container is determined.

2. The liquid level detection method according to claim 1, characterized in that, The acquisition of the first image includes: Each image frame of the first video is taken as the first image; or The image frame located at a preset position in the first video is taken as the first image; Determining the liquid level information of the liquid in the first container based on the first area ratio includes: The liquid level information is determined based on the changing trend of the first area ratio.

3. The liquid level detection method according to claim 2, characterized in that, Determining the liquid level information based on the changing trend of the first area ratio includes: Among multiple preset change trends, a first preset change trend that matches the change trend of the first area ratio is determined, wherein the preset change trend is used to characterize the change trend of the second area ratio of the upper surface of the liquid in the second container to the container opening of the second container with the liquid level information of the liquid in the second container, and different preset change trends correspond to different second containers. Based on the first preset change trend, liquid level information corresponding to the first area ratio determined based on the most recent frame of the first image is determined as the liquid level information of the liquid in the first container.

4. The liquid level detection method according to claim 3, characterized in that, The liquid level detection method further includes: Get the height of the first container; The first volume of the first container is determined based on the height of the first container and the first preset change trend; Based on the first volume, determine the height threshold; If the liquid level information reaches the height threshold, then the injection of liquid into the first container is stopped.

5. The liquid level detection method according to claim 4, characterized in that, Determining the first volume of the first container based on its height and the preset change trend includes: Acquire a second image, the second image including the bottom region of the first container; The second image is identified to obtain the bottom area of ​​the first container; The second volume of the first container is determined based on the height of the first container and the bottom area; Based on the predetermined relationship between the preset change trend, container height and volume correction value, a first volume correction value corresponding to the first preset change trend and the height of the first container is determined. The second volume is corrected based on the first volume correction value to obtain the first volume.

6. The liquid level detection method according to claim 4, characterized in that, Determining the first volume of the first container based on its height and the preset change trend includes: Based on the predetermined preset change trend and the correspondence between container height and container volume, the volume corresponding to the first preset change trend and the height of the first container is determined as the first volume.

7. The liquid level detection method according to claim 1, characterized in that, Determining the ratio of the upper surface area of ​​the liquid to the first area of ​​the container opening based on the first image includes: The first image is processed using a semantic segmentation model to obtain the first area ratio.

8. The liquid level detection method according to claim 7, characterized in that, The semantic segmentation model is trained in the following way: The first neural network model is trained using the first sample dataset to obtain the teacher model. The first sample dataset includes a third image and a third area ratio of the upper surface of the liquid in the third container to the container opening of the third container at the acquisition time corresponding to the third image. The third image is an image frame of the second video, and the second video is a video of the container opening of the third container acquired during the process of injecting the liquid into the third container. The teacher model is used to train the student model through knowledge distillation, and the trained student model is determined as the semantic segmentation model.

9. The liquid level detection method according to claim 8, characterized in that, The teacher model includes an encoder and a decoder, and the student model includes the encoder, convolutional layer, activation layer, downsampling layer, fusion layer and fully connected layer connected in sequence. The step of processing the first image using a semantic segmentation model to obtain the first area ratio includes: The first image is input into the encoder to obtain first feature maps at multiple scales; Each of the first feature maps is processed sequentially through the convolutional layer, the activation layer, and the downsampling layer to obtain a second feature map of a preset scale. The third feature map is obtained by connecting the various second feature maps using the fusion layer; The third feature map is input into the fully connected layer for processing to obtain the first area ratio.

10. The liquid level detection method according to claim 8, characterized in that, The process of using the teacher model to perform knowledge distillation training on the student model includes: Based on the third image and the third area ratio at the acquisition time corresponding to the third image, generate real training sample pairs; Acquire a fourth image, which is an image frame of the third video, which is a video of the opening of the fourth container captured during the process of injecting the liquid into the fourth container; The teacher model is used to process the fourth image to obtain the fourth area ratio of the upper surface of the liquid in the fourth container to the container opening of the fourth container at the acquisition time corresponding to the fourth image. Based on the fourth image and the fourth area ratio at the acquisition time corresponding to the fourth image, pseudo-training sample pairs are generated. The student model is trained using the real training sample pairs, the pseudo training sample pairs, and the teacher model through knowledge distillation.

11. A liquid level detection device, characterized in that, include: The first acquisition module is used to acquire a first image, wherein the first image is an image frame of a first video, and the first video is a video of the container opening of the first container captured during the process of injecting liquid into the first container. The first determining module is used to determine, based on the first image, a first area ratio between the upper surface of the liquid and the container opening; The second determining module is used to determine the liquid level information of the liquid in the first container based on the first area ratio.

12. A liquid level detection device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions in the memory to implement the steps of the liquid level detection method according to any one of claims 1-10.

13. A water dispenser, characterized in that, include: An image acquisition device is used to acquire video below the water outlet of the water dispenser, the video including a first video at the opening of the first container during the process of the water dispenser injecting liquid into the first container located below the water outlet. processor; Memory used to store processor-executable instructions; The processor is configured as follows: Acquire a first image, which is an image frame of a first video, and the first video is a video of the opening of the first container captured during the process of injecting liquid into the first container. Based on the first image, determine a first area ratio between the upper surface of the liquid and the opening of the container; Based on the first area ratio, the liquid level information of the liquid in the first container is determined.

14. The water dispenser according to claim 13, characterized in that, The processor is also configured to: If the liquid level information reaches a height threshold, then the injection of liquid into the first container is stopped.

15. The water dispenser according to claim 13, characterized in that, The water dispenser is also equipped with a height sensor to detect the height of the first container.

16. The water dispenser according to claim 13, characterized in that, The water dispenser includes a base for placing the first container, and the base is provided with a positioning mark, which is used by the processor to identify the bottom area of ​​the first container.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the liquid level detection method according to any one of claims 1-10.

18. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the steps of the liquid level detection method according to any one of claims 1-10.

Images