Method and device for measuring volume of liquid in container, storage medium and electronic equipment

Abstract

The invention relates to a method and device for measuring the volume of liquid in a container, a storage medium and electronic equipment. The method comprises the following steps: inputting a to-be-detected picture into a target detection model for container identification processing to obtain a container identification result; under the condition that the container identification result indicates that the target container is detected in the to-be-detected picture, cutting out the target container from the to-be-detected picture based on the container identification result to obtain a container picture; inputting the container picture into a semantic segmentation model for liquid region segmentation to obtain a target liquid region; acquiring a first height of the target container based on the container picture; acquiring a second height of the liquid level in the target container based on the target liquid area; and calculating the liquid volume of the to-be-detected liquid in the target container according to the first height, the second height and a preset container volume corresponding to the target container. The technical problem that the volume content in a container without scales cannot be measured is solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, storage medium and electronic device for measuring the volume of liquid in a container. Background Art

[0002] In many situations in life, it is necessary to obtain the volume of objects in a container. In order to solve this problem, visual technology can be used to detect the volume of objects in the container and the change of the volume in real time through mobile phone photography or other cameras. However, currently mobile phone photography or other camera detection can directly display the volume content of objects in containers with scales, but it is currently impossible to measure the volume content of objects in containers without scales. Summary of the invention

[0003] The present application provides a method, device, storage medium and electronic device for measuring the volume of liquid in a container to solve the technical problem that the volume content in a container without a scale cannot be measured.

[0004] In a first aspect, the present application provides a method for measuring the volume of liquid in a container, comprising: inputting a picture to be detected into a target detection model for container recognition processing to obtain a container recognition result; when the container recognition result indicates that a target container is detected in the picture to be detected, based on the container recognition result, cropping the target container from the picture to be detected to obtain a container picture; inputting the container picture into a semantic segmentation model for liquid region segmentation to obtain a target liquid region; based on the container picture, obtaining a first height of the target container; based on the target liquid region, obtaining a second height of the liquid level in the target container; and calculating the liquid volume of the liquid to be detected in the target container according to the first height, the second height and a preset container volume corresponding to the target container.

[0005] In a second aspect, the present application provides a device for measuring the volume of liquid in a container, comprising: an identification module, used to input a picture to be detected into a target detection model for container identification processing to obtain a container identification result; a cropping module, used to crop the target container from the picture to be detected based on the container identification result when the container identification result indicates that the target container is detected in the picture to be detected to obtain a container picture; a segmentation module, used to input the container picture into a semantic segmentation model for liquid area segmentation to obtain a target liquid area; a first acquisition module, used to obtain a first height of the target container based on the container picture; a second acquisition module, used to obtain a second height of the liquid level in the target container based on the target liquid area; and a calculation module, used to calculate the liquid volume of the liquid to be detected in the target container based on the first height, the second height and a preset container volume corresponding to the target container.

[0006] As an optional example, the target detection model includes a feature extraction layer and a target detection layer, and the recognition module includes: a first extraction unit, used to input the image to be detected into the feature extraction layer for feature extraction, so as to obtain a feature map of the image to be detected; and a recognition unit, used to input the feature map into the target detection layer for container recognition, so as to obtain the container recognition result, wherein the container recognition result includes detecting the target container in the image to be detected and not detecting the target container in the image to be detected.

[0007] As an optional example, the feature extraction layer includes a slicing layer, a convolution layer, a pooling layer and a fusion layer, and the first extraction unit includes: a slicing subunit, used to input the image to be detected into the slicing layer, perform a slicing operation on the image to be detected, and obtain multiple first feature maps, wherein the resolution of the image to be detected is higher than the resolution of the first feature map; a first convolution subunit, used to input the multiple first feature maps into the convolution layer, perform a convolution operation on the multiple first feature maps, and obtain multiple second feature maps; a pooling subunit, used to input the multiple second feature maps into the pooling layer, perform feature extraction on the multiple second feature maps at multiple different scales, and obtain multiple third feature maps; a first fusion subunit, used to input the multiple third feature maps into the fusion layer, fuse the multiple third feature maps, and obtain the feature map of the image to be detected.

[0008] As an optional example, the target detection layer includes a multi-scale fusion layer and a detection layer, and the recognition unit includes: a second fusion subunit, used to input the feature map into the multi-scale fusion layer, perform a multi-scale fusion operation on the feature map, and obtain a fourth feature map; a second convolution subunit, used to input the fourth feature map into the detection layer, perform a multi-layer convolution operation on the fourth feature map, and generate the container recognition result.

[0009] As an optional example, the semantic segmentation model includes a deep feature processing layer, a shallow feature processing layer and a feature fusion layer, and the segmentation module includes: a second extraction unit, used to input the container image into the deep feature processing layer, perform deep feature extraction on the liquid area, and obtain a deep feature map; a third extraction unit, used to input the container image into the shallow feature processing layer, perform shallow feature extraction on the liquid area, and obtain a shallow feature map; a processing unit, used to input the deep feature map and the shallow feature map into the feature fusion layer to obtain the target liquid area.

[0010] As an optional example, there are multiple target liquid areas, and the second acquisition module includes: a first calculation unit, used to calculate the area of ​​each of the target liquid areas; a first determination unit, used to determine the maximum area from the areas of all the target liquid areas; and a second determination unit, used to determine the height of the target liquid area with the largest area as the second height.

[0011] As an optional example, the above-mentioned calculation module includes: a second calculation unit, used to calculate the ratio of the above-mentioned second height to the above-mentioned first height, and obtain the volume ratio of the above-mentioned liquid to be detected and the above-mentioned target container; a third calculation unit, used to calculate the product of the above-mentioned volume ratio and the above-mentioned preset container volume, and obtain the liquid volume of the above-mentioned liquid to be detected.

[0012] In a third aspect, the present application provides a storage medium storing a computer program, wherein the computer program executes the above-mentioned method for measuring the volume of liquid in a container when executed by a processor.

[0013] In a fourth aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-mentioned method for measuring the volume of liquid in a container through the computer program.

[0014] In an embodiment of the present application, the image to be detected is input into a target detection model for container recognition processing to obtain a container recognition result; when the container recognition result indicates that the target container is detected in the image to be detected, the target container is cropped from the image to be detected based on the container recognition result to obtain a container image; the container image is input into a semantic segmentation model for liquid region segmentation to obtain a target liquid region; based on the container image, a first height of the target container is obtained; based on the target liquid region, a second height of the liquid level in the target container is obtained; and the volume of the target container in the target container is calculated according to the first height, the second height and a preset container volume corresponding to the target container. A method for detecting the liquid volume of a liquid, since in the above method, a container recognition result is identified on a target container to be detected where the liquid to be detected is located by a trained target detection model, and then a container image is cut out from the image to be detected according to the container recognition result, and then a target liquid area of ​​the liquid to be detected is segmented from the container image by a trained semantic segmentation model, and finally the liquid volume of the object to be detected in the container is calculated according to a first height of the target container in the container image, a second height of the liquid level in the target container, and a preset container volume corresponding to the target container, thereby achieving the purpose of photographing and detecting the liquid volume in a transparent container without a scale, and further solving the technical problem of being unable to measure the volume content in a container without a scale. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0017] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0018] Figure 1 is a flow chart of an optional method for measuring the volume of liquid in a container according to an embodiment of the present application;

[0019] Figure 2 is a schematic diagram of a target image of an optional method for measuring the volume of liquid in a container according to an embodiment of the present application;

[0020] Figure 3 is a contour diagram of a liquid to be detected according to an optional method for measuring the volume of liquid in a container according to an embodiment of the present application;

[0021] Figure 4 is a schematic structural diagram of an optional device for measuring the volume of liquid in a container according to an embodiment of the present application;

[0022] Figure 5 is a schematic diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0024] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0025] According to a first aspect of an embodiment of the present application, a method for measuring the volume of liquid in a container is provided. Optionally, as follows: Figure 1 As shown, the above method includes:

[0026] S102, inputting the image to be detected into the target detection model for container recognition processing to obtain a container recognition result;

[0027] S104, when the container recognition result indicates that the target container is detected in the image to be detected, based on the container recognition result, the target container is cropped from the image to be detected to obtain a container image;

[0028] S106, inputting the container image into a semantic segmentation model to perform liquid region segmentation to obtain a target liquid region;

[0029] S108, obtaining a first height of the target container based on the container image;

[0030] S110, obtaining a second height of the liquid level in the target container based on the target liquid area;

[0031] S112, calculating the liquid volume of the liquid to be detected in the target container according to the first height, the second height, and a preset container volume corresponding to the target container.

[0032] Optionally, in this embodiment, the target detection model can be YOLOv5 (You Only Look Once, version 5), which is a deep learning model for target detection tasks, capable of detecting and identifying multiple targets in an image, while providing the category and position of each target (usually represented by a bounding box). YOLOv5 has a very fast reasoning speed while maintaining high accuracy. The model parameters of YOLOv5 are relatively small, and the memory and computing resources occupied are low, which is suitable for running on resource-constrained devices, such as mobile devices and embedded systems. YOLOv5 provides a wealth of pre-trained models that can be easily fine-tuned on various data sets. Its code structure is clear and easy to modify and expand. YOLOv5 uses a multi-scale feature pyramid structure, which can effectively detect targets of different sizes. The semantic segmentation model can be DeepLabv3+, which can improve the accuracy and efficiency in semantic segmentation tasks. The goal of semantic segmentation is to classify each pixel in an image into a specific category, and DeepLabv3+ adopts an encoder-decoder structure. The encoder extracts the features of the image, and the decoder upsamples these features to restore the spatial resolution of the image, thereby providing more accurate boundary information. By adjusting the expansion rate of the convolution kernel, the receptive field can be expanded without increasing the amount of computation. This helps capture more contextual information. The Atrous Spatial Pyramid Pooling (ASPP) module is an important part of DeepLabv3+. The ASPP module uses atrous convolutions of different scales and global average pooling to process feature maps in a multi-scale manner, thereby improving the model's ability to detect objects of different scales.

[0033] Optionally, in this embodiment, the method can be applied to smart devices such as mobile phones, tablet computers, and smart watches. The image to be detected of the container where the liquid to be detected is located can be a picture taken by a camera of a smart device, or a picture obtained by intercepting a video taken by a camera of a smart device, or a picture locally saved by the smart device. The container where the liquid to be detected is located can be a transparent container without scale lines and with regular shapes.

[0034] Optionally, in this embodiment, before detecting the liquid volume of the liquid to be detected in the container in the image to be detected, the target detection model and the semantic segmentation model are trained in advance, so that the trained target detection model can accurately and quickly mark the outer frame of the container on the image to be detected, and the trained target detection model can accurately and quickly segment the outline of the object to be detected from the container image, thereby improving the accuracy and efficiency of measuring the volume content in containers without scales.

[0035] Optionally, in this embodiment, after completing the training of the target detection model, in order for the target detection model to be deployed on different platforms (such as Android and iOS). The model needs to be converted into a format suitable for these platforms, such as TensorFlow Lite (TFLite) or Torch Script. First, it needs to be converted into ONNX format. ONNX (Open NeuralNetwork Exchange) is an open source format for interoperability of AI models across different frameworks. Tensor Flow Lite (TFLite) is a lightweight version of Tensor Flow, designed for mobile and embedded devices, which converts the ONNX model to TFLite format to make it suitable for deployment on Android and iOS devices. Torch Script is an intermediate representation of PyTorch, which can convert the PyTorch model into the Torch Script format so that it can run without Python dependencies. Through format conversion, the trained YOLOv5 target detection model can be successfully deployed on mobile devices to achieve real-time target detection and processing.

[0036] Optionally, in this embodiment, after obtaining the image to be detected including the container where the liquid to be detected is located, the trained target detection model and the trained semantic segmentation model, as shown in FIG. Figure 2 The figure shows a schematic diagram of a picture to be detected including a container where the liquid to be detected is located. The picture to be detected is input into a trained target detection model, which can identify the outer frame of the container in the picture to be detected. Opencv (Open Source Computer Vision Library, an open source computer vision library that provides a wealth of image processing and computer vision functions) is used to crop the container part from the picture to be detected using the outer frame coordinates of the container output by the target detection model to obtain a container picture. The cropped container picture is input into a trained semantic segmentation model, which can segment the outline of the liquid to be detected from the container picture, as shown in FIG. Figure 3 The figure shows the outline of the liquid to be detected. The first height of the container image and the second height of the liquid level of the liquid to be detected, as well as the preset container volume corresponding to the known target container, are obtained respectively. The liquid volume of the liquid to be detected in the target container is calculated by proportional relationship using the obtained first height, second height and preset container volume. Through the above steps, the liquid to be detected can be accurately detected and segmented from the image to be detected, and its actual liquid volume can be calculated. The target detection model and semantic segmentation model are used to ensure high accuracy and efficiency.

[0037] Optionally, in this embodiment, a container recognition result is identified on the target container to be detected where the liquid to be detected is located through the trained target detection model, and then a container image is cut out from the image to be detected according to the container recognition result, and then a target liquid area of ​​the liquid to be detected is segmented from the container image through the trained semantic segmentation model, and finally the liquid volume of the object to be detected in the container is calculated according to the first height of the target container in the container image, the second height of the liquid level in the target container, and the preset container volume corresponding to the target container, thereby achieving the purpose of photographing and detecting the liquid volume in a transparent container without a scale, and further solving the technical problem of being unable to measure the volume content in a container without a scale.

[0038] As an optional example, the target detection model includes a feature extraction layer and a target detection layer. The image to be detected is input into the target detection model for container recognition processing. The obtained container recognition results include:

[0039] Input the image to be detected into the feature extraction layer for feature extraction to obtain a feature map of the image to be detected;

[0040] The feature map is input into the target detection layer for container recognition to obtain a container recognition result, wherein the container recognition result includes detecting the target container in the image to be detected and not detecting the target container in the image to be detected.

[0041] Optionally, in this embodiment, the image to be detected is first input into the Backbone network of the trained target detection model. The Backbone network is the backbone network of the target detection model, which is used to extract the features of the input image. The Backbone network includes the following modules: Focus module, CSP1_X module and SPP (Spatial Pyramid Pooling) module. The image to be detected is first passed into the Focus module, which splits the high-resolution image to be detected into multiple low-resolution feature maps through slicing operations. Then, these low-resolution feature maps are convoluted to pull them up to a high-dimensional space, and the feature representation is enriched by increasing the number of channels, so that subsequent modules can process and extract features more effectively. The feature map processed by the Focus module is passed to the CSP1_X module, which uses convolution and residual structures to further extract richer features and enhance the back propagation gradient values ​​between layers. Specifically, the input feature map is convolved to extract local features, and residual connections are introduced to enable the feature map to propagate more stably in the deep network to avoid the gradient disappearance problem. The input feature map is divided into two parts, one part is directly passed to the next layer, and the other part is processed by convolution and then fused with the previous part of the feature map to obtain multiple feature maps. This structure enhances the feature extraction capability and reduces the amount of calculation. Then, these feature maps are passed to the SPP module, which performs pooling operations at different scales to extract multi-scale features. Specifically, different sizes of pooling kernels (for example, 1x1, 3x3, 5x5) are used to perform pooling operations on the feature map to extract features of different scales and obtain multiple feature maps. The pooling features of different scales are fused to enhance the model's perception of objects of different sizes. After these processes, the Backbone network completes the feature extraction of the input image. The extracted feature map contains rich spatial and semantic information, providing a solid foundation for subsequent target detection. The feature map output by the Backbone network is passed to the Neck network. The main task of the Neck network is to perform feature fusion and enhancement so that the model can better detect targets. The Neck network mainly includes: PANet module and YOLOv5 head. The PANet module (Path Aggregation Network) enhances the model's detection ability for targets of different scales through multi-scale fusion operations. The feature maps of different layers are upsampled and downsampled so that features of different scales can be fused. By fusing feature maps from different levels, the model's recognition ability for targets of different sizes is enhanced. The feature map processed by the PANet module is passed to the YOLOv5 head. This part finally outputs the container recognition result through multiple convolution operations and comparison with the Ground Truth frame, and marks the outer frame of the target container where the liquid to be tested is located in the image to be detected to obtain the container outer frame map.

[0042] Optionally, in this embodiment, the Focus, CSP1_X and SPP modules in the Backbone network are used to gradually extract and enhance image features, and then the PANet module in the Neck network is used to perform multi-scale feature fusion, and finally the YOLOv5 head is used to perform target detection. Ultimately, the target detection model can accurately mark the outer frame of the target container where the liquid to be detected is located in the image to be detected, so that the container image of the target container can be cropped.

[0043] As an optional example, the feature extraction layer includes a slicing layer, a convolution layer, a pooling layer, and a fusion layer. The image to be detected is input into the feature extraction layer, and the feature map of the image to be detected is obtained, including:

[0044] Inputting the image to be detected into the slicing layer, performing a slicing operation on the image to be detected, and obtaining a plurality of first feature maps, wherein the resolution of the image to be detected is higher than the resolution of the first feature map;

[0045] Inputting the multiple first feature maps into a convolution layer, performing a convolution operation on the multiple first feature maps, and obtaining multiple second feature maps;

[0046] Inputting the plurality of second feature maps into a pooling layer, performing feature extraction on the plurality of second feature maps at a plurality of different scales, and obtaining a plurality of third feature maps;

[0047] The multiple third feature maps are input into the fusion layer, and the multiple third feature maps are fused to obtain the feature map of the image to be detected.

[0048] Optionally, in this embodiment, the image to be detected is first input into the Backbone network of the trained target detection model. The Backbone network is the backbone network of the target detection model, which is used to extract the features of the input image. The Backbone network includes the following modules: Focus module, CSP1_X module and SPP (Spatial Pyramid Pooling) module. The image to be detected is first passed into the Focus module, which splits the high-resolution image to be detected into multiple low-resolution feature maps through slicing operations. Then, these low-resolution feature maps are convoluted to pull them up to a high-dimensional space, and the feature representation is enriched by increasing the number of channels, so that subsequent modules can process and extract features more effectively. The feature map processed by the Focus module is passed to the CSP1_X module, which uses convolution and residual structures to further extract richer features and enhance the back-propagation gradient values ​​between layers. Specifically, the input feature map is convolved to extract local features and residual connections are introduced to enable the feature map to propagate more stably in the deep network to avoid the gradient disappearance problem. The input feature map is divided into two parts, one part is directly passed to the next layer, and the other part is processed by convolution and then fused with the previous part of the feature map to obtain multiple feature maps. This structure enhances the feature extraction capability and reduces the amount of calculation.

[0049] As an optional example, the target detection layer includes a multi-scale fusion layer and a detection layer. The feature map is input into the target detection layer, and the container recognition result obtained includes:

[0050] Input the feature map to the multi-scale fusion layer, perform a multi-scale fusion operation on the feature map, and obtain a fourth feature map;

[0051] The fourth feature map is input into the detection layer, a multi-layer convolution operation is performed on the fourth feature map, and a container recognition result is generated.

[0052] Optionally, in this embodiment, the feature map is passed to the SPP module, which performs pooling operations at different scales to extract multi-scale features. Specifically, different sizes of pooling kernels (for example, 1x1, 3x3, 5x5) are used to perform pooling operations on the feature map to extract features of different scales, obtain multiple feature maps, and fuse the pooling features of different scales to enhance the model's perception of objects of different sizes. After these processes, the Backbone network completes the feature extraction of the input image. The extracted feature map contains rich spatial and semantic information, providing a solid foundation for subsequent target detection. The feature map output by the Backbone network is passed to the Neck network. The main task of the Neck network is to perform feature fusion and enhancement so that the model can better perform target detection. The Neck network mainly includes: a PANet module and a YOLOv5 head. The PANet module (Path Aggregation Network) enhances the model's detection ability for targets of different scales through multi-scale fusion operations. The feature maps of different layers are upsampled and downsampled so that features of different scales can be fused. By fusing feature maps from different levels, the model's recognition ability for targets of different sizes is enhanced. The feature map processed by the PANet module is passed to the YOLOv5 head. This part finally outputs the container recognition result through multiple convolution operations and comparison with the Ground Truth frame. The outer frame of the target container where the tested liquid is located is marked in the image to be detected to obtain the container outer frame image.

[0053] As an optional example, the semantic segmentation model includes a deep feature processing layer, a shallow feature processing layer, and a feature fusion layer. The container image is input into the semantic segmentation model for liquid area segmentation, and the target liquid area includes:

[0054] Input the container image into the deep feature processing layer, perform deep feature extraction on the liquid area, and obtain a deep feature map;

[0055] Input the container image into the shallow feature processing layer, perform shallow feature extraction on the liquid area, and obtain a shallow feature map;

[0056] The deep feature map and the shallow feature map are input into the feature fusion layer to obtain the target liquid area.

[0057] Optionally, in this embodiment, since the pictures to be detected can be pictures taken by different cameras, these pictures may have slight differences in resolution, white balance, color processing and picture clarity. Therefore, the robustness of the semantic segmentation model is improved by data enhancement technology, so that it can better process pictures taken by different cameras. Specifically, the brightness and contrast of the pictures to be detected are changed so that the semantic segmentation model can adapt to images under different lighting conditions, and random noise is added to the pictures to be detected to enhance the robustness of the semantic segmentation model to image noise. The semantic segmentation model is divided into two parts, an encoder and a decoder. The main task of the encoder part is to extract the features of the image. First, the enhanced container picture is input into the encoder part, which includes a deep convolutional neural network. In the encoder, two effective feature layers are obtained: a deeper deep feature map and a shallower shallow feature map. In the deep feature map, convolutions with different expansion rates are used to expand the receptive field, capture contextual information of different scales, and obtain multiple feature maps. Multiple feature maps are stacked, multi-scale information is fused, and the number of channels is adjusted using 1x1 convolution to reduce the amount of parameters while maintaining the semantic information of the features to obtain feature maps with semantic information. The main task of the decoder part is to restore the features extracted by the encoder to the resolution of the original image and perform fine-grained classification. The shallow feature map is processed by 1x1 convolution to extract local features, and then fused with the sixth feature map with semantic information. The fused feature map is convolved by 3x3 to further extract features and enhance the classification ability of the semantic segmentation model. The final feature map is adjusted to the same size as the input container image through upsampling. Each pixel corresponds to a category, so that semantic segmentation can be performed efficiently, the liquid to be detected can be accurately segmented from the container image, and the target liquid area of ​​the liquid to be detected is output.

[0058] As an optional example, there are multiple target liquid regions, and based on the target liquid regions, obtaining the second height of the liquid level in the target container includes:

[0059] Calculate the area of ​​each target liquid region;

[0060] Determine the maximum area from the areas of all target liquid regions;

[0061] The height of the target liquid region with the largest area is determined as the second height.

[0062] Optionally, in this embodiment, in order to finally calculate the liquid volume of the measured object in the container, it is necessary to obtain the second height of the liquid surface in the target container. Specifically, the target liquid area is an image containing the measured liquid segmented from the container image by a semantic segmentation model. In order to find the target liquid area with the largest area, opencv (Open Source Computer Vision Library, an open source computer vision library that provides a wealth of image processing and computer vision functions) is used to calculate the area of ​​each target liquid area in the target liquid area, traverse all target liquid areas, compare the area of ​​each target liquid area, and finally determine the height of the target liquid area with the largest area as the second height, and obtain the maximum height of the contour by calculating the height of the circumscribed rectangle of the target liquid area with the largest area and determine it as the second height of the liquid surface in the target container.

[0063] As an optional example, calculating the liquid volume of the liquid to be detected in the target container according to the first height, the second height, and a preset container volume corresponding to the target container includes:

[0064] Calculating the ratio of the second height to the first height to obtain the volume ratio of the liquid to be detected to the target container;

[0065] The product of the volume ratio and the preset container volume is calculated to obtain the liquid volume of the liquid to be detected.

[0066] Optionally, in this embodiment, since the bottom area of ​​the liquid to be detected is the same as that of the container in which it is located, the height ratio of the liquid to be detected in the target container can be obtained by calculating the ratio of the second height of the liquid surface in the target container to the first height of the target container. The height ratio can be used to reflect the proportion of the object to be detected in the height of the entire container. After obtaining the height ratio, it can be multiplied by the preset container volume to obtain the liquid volume of the liquid to be detected in the container.

[0067] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0068] According to another aspect of the embodiment of the present application, a device for measuring the volume of liquid in a container is also provided. Figure 4 As shown, including:

[0069] The recognition module 402 is used to input the image to be detected into the target detection model for container recognition processing to obtain a container recognition result;

[0070] A cropping module 404 is used to crop the target container from the image to be detected based on the container recognition result to obtain a container image when the container recognition result indicates that the target container is detected in the image to be detected;

[0071] The segmentation module 406 is used to input the container image into the semantic segmentation model to perform liquid region segmentation to obtain the target liquid region;

[0072] A first acquisition module 408 is used to acquire a first height of a target container based on the container image;

[0073] A second acquisition module 410 is used to acquire a second height of the liquid level in the target container based on the target liquid area;

[0074] The calculation module 412 is used to calculate the liquid volume of the liquid to be detected in the target container according to the first height, the second height and a preset container volume corresponding to the target container.

[0075] It should be noted that the recognition module 402 in this embodiment can be used to execute step S102 in the embodiment of the present application, the cropping module 404 in this embodiment can be used to execute step S104 in the embodiment of the present application, the segmentation module 406 in this embodiment can be used to execute step S106 in the embodiment of the present application, the first acquisition module 408 in this embodiment can be used to execute step S108 in the embodiment of the present application, the second acquisition module 410 in this embodiment can be used to execute step S110 in the embodiment of the present application, and the calculation module 412 in this embodiment can be used to execute step S112 in the embodiment of the present application.

[0076] As an optional example, the target detection model includes a feature extraction layer and a target detection layer, and the recognition module includes:

[0077] The first extraction unit is used to input the image to be detected into the feature extraction layer for feature extraction to obtain a feature map of the image to be detected;

[0078] The recognition unit is used to input the feature map into the target detection layer for container recognition to obtain a container recognition result, wherein the container recognition result includes detecting the target container in the image to be detected and not detecting the target container in the image to be detected.

[0079] As an optional example, the feature extraction layer includes a slicing layer, a convolution layer, a pooling layer, and a fusion layer, and the first extraction unit includes:

[0080] A slicing subunit, used for inputting the image to be detected into the slicing layer, performing a slicing operation on the image to be detected, and obtaining a plurality of first feature maps, wherein the resolution of the image to be detected is higher than the resolution of the first feature map;

[0081] A first convolution subunit, used for inputting the plurality of first feature maps into the convolution layer, performing a convolution operation on the plurality of first feature maps, and obtaining a plurality of second feature maps;

[0082] A pooling subunit, used for inputting the plurality of second feature maps into a pooling layer, performing feature extraction on the plurality of second feature maps at a plurality of different scales, and obtaining a plurality of third feature maps;

[0083] The first fusion subunit is used to input the multiple third feature maps into the fusion layer, fuse the multiple third feature maps, and obtain the feature map of the image to be detected.

[0084] As an optional example, the target detection layer includes a multi-scale fusion layer and a detection layer, and the recognition unit includes:

[0085] A second fusion subunit is used to input the feature map into the multi-scale fusion layer, perform a multi-scale fusion operation on the feature map, and obtain a fourth feature map;

[0086] The second convolution subunit is used to input the fourth feature map into the detection layer, perform multi-layer convolution operations on the fourth feature map, and generate a container recognition result.

[0087] As an optional example, the semantic segmentation model includes a deep feature processing layer, a shallow feature processing layer, and a feature fusion layer, and the segmentation module includes:

[0088] The second extraction unit is used to input the container image into the deep feature processing layer, perform deep feature extraction on the liquid area, and obtain a deep feature map;

[0089] The third extraction unit is used to input the container image into the shallow feature processing layer, perform shallow feature extraction on the liquid area, and obtain a shallow feature map;

[0090] The processing unit is used to input the deep feature map and the shallow feature map into the feature fusion layer to obtain the target liquid area.

[0091] As an optional example, there are multiple target liquid areas, and the second acquisition module includes:

[0092] A first calculation unit, used for calculating the area of ​​each target liquid region;

[0093] A first determining unit, configured to determine a maximum area from areas of all target liquid regions;

[0094] The second determining unit is used to determine the height of the target liquid region with the largest area as the second height.

[0095] As an optional example, the calculation module includes:

[0096] A second calculation unit is used to calculate the ratio of the second height to the first height to obtain the volume ratio of the liquid to be detected to the target container;

[0097] The third calculation unit is used to calculate the product of the volume ratio and the preset container volume to obtain the liquid volume of the liquid to be detected.

[0098] For other examples of this embodiment, please refer to the above examples and will not be repeated here.

[0099] Figure 5 is a schematic diagram of an optional electronic device according to an embodiment of the present application, such as Figure 5 As shown, it includes a processor 502, a communication interface 504, a memory 506 and a communication bus 508, wherein the processor 502, the communication interface 504 and the memory 506 communicate with each other through the communication bus 508, wherein,

[0100] Memory 506, used for storing computer programs;

[0101] The processor 502 is used to execute the computer program stored in the memory 506 to implement the following steps:

[0102] Input the image to be detected into the target detection model for container recognition processing to obtain the container recognition result;

[0103] When the container recognition result indicates that the target container is detected in the image to be detected, based on the container recognition result, the target container is cropped from the image to be detected to obtain a container image;

[0104] Input the container image into the semantic segmentation model to segment the liquid area and obtain the target liquid area;

[0105] Based on the container image, obtain the first height of the target container;

[0106] Based on the target liquid area, obtaining a second height of the liquid level in the target container;

[0107] The liquid volume of the liquid to be detected in the target container is calculated according to the first height, the second height and a preset container volume corresponding to the target container.

[0108] Optionally, in this embodiment, the communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The communication interface is used for communication between the above electronic device and other devices.

[0109] The memory may include RAM, or may include non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0110] As an example, the memory 506 may include, but is not limited to, the recognition module 402, the cropping module 404, the segmentation module 406, the first acquisition module 408, the second acquisition module 410, and the calculation module 412 in the device for measuring the volume of liquid in the container. In addition, other module units in the device for measuring the volume of liquid in the container may also be included but are not limited to, which will not be described in detail in this example.

[0111] The above-mentioned processor can be a general-purpose processor, which can include but not be limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0112] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0113] It can be understood by those skilled in the art that Figure 5The structure shown is for illustration only. The device for implementing the above-mentioned method for measuring the volume of liquid in a container may be a terminal device, which may be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Figure 5 The structure of the electronic device is not limited. For example, the electronic device may also include Figure 5 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 5 Different configurations shown.

[0114] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0115] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program executes the steps in the above-mentioned method for measuring the volume of liquid in a container when executed by a processor.

[0116] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0117] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0118] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0119] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0120] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0121] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0122] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0123] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for measuring the volume of liquid in a container, characterized in that: include: Input the image to be detected into the target detection model for container recognition processing to obtain the container recognition result; If the container recognition result indicates that a target container is detected in the image to be detected, based on the container recognition result, the target container is cropped from the image to be detected to obtain a container image; Inputting the container image into a semantic segmentation model to perform liquid region segmentation to obtain a target liquid region; Based on the container image, obtaining a first height of the target container; Based on the target liquid area, obtaining a second height of the liquid level in the target container; The liquid volume of the liquid to be detected in the target container is calculated according to the first height, the second height, and a preset container volume corresponding to the target container.

2. The method according to claim 1, characterized in that The target detection model includes a feature extraction layer and a target detection layer. The image to be detected is input into the target detection model for container recognition processing to obtain a container recognition result, which includes: Inputting the image to be detected into the feature extraction layer for feature extraction to obtain a feature map of the image to be detected; The feature map is input into the target detection layer for container recognition to obtain the container recognition result, wherein the container recognition result includes detecting the target container in the image to be detected and not detecting the target container in the image to be detected.

3. The method according to claim 2, characterized in that The feature extraction layer includes a slicing layer, a convolution layer, a pooling layer and a fusion layer. Inputting the image to be detected into the feature extraction layer to obtain a feature map of the image to be detected includes: Inputting the image to be detected into the slicing layer, performing a slicing operation on the image to be detected to obtain a plurality of first feature maps, wherein the resolution of the image to be detected is higher than the resolution of the first feature maps; Inputting the plurality of first feature maps into the convolution layer, performing a convolution operation on the plurality of first feature maps to obtain a plurality of second feature maps; Inputting the plurality of second feature maps into the pooling layer, performing feature extraction on the plurality of second feature maps at a plurality of different scales, and obtaining a plurality of third feature maps; The multiple third feature maps are input into the fusion layer, and the multiple third feature maps are fused to obtain the feature map of the image to be detected.

4. The method according to claim 2, characterized in that: The target detection layer includes a multi-scale fusion layer and a detection layer, and the inputting the feature map into the target detection layer to obtain the container recognition result includes: Inputting the feature map into the multi-scale fusion layer, performing a multi-scale fusion operation on the feature map to obtain a fourth feature map; The fourth feature map is input into the detection layer, a multi-layer convolution operation is performed on the fourth feature map, and the container recognition result is generated.

5. The method according to claim 1, characterized in that The semantic segmentation model includes a deep feature processing layer, a shallow feature processing layer and a feature fusion layer. The container image is input into the semantic segmentation model for liquid area segmentation to obtain the target liquid area including: Inputting the container image into the deep feature processing layer, performing deep feature extraction on the liquid area, and obtaining a deep feature map; Inputting the container image into the shallow feature processing layer, performing shallow feature extraction on the liquid area, and obtaining a shallow feature map; The deep feature map and the shallow feature map are input into the feature fusion layer to obtain the target liquid area.

6. The method according to claim 1, characterized in that There are multiple target liquid regions, and obtaining the second height of the liquid level in the target container based on the target liquid regions includes: Calculating the area of ​​each target liquid region; Determine the maximum area from the areas of all target liquid regions; The height of the target liquid region with the largest area is determined as the second height.

7. The method according to claim 1, characterized in that The step of calculating the liquid volume of the liquid to be detected in the target container according to the first height, the second height, and a preset container volume corresponding to the target container comprises: Calculating the ratio of the second height to the first height to obtain a volume ratio of the liquid to be detected to the target container; The product of the volume ratio and the preset container volume is calculated to obtain the liquid volume of the liquid to be detected.

8. A device for measuring the volume of liquid in a container, characterized in that: include: The recognition module is used to input the image to be detected into the target detection model for container recognition processing to obtain the container recognition result; A cropping module, configured to crop the target container from the image to be detected based on the container recognition result to obtain a container image when the container recognition result indicates that the target container is detected in the image to be detected; A segmentation module, used for inputting the container image into a semantic segmentation model to perform liquid region segmentation to obtain a target liquid region; A first acquisition module, configured to acquire a first height of the target container based on the container image; A second acquisition module, configured to acquire a second height of a liquid level in the target container based on the target liquid region; A calculation module is used to calculate the liquid volume of the liquid to be detected in the target container according to the first height, the second height and a preset container volume corresponding to the target container.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images