A transparent container liquid content detection method based on I-YOLOF

Abstract

The application discloses a kind of transparent container liquid content detection methods based on I-YOLOF, it is related to transparent container liquid content identification technical field.The picture of transparent container is collected, the picture is marked according to empty bottle, little water, half water, more water and full water 5 kinds of categories, and the collected data set is divided into training set and test set, I-YOLOF model is trained using training set, test set is used to test model effect, finally obtain the I-YOLOF model that can be used to detect the liquid content of transparent container.I-YOLOF network model is the variant of YOLOF, and the improvement method is: the main network uses ResNet50 to extract features;Decoder introduces a new head branch for predicting liquid content, which is parallel to the branch used for container classification, and a ternary attention module is used.The application can quickly and accurately identify the liquid content category in the transparent container by designing and training the I-YOLOF model based on detecting the transparent container.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of transparent container detection, and in particular to a transparent container liquid content detection method based on I-YOLOF. BACKGROUND

[0002] In our daily life, there are various kinds of substances with liquid state, such as drinking water, various fuels, medicines and chemicals, etc., which play a very important role in our daily life. Transparent container liquid content detection has very wide application scenarios in our real life, such as service robots, pouring robots and industrial observation systems, etc.

[0003] Traditional transparent container detection methods use the principle that transparent objects absorb light of a specific wavelength, use infrared emitters and intensity sensors to identify transparent containers, or use light detection and ranging data to accurately segment transparent objects. These methods can successfully identify transparent container targets, but the detection process is cumbersome, the performance is limited, and it is difficult to meet the needs of transparent container detection in actual application scenarios.

[0004] With the rapid development of convolutional neural networks (CNN), algorithms based on deep learning have rapidly replaced traditional methods for detecting transparent containers and other objects due to their excellent performance. Compared with traditional target detection algorithms, target detection algorithms based on convolutional neural networks have strong generalization ability and can automatically learn deep semantic information of targets. Transparent container detection based on deep convolutional networks generally has the following two methods: one-stage methods such as the YOLO series and SSD; and two-stage detection such as the Region-based CNN series (R-CNN, Fast R-CNN, Faster R-CNN, etc.). One-stage target detection methods complete the recognition and positioning tasks of target frames in one prediction, which can greatly improve the detection speed at the expense of some accuracy. Two-stage detection needs to generate proposal boxes on the feature layer, and then classify and locate through the proposal boxes, which has high accuracy but slow speed. Real-time performance is one of the characteristics that must be possessed by transparent container detection, so it is more practical to improve the accuracy of one-stage detection methods while ensuring that the speed meets the real-time requirements of transparent container detection.

[0005] Liquid content estimation has wide applications in industries such as food and beverage, pharmaceuticals, and chemical processing. Combining style transfer and segmentation networks can be used to detect whether a transparent container contains liquid. By analyzing contextual cues from neighboring objects, deep learning-based methods can estimate the volume of a transparent container and the specific height of the liquid it contains. However, these methods are cumbersome and lack practicality. Considering that daily decision-making often only requires a rough estimate of the liquid content in a transparent container, there is an urgent need in this field for a method that can quickly and accurately identify the type of liquid content in a transparent container. Summary of the Invention

[0006] The purpose of this invention is to provide a method for detecting the liquid content category in transparent containers based on the I-YOLOF model.

[0007] The technical solution of this invention is to design and train an I-YOLOF model, and use the model to identify transparent containers in an image and the type of liquid content in the containers.

[0008] The I-YOLOF model framework is as follows: Figure 2 As shown, it is the YOLOF model ( Figure 1 A variant of YOLOF. The main difference between I-YOLOF and YOLOF in terms of model framework is that the decoder of the I-YOLOF network model contains three parallel head branches: the first head branch is used for regression localization, the second head branch is used for container classification, and the third head branch is used for predicting liquid content. It runs in parallel with the second branch and uses a triplet attention module.

[0009] The total loss function of I-YOLOF By classification loss function Regression loss function Liquid content prediction loss function The composition is defined by formula (1).

[0010] (1)

[0011] in It is a constant; They are defined by formulas (2), (3), and (4) respectively.

[0012] (2)

[0013] (3)

[0014] (4)

[0015] where, and represent the true labels of classification and liquid content estimation, respectively; represent the predicted values of classification, liquid content and bounding box, respectively; and represent the true and predicted bounding boxes, respectively; represents the number of generated correct bounding boxes, is a scalar multiplication operation; FL() and represent the focal loss function and the smooth L1 loss function, respectively.

[0016] The I-YOLOF model is iteratively trained using the training set, and when the loss value tends to be stable, it is saved as a training model. The trained model is used to evaluate on the test set; if the evaluation result is not ideal, the parameters are adjusted and retrained.

[0017] The present application proposes a method for detecting transparent containers and their liquid content types contained in RGB images based on the I-YOLOF model. This method breaks through the limitations of traditional transparent container recognition, not only can identify the position information of transparent containers in the picture, but also can identify the liquid content information contained in it, and widens the application scenario of transparent container detection. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 YOLOF model architecture

[0019] Figure 2 I-YOLOF model architecture

[0020] Figure 3 5 liquid content state annotation examples

[0021] Figure 4 Transparent container and its liquid content category recognition result example DETAILED DESCRIPTION

[0022] The present application is a transparent container liquid content detection method based on the I-YOLOF model, the specific steps are as follows:

[0023] (1) Through the Internet mobile phone and using the mobile phone to shoot from the surrounding living environment, about 6000 pictures containing transparent containers are obtained, and the water content state of the transparent containers contained in the pictures is labeled according to 5 states of empty, little, half, much and full. Figure 3 Two labeling examples are given for each state.

[0024] ​​(2) The correctly labeled pictures are randomly divided into training set and test set in the ratio of 7:3.

[0025] (3) Set the training parameters of I-YOLOF model, including: learning rate is set to 0.001, weight decay is set to 0.0001, momentum is set to 0.9. The learning rate of the 8th and 11th rounds is 1 / 3 of the previous batch, and the learning rate of the remaining rounds remains unchanged.

[0026] (4) Use the pictures in the training set to iteratively train the I-YOLOF model until the total loss value of the model tends to be stable. Save the I-YOLOF model obtained by training.

[0027] (5) Use the saved I-YOLOF model to evaluate the performance on the test set. If the expected effect is not achieved, adjust the training parameters of I-YOLOF model and start step (4) again.

[0028] (6) Use the I-YOLOF model that passes the evaluation to identify the transparent containers and their contained liquid content categories in the pictures. Figure 4 10 examples of recognition results are given, where the purple box represents the transparent container detected by the I-YOLOF model, the number before the vertical line in the box represents the state value of the liquid content (1 represents empty bottle, 2 represents little water, 3 represents half water, 4 represents more water, 5 represents full water), and the number after the vertical line represents the confidence of the prediction result, the larger the value, the more reliable.

Claims

1. A method for detecting the liquid content of a transparent container based on I-YOLOF, characterized mainly in that, Specifically comprising the following steps: S1: Collecting a transparent container picture dataset, labeling the pictures according to five categories of empty bottle, little water, half water, much water and full water, and dividing the dataset into a training set and a test set; constructing an I-YOLOF network model, which is mainly characterized by: The I-YOLOF network model is a variant of YOLOF, and like YOLOF, it sequentially includes a backbone network, an encoder and a decoder; the backbone network of the I-YOLOF network model uses ResNet50 for feature extraction, outputs a feature map C5 with 2048 channel numbers, and has a down-sampling rate of 32; the encoder of the I-YOLOF network model is a dilated encoder, which outputs a feature map with 512 channel numbers; the decoder of the I-YOLOF network model contains three parallel head branches, the first head branch is used for regression positioning, the second head branch is used for container classification, and these two branches are the same as YOLOF; the third head branch is used for predicting the liquid content, which is parallel to the second branch and uses a three-tuple attention module; S2: Training the I-YOLOF network model using the training set described in S1 to obtain a transparent container liquid content detection model; S3: Using the trained I-YOLOF network model in S2 to test the transparent container detection in the test set and predict the liquid content category.

2. The method according to claim 1, wherein, The loss function used in the training of the I-YOLOF network model in step S2 is mainly characterized by: S2.1 Total loss function for I-YOLOF consisting of a classification loss function , a regression loss function , and a liquid content prediction loss function defined by equation (1), where is a constant; (1) S2.2 、 、 are defined by equations (2), (3), (4), respectively; (2) (3) (4) where, and denote the true labels for classification and liquid content estimation, respectively; , , denote the predicted values for classification, liquid content, and bounding box, respectively; and denote the true and predicted bounding boxes, respectively; denotes the number of generated correct bounding boxes, is a scalar multiplication operation; FL() and denote the focal loss function and the smooth L1 loss function, respectively.

Images