Self-adaptive visual liquid level identification method

Abstract

The invention relates to the technical field of visual liquid level identification, in particular to a self-adaptive visual liquid level identification method, which comprises the following steps of: inputting an original image, performing a target detection algorithm, performing an instance segmentation algorithm and performing scale mapping calculation. The existing container device does not need to be transformed, and automatic liquid level reading identification can be realized only by adding a camera and a program; the influence of various colors on liquid level line detection is removed through gray-scale detection network input, various solution liquid level surfaces can be accurately identified, and the generalization ability is high; through optimization of the instance segmentation process, instance segmentation is introduced when needed in the algorithm, calculation is not carried out when not needed, accuracy and real-time performance are ensured, and meanwhile the hardware requirement of the algorithm is lowered. And through a distortion correction algorithm, the problem of inaccurate liquid level height identification caused by distortion of the liquid level due to the change of the position of the camera is corrected.

Description

Technical Field

[0001] The present invention relates to the technical field of visual liquid level recognition, and in particular to an adaptive visual liquid level recognition method. Background Art

[0002] Liquid level recognition is widely used in the industrial field, especially in chemical plants and production workshops. It mainly includes the reading of various container liquid levels, high and low identification, etc., to ensure the safety of the production process and production status. At present, the mainstream method is manual reading. Although manual reading has the advantages of high accuracy and high flexibility, it also has many disadvantages, such as low reading efficiency, inability to monitor in real time, and many hidden dangers.

[0003] Therefore, with the advancement of science and technology, semi-automatic and automatic liquid level recognition has become the direction of development. For example, a chemical reactor liquid level recognition method based on an improved YoloV4-Tiny disclosed in the invention patent with publication number CN117726828A and a measuring cylinder liquid level recognition method and device disclosed in the invention patent with publication number CN112132131B have appeared to automatically identify the liquid level.

[0004] However, during use, it was found that liquid surface distortion is prone to occur during camera shooting, resulting in large recognition errors. In addition, color changes in the container are also likely to affect the recognition of the liquid level, which also results in large recognition errors and poor practicality. Therefore, an adaptive visual liquid level recognition method is urgently needed to improve the above problems. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method that does not require modification of the existing container device, but only requires the addition of a camera and a program to realize automatic liquid level reading recognition; by graying the detection network input to remove the influence of various colors on liquid level line detection, it is possible to accurately identify various liquid level surfaces of solutions, and has strong generalization ability; by optimizing the instance segmentation process, it is possible to introduce instance segmentation when needed in the algorithm, and not perform calculations when not needed, thereby ensuring accuracy and real-time performance while reducing the hardware requirements of the algorithm; by checking the scale lines, accurate matching of the scale lines is achieved, including deduplication of repeatedly identified scale lines and completion of undetected scale lines, thereby avoiding errors in the final reading caused by incorrect matching of the scale lines; by using a distortion correction algorithm, an adaptive visual liquid level recognition method is provided to correct the problem of inaccurate liquid level height recognition caused by distortion of the liquid surface due to changes in the camera position.

[0006] An adaptive visual liquid level recognition method of the present invention comprises the following steps:

[0007] S1. Input original image: Pull the RTSP video stream of the camera through the network as the data input of the intelligent liquid level recognition algorithm. The program intercepts the video stream into image frames and passes them to the liquid level recognition algorithm for recognition;

[0008] S2, target detection algorithm: the target detection algorithm is used to detect the container type and liquid level height information;

[0009] S3, instance segmentation algorithm: the identification of scale line position is realized by integrating instance segmentation algorithm;

[0010] S4. Scale mapping calculation: Calculate the specific liquid level reading by combining the proportional relationship between the scales with the height of the liquid level line.

[0011] Preferably, the target detection algorithm in S2 adopts YOLOv8.

[0012] Preferably, the instance segmentation algorithm in S3 adopts YOLOv8-seg.

[0013] Preferably, the target detection algorithm specifically implements the steps including:

[0014] S2-1. Dataset collection:

[0015] By collecting camera images of common scenes, we try to include samples of different container styles, different scale styles, different container sizes, different brightness levels, and different background environments during the collection process to ensure the diversity of samples and provide a data foundation for the generalization ability of subsequent models.

[0016] S2-2. Annotation of the dataset:

[0017] According to the needs, the collected data is annotated, including the following two aspects: the type of container and the position of the liquid level line. The type of container is used to determine the type of container in the future, and then determine the scale range of the container to achieve adaptive recognition of various containers. The position of the liquid level line is used to determine the position of the liquid level line. By comparing the position of the liquid level line, the scale reading is calculated;

[0018] S2-3. Model training:

[0019] YOLOv8 is used as a target detection algorithm. Two target detection models are trained for the container type and the liquid level line position through the above data annotation. They are used to determine the container type and identify the liquid level line position. In addition, the YOLOv8 algorithm is improved during the training process. The input and prediction of the liquid level line position detection model are replaced with grayscale images to ensure that liquid levels of various colors can be identified, thereby improving the detection accuracy.

[0020] S2-4. Reasoning of the model:

[0021] Similar to the training, the input of the liquid level line position detection model is replaced with a grayscale image, which greatly improves the detection accuracy.

[0022] Preferably, when the height of the liquid level is less than a set threshold, the bottom of the liquid level detection frame is taken as the height of the liquid level;

[0023] When the liquid level height line is greater than the set threshold, the top of the liquid level line detection frame is taken as the liquid level line height;

[0024] In other cases, the midpoint of the liquid level detection frame is taken as the liquid level height.

[0025] Preferably, the specific implementation steps of the instance segmentation algorithm include:

[0026] S3-1. Dataset collection:

[0027] By collecting camera images of common scenes, we try to include samples of different container styles, different scale styles, different container sizes, different brightness levels, and different background environments during the collection process to ensure the diversity of samples and provide a data foundation for the generalization ability of subsequent models.

[0028] S3-2. Annotation of the dataset:

[0029] Label the collected data as needed, including labeling the scale line pixels;

[0030] S3-3. Model training:

[0031] Select YOLOv8-seg as the instance segmentation algorithm, train YOLOv8-seg with the above-mentioned labeled data set to obtain an instance segmentation model;

[0032] S3-4. Reasoning of the model:

[0033] When the algorithm starts, the type of container is identified and determined once, and then the scale of the container is detected and identified. If the type, position, and size of the container do not change, it is considered that the current container has not changed, and the instance segmentation model is no longer used to segment the scale. Instead, the result of the previous segmentation is used. When the type, position, and size of the container changes, the instance segmentation model is called again to determine the position of the scale line. This method can ensure dynamic adjustment of the scale and small computing resources of the algorithm, and can be widely used in small computing devices.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. No need to modify the existing container device, just add a camera and program to realize automatic liquid level reading recognition;

[0036] 2. The influence of various colors on liquid level detection is removed by graying the detection network input, which can accurately identify various solution levels and has strong generalization ability;

[0037] 3. By optimizing the instance segmentation process, instance segmentation can be introduced into the algorithm when needed, and no calculation is performed when not needed, thus ensuring accuracy and real-time performance while reducing the hardware requirements of the algorithm;

[0038] 4. Accurate matching of scale lines is achieved through the inspection of scale lines, including deduplication of repeatedly identified scale lines and completion of undetected scale lines, thus avoiding errors in the final reading caused by incorrect matching of scale lines;

[0039] 5. The distortion correction algorithm is used to correct the problem of inaccurate liquid level recognition caused by distortion of the liquid surface due to changes in the camera position. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the structure of the adaptive visual liquid level recognition method of the present invention;

[0041] Figure 2 It is a schematic diagram of the process of the present invention;

[0042] Figure 3 This is a comparison chart of the detection results of the present invention and ordinary YOLOv8;

[0043] Figure 4 This is a diagram showing the actual reasoning effect of the target detection algorithm of the present invention;

[0044] Figure 5 It is a detection schematic diagram of the present invention when the height of the liquid level line is less than a set threshold value;

[0045] Figure 6 It is a schematic diagram when the liquid level height line of the present invention is greater than a set threshold value;

[0046] Figure 7 It is a schematic diagram of the present invention when taking the midpoint of the liquid level detection frame as the liquid level height;

[0047] Figure 8 This is a diagram showing the actual detection effect of the instance segmentation algorithm of the present invention;

[0048] Fig. 9 It is a scale mapping calculation effect diagram of the present invention.

[0049] Markings in the attached figure: 1. liquid level container to be identified; 2. camera; 3. server; 4. intelligent liquid level identification algorithm; 5. display screen. DETAILED DESCRIPTION

[0050] In order to facilitate understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. The present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0051] Example

[0052] An adaptive visual liquid level recognition method, comprising the following steps:

[0053] S1. Input original image: Pull the RTSP video stream of the camera through the network as the data input of the intelligent liquid level recognition algorithm. The program intercepts the video stream into image frames and passes them to the liquid level recognition algorithm for recognition;

[0054] S2, target detection algorithm: the target detection algorithm is used to detect the container type and liquid level height information;

[0055] S3, instance segmentation algorithm: the identification of scale line position is realized by integrating instance segmentation algorithm;

[0056] S4, scale mapping calculation: calculate the specific reading of the liquid level by combining the proportional relationship between the scales with the height of the liquid level line;

[0057] The target detection algorithm in S2 adopts YOLOv8;

[0058] The instance segmentation algorithm in S3 adopts YOLOv8-seg;

[0059] The target detection algorithm specifically implements the following steps:

[0060] S2-1. Dataset collection:

[0061] By collecting camera images of common scenes, we try to include samples of different container styles, different scale styles, different container sizes, different brightness levels, and different background environments during the collection process to ensure the diversity of samples and provide a data foundation for the generalization ability of subsequent models.

[0062] S2-2. Annotation of the dataset:

[0063] According to the needs, the collected data is annotated, including the following two aspects: the type of container and the position of the liquid level line. The type of container is used to determine the type of container in the future, and then determine the scale range of the container to achieve adaptive recognition of various containers. The position of the liquid level line is used to determine the position of the liquid level line. By comparing the position of the liquid level line, the scale reading is calculated;

[0064] S2-3. Model training:

[0065] YOLOv8 is used as a target detection algorithm. Two target detection models are trained for the container type and the liquid level line position through the above data annotation. They are used to determine the container type and identify the liquid level line position. In addition, the YOLOv8 algorithm is improved during the training process. The input and prediction of the liquid level line position detection model are replaced with grayscale images to ensure that liquid levels of various colors can be identified, thereby improving the detection accuracy.

[0066] like Figure 3 As shown in the figure, through testing, the model replaced with grayscale images has a much better detection effect than the commonly used RGB images. This operation can greatly improve the detection accuracy in this scenario. The YOLOv8 model trained with the original RGB images can only accurately recognize the images (a) similar to the training, but cannot recognize other colors normally. The grayscale model is insensitive to color and can correctly recognize images of various colors.

[0067] S2-4. Reasoning of the model:

[0068] Similar to the training, the input of the liquid level line position detection model is replaced with a grayscale image, which greatly improves the detection accuracy.

[0069] like Figure 4 As shown, the green frame is the container type detection frame, and the red frame is the liquid level detection frame;

[0070] The position of the liquid level line is processed. Since the camera is generally placed horizontally on one side of the container, the higher and lower liquid levels under the camera's perspective are not straight lines when observed by the camera, but have a large arc. If the same method is used for processing, it will cause a large error. Therefore, the following processing is performed:

[0071] like Figure 5 As shown, when the liquid level height is less than the set threshold, the bottom of the liquid level detection frame is taken as the liquid level height;

[0072] like Figure 6 As shown, when the liquid level height line is greater than the set threshold, the top of the liquid level line detection frame is taken as the liquid level line height;

[0073] like Figure 7 As shown, in other cases, the midpoint of the liquid level detection frame is taken as the liquid level height;

[0074] The specific implementation steps of the instance segmentation algorithm include:

[0075] S3-1. Dataset collection:

[0076] By collecting camera images of common scenes, we try to include samples of different container styles, different scale styles, different container sizes, different brightness levels, and different background environments during the collection process to ensure the diversity of samples and provide a data foundation for the generalization ability of subsequent models.

[0077] S3-2. Annotation of the dataset:

[0078] Label the collected data as needed, including labeling the scale line pixels;

[0079] S3-3. Model training:

[0080] Select YOLOv8-seg as the instance segmentation algorithm, train YOLOv8-seg with the above-mentioned labeled data set to obtain an instance segmentation model;

[0081] S3-4. Reasoning of the model:

[0082] When the algorithm starts, the type of the container is identified and determined once, and then the scale of the container is detected and identified. If the type, position, and size of the subsequent container do not change, it is considered that the current container has not changed, and the instance segmentation model is no longer used to segment the scale. Instead, the result of the previous segmentation is used. When the type, position, and size of the container changes, the instance segmentation model is called again to determine the position of the scale line. This method can ensure dynamic adjustment of the scale and ensure that the algorithm's computing resources are small, and can be widely used in small computing devices;

[0083] like Figure 8 As shown, the blue is the scale line of instance segmentation, the green box is the container type detection box, and the red box is the liquid level detection box;

[0084] Process the identified scale lines. Because some scale lines are repeated or not identified, etc., the following verification is performed using a 3L container as an example:

[0085] (1) Calculate the midpoint of the Y axis (vertical axis) of the scale line as the scale line position;

[0086] (2) Perform an arithmetic check on the scale lines and delete the values ​​that do not meet the check. The check rules are as follows:

[0087]

[0088] Where: S n-1 is the previous scale value, S n is the current scale value, S n+1is the next scale value, L is the total height of the container, n is a constant, i is the number of scales set for the current container, and values ​​that meet this expression are added to the scale list, and those that do not meet this expression are deleted. Through this rule, duplicate detected scale values ​​are deleted;

[0089] (3) When the scale list is not equal to the number of scales, the scales are completed using the following rules:

[0090]

[0091] Then add a value of:

[0092]

[0093] By looping through this rule, undetected scale values ​​can be completed until all scales are matched;

[0094] The specific implementation steps of the scale mapping calculation include:

[0095] S4-1, such as Fig. 9 As shown, the scale of the bottle is converted into a scale. Taking the total height of the container as 1, the scale of 700 is at the position of 0.056 of the total height of the bottle, and the scale of 1600 is at the position of 0.495 of the total height of the bottle. The purpose of this step is to make the subsequent identification reading more accurate;

[0096] S4-2, the reading calculation formula is:

[0097]

[0098] Where D is the liquid level height reading, d is the maximum scale value less than the liquid level height, s is the ratio corresponding to the liquid level, s n+1 is the scale line ratio greater than the liquid level line height, s n-1 is the scale ratio that is less than the liquid level height, and i is the value represented by each scale unit;

[0099] For example, if the ratio of the detected liquid level line to the bottle is 0.36, then the reading D=1300+(0.36-0.343) / (0.495-0.343)*300=1334.

[0100] In factory production lines and laboratory scenes, computer vision technology is combined to realize adaptive liquid level height recognition. By detecting the liquid level height and scale line position, liquid level reading recognition in various scenes is realized. After actual scene testing, the reading accuracy rate in dozens of container and solution mixing scenes is more than 99%, and it can read and recognize scenes of various containers, different reading ranges, and different solutions; compared with other existing methods, this method does not limit the type of container and solution, and can perform adaptive recognition; compared with manual, it has the advantages of high reading efficiency and all-weather real-time monitoring, and has broad application prospects.

[0101] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An adaptive visual liquid level recognition method, characterized in that: The following steps are involved: S1. Input original image: Pull the RTSP video stream of the camera through the network as the data input of the intelligent liquid level recognition algorithm. The program intercepts the video stream into image frames and passes them to the liquid level recognition algorithm for recognition; S2, target detection algorithm: detect the container type and liquid level height information through the target detection algorithm; S3, instance segmentation algorithm: the identification of scale line position is realized by integrating instance segmentation algorithm; S4. Scale mapping calculation: Calculate the specific liquid level reading by combining the proportional relationship between the scales with the height of the liquid level line.

2. An adaptive visual liquid level recognition method as claimed in claim 1, characterized in that: The target detection algorithm in S2 adopts YOLOv8.

3. The method for adaptive visual liquid level recognition according to claim 1, characterized in that: The instance segmentation algorithm in S3 adopts YOLOv8-seg.

4. The method for adaptive visual liquid level recognition according to claim 2, characterized in that: The target detection algorithm specifically implements the following steps: S2-1. Dataset collection: By collecting camera images of common scenes, we try to include samples of different container styles, different scale styles, different container sizes, different brightness levels, and different background environments during the collection process to ensure the diversity of samples and provide a data foundation for the generalization ability of subsequent models. S2-2. Annotation of the dataset: According to the needs, the collected data is annotated, including the following two aspects: the type of container and the position of the liquid level line. The type of container is used to determine the type of container in the future, and then determine the scale range of the container to achieve adaptive recognition of various containers. The position of the liquid level line is used to determine the position of the liquid level line. By comparing the position of the liquid level line, the scale reading is calculated; S2-3. Model training: YOLOv8 is used as a target detection algorithm. Two target detection models are trained for the container type and the liquid level line position through the above data annotation. They are used to determine the container type and identify the liquid level line position. In addition, the YOLOv8 algorithm is improved during the training process. The input and prediction of the liquid level line position detection model are replaced with grayscale images to ensure that liquid levels of various colors can be identified, thereby improving the detection accuracy. S2-4. Reasoning of the model: Similar to the training, the input of the liquid level line position detection model is replaced with a grayscale image, which greatly improves the detection accuracy.

5. An adaptive visual liquid level recognition method as claimed in claim 4, characterized in that: When the height of the liquid level is less than the set threshold, the bottom of the liquid level detection frame is taken as the height of the liquid level; When the liquid level height line is greater than the set threshold, the top of the liquid level line detection frame is taken as the liquid level line height; In other cases, the midpoint of the liquid level detection frame is taken as the liquid level height.

6. The method for adaptive visual liquid level recognition according to claim 3, characterized in that: The specific implementation steps of the instance segmentation algorithm include: S3-1. Dataset collection: By collecting camera images of common scenes, we try to include samples of different container styles, different scale styles, different container sizes, different brightness levels, and different background environments during the collection process to ensure the diversity of samples and provide a data foundation for the generalization ability of subsequent models. S3-2. Annotation of the dataset: Label the collected data as needed, including labeling the scale line pixels; S3-3. Model training: Select YOLOv8-seg as the instance segmentation algorithm, train YOLOv8-seg with the above-mentioned labeled data set to obtain an instance segmentation model; S3-4. Reasoning of the model: When the algorithm starts, the type of container is identified and determined once, and then the scale of the container is detected and identified. If the type, position, and size of the container do not change, it is considered that the current container has not changed, and the instance segmentation model is no longer used to segment the scale. Instead, the result of the previous segmentation is used. When the type, position, and size of the container changes, the instance segmentation model is called again to determine the position of the scale line. This method can ensure dynamic adjustment of the scale and small computing resources of the algorithm, and can be widely used in small computing devices.

Images