A water level calculation method based on image recognition and dual-region convolutional neural network

Abstract

The present invention discloses an image recognition water level calculation method based on a dual-region convolutional neural network, including the steps of image preprocessing and water level annotation, building a training set and model training, performing model testing, projection transformation, and calculating water level. This method can still have recognition stability and high accuracy in extreme environments and when the computer computing power is low, the number of image samples and the number of model training times are small, and the recognition stability and accuracy can be further improved by increasing the number of image samples and the number of model training times. At the same time, this method can achieve rapid deployment, and is particularly suitable for promotion and use in the field of extensive water level monitoring and water quantity statistics.

Description

Technical Field

[0001] The invention belongs to the technical field of water level monitoring in the water conservancy industry, and specifically relates to an image recognition water level calculation method based on a dual-region convolutional neural network. Background Art

[0002] In recent years, artificial intelligence technology has developed rapidly and has shown broad application prospects in various fields. With the continuous development of technologies such as big data, computer vision, and deep learning, image-based automated metering technology has received widespread attention and application. In water resources management, the effective utilization coefficient of farmland irrigation water is one of the most stringent water resources management assessments, and effective water level automated metering technology is of great significance for the precise regulation and measurement of irrigation water. However, the existing automated metering facilities have problems such as high prices, poor anti-interference ability, and high maintenance costs. There are still huge challenges for the large-scale promotion and use of automated metering facilities in underdeveloped areas. In this case, the use of image-based automated metering technology for automatic identification and metering of water levels is an effective solution.

[0003] At present, there are many studies devoted to using computer vision technology and machine learning methods to automatically identify water gauges and water levels. However, the implementation methods are relatively complex. Most studies require "waterline" recognition monitoring or single model training, which requires a large amount of data to train the network. The recognition time is also long, and the recognition rate is easily affected by interference such as lighting and water gauge tilt. This paper uses a target recognition algorithm based on the combination of Mask R-CNN and Faster R-CNN models to locate water gauges and identify water gauge scales. In this method, there is no need to care about the "waterline", but the actual water level is finally calculated using the projection transformation principle, which can effectively improve the success rate and stability of recognition. It is particularly suitable for promotion and use in the field of extensive water level monitoring and water volume statistics. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a method for combining the Mask R-CNN and Faster R-CNN model algorithms, which can still have a high recognition success rate when the computer computing power is low, the number of image samples and the number of model training times are small, and can solve the problem that the Mask R-CNN model algorithm is prone to target loss when the size difference between different targets in the image is too large; at the same time, by taking the four corner pixel coordinates of the two scales with the largest scores of the Faster R-CNN model output results, instead of selecting the scales close to the water surface line for recognition and calculation of the water level, and without calibrating preset points on the water ruler, the image recognition water level calculation method based on the dual-region convolutional neural network can effectively solve the problem that the water line is unclear in extreme environments, the water ruler scale and the preset points of punctuation are covered and cannot be recognized and located.

[0005] The technical solution of the present invention is as follows:

[0006] An image recognition water level calculation method based on a dual-region convolutional neural network comprises the following steps:

[0007] (1) Capture and collect water gauge images in the water conservancy field and perform image preprocessing. Then, label the water gauge contours in the images to produce the first-stage Mask R-CNN model training set.

[0008] (2) Construct a Mask R-CNN mask recurrent convolutional neural network model, set training parameters, and then define a multi-task loss function for each RoI (region of interest), and use the training set to train Mask R-CNN; at the same time, determine whether the model is trained correctly, whether it is trained sufficiently, whether it is overfitting, and whether the initialization weight setting is correct based on the loss function and form the first validation set. When the loss function Loss value reaches the optimal value, the training is completed;

[0009] (3) After Mask R-CNN is trained, the Mask R-CNN model is verified using the first-stage Mask R-CNN model training set and the first validation set. The water gauge contour mask is segmented and the segmented and cropped water gauge image is output. The scale numbers on the cropped water gauge image are then labeled to produce the second-stage Faster R-CNN model training set.

[0010] (4) Perform image standardization, build the Faster R-CNN convolutional neural network model, set training parameters, define a multi-task loss function for each RoI (region of interest), train Faster R-CNN using the second-stage Faster R-CNN model training set, and determine whether the model is trained correctly, sufficiently trained, overfitted, and whether the initialization weight setting is correct based on the loss function, and form a second validation set. When the loss function Loss value reaches the optimal value, the training is completed.

[0011] (5) Test the Faster R-CNN model and output the pixel coordinates of the four corners of the scale with the largest model test result. Then, calculate the real water level through the projection transformation relationship between the water ruler scale in the image pixel coordinate system and the world coordinate system. The relationship equation between the water ruler scale and the actual water level is:

[0012] in,

[0013] ΔH'0 is the height of the character scale in the image, ΔH' is the height from the ruler scale reading to the water surface line in the image, ΔH0 is the actual height of the ruler scale, ΔH is the actual height from the ruler scale reading to the water surface line, H is the actual water level, the angular coordinates of the two scales are (x1, y1), (x2, y2), κ is the projection transformation coefficient between the pixel coordinate system and the world coordinate system;

[0014] Ignoring the coefficient κ, the relationship between the actual water level H and the water gauge scale reading N identified by the image can be derived, that is, the water level calculation formula:

[0015]

[0016] (6) Based on the real water level value and the calculated water level value, a fitting calculation is performed to obtain the optimal solution projection transformation coefficient κ; then, the final water level calculation formula is obtained through the projection transformation relationship between the water scale scale in the image pixel coordinate system and the world coordinate system, as well as the optimal solution projection transformation coefficient:

[0017]

[0018] Due to the influence of various environmental factors (such as image tilt, camera lens distortion, etc.), the water level calculated in the above step (6) deviates from the actual water level. Therefore, it is necessary to perform fitting calculations based on the real water level value and the calculated water level value to obtain the optimal solution projection transformation coefficient κ. Fitting refers to the process of fitting or approximating a set of data points or observations through an appropriate mathematical model or function. In the field of statistics and machine learning, fitting refers to finding the most appropriate model parameters or function form so that the model can best describe or predict the characteristics and trends of the data on a given data set. Fitting can be performed by different methods, such as least squares, maximum likelihood estimation or gradient descent. In the fitting process, different models or function forms are usually tried, and the difference between the model and the data is minimized by adjusting the parameters to find a model that performs well on the entire data set and can be generalized to new data.

[0019] To further illustrate the present invention, when marking the outline of the water gauge in the image, the boundary between the water gauge and the water surface should be marked along the boundary of the water surface, and the water gauge covered by silt or debris should also be marked; the water gauge outline in the image should be marked to fit the water surface line. The advantage of this is that there is no need to find and identify the water surface line. Step (5) takes the four corner pixel coordinates of the scale with the maximum score output by the Faster R-CNN model to avoid identifying certain scales covered by something, which may lead to recognition errors. This method for identifying water gauges and scales combines the Mask R-CNN and Faster R-CNN algorithms to solve the problems of accurate segmentation of water gauge edges and rapid recognition of multi-target and multi-size water gauge scales, greatly improving the stability and accuracy of recognition.

[0020] The captured image can be captured by monitoring equipment such as cameras and water gauges installed in reservoirs, rivers, channels, etc. for artificially measuring water levels. The influencing factors are related to influencing factors such as water gauge tilt, camera internal parameters, external parameters, lens distortion parameters, etc. The image preprocessing includes steps such as image cropping, normalization, and standardization.

[0021] Beneficial effects of the present invention:

[0022] The method of the present invention combines the Mask R-CNN and Faster R-CNN model algorithms, and can still have a high recognition success rate when the computer computing power is low, the number of image samples and the number of model training times are small, and solves the problem that the Mask R-CNN model algorithm is prone to target loss when the size difference between different targets in the image is too large. By taking the four corner pixel coordinates of the two scales with the largest scores of the Faster R-CNN model output results, instead of selecting the scale close to the water surface line for identification and calculation of the water level, there is no need to calibrate the preset points on the water ruler, which solves the problem that the water line is unclear in extreme environments, the water ruler scale and the punctuation preset points are covered and cannot be identified and located. The present invention uses the projection transformation relationship between the water ruler scale in the image pixel coordinate system and the world coordinate system, and combines the influencing factor fitting algorithm, without directly identifying specific water ruler scales and water surface lines, and does not need to perform image tilt correction, and can also identify and calculate relatively stable and accurate water level values.

[0023] At the same time, this method still has recognition stability and high accuracy in extreme environments, when the computer computing power is low, the number of image samples and the number of model training times are small, and the recognition stability and accuracy can be further improved by increasing the number of image samples and the number of model training times. It can achieve rapid deployment and is particularly suitable for promotion and use in the field of extensive water level monitoring and water volume statistics. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the process of the present invention;

[0025] Figure 2 It is the Mask R-CNN model training loss function curve;

[0026] Figure 3 It is the loss function curve of Faster R-CNN model training;

[0027] Figure 4 It is a schematic diagram of the principle of water level calculation;

[0028] Figure 5 It is a fitting graph of actual water level value and predicted value. DETAILED DESCRIPTION

[0029] The present invention will be further described below in conjunction with specific embodiments.

[0030] Example:

[0031] An image recognition water level calculation method based on a dual-region convolutional neural network comprises the following steps:

[0032] (1) Capture and collect water gauge images in the water conservancy field and perform image preprocessing. Then, label the water gauge contours in the images to produce the first-stage Mask R-CNN model training set.

[0033] (2) Construct a Mask R-CNN mask recurrent convolutional neural network model, set training parameters, and then define a multi-task loss function for each RoI (region of interest), and use the training set to train Mask R-CNN; at the same time, determine whether the model is trained correctly, whether it is trained sufficiently, whether it is overfitting, and whether the initialization weight setting is correct based on the loss function and form the first validation set. When the loss function Loss value reaches the optimal value, the training is completed;

[0034] (3) After Mask R-CNN is trained, the Mask R-CNN model is verified using the first-stage Mask R-CNN model training set and the first validation set. The water gauge contour mask is segmented and the segmented and cropped water gauge image is output. The scale numbers on the cropped water gauge image are then labeled to produce the second-stage Faster R-CNN model training set.

[0035] (4) Perform image standardization, build the Faster R-CNN convolutional neural network model, set training parameters, define a multi-task loss function for each RoI (region of interest), train Faster R-CNN using the second-stage Faster R-CNN model training set, and determine whether the model is trained correctly, sufficiently trained, overfitted, and whether the initialization weight setting is correct based on the loss function, and form a second validation set. When the loss function Loss value reaches the optimal value, the training is completed.

[0036] (5) Test the Faster R-CNN model and output the pixel coordinates of the four corners of the scale with the largest model test result. Then, calculate the real water level through the projection transformation relationship between the water ruler scale in the image pixel coordinate system and the world coordinate system. The relationship equation between the water ruler scale and the actual water level is:

[0037] in,

[0038] ΔH'0 is the height of the character scale in the image, ΔH' is the height from the ruler scale reading to the water surface line in the image, ΔH0 is the actual height of the ruler scale, ΔH is the actual height from the ruler scale reading to the water surface line, H is the actual water level, the angular coordinates of the two scales are (x1, y1), (x2, y2), κ is the projection transformation coefficient between the pixel coordinate system and the world coordinate system;

[0039] Ignoring the coefficient κ, the relationship between the actual water level H and the water gauge scale reading N identified by the image can be derived, that is, the water level calculation formula:

[0040]

[0041] (6) Based on the real water level value and the calculated water level value, a fitting calculation is performed to obtain the optimal solution projection transformation coefficient κ; then, the final water level calculation formula is obtained through the projection transformation relationship between the water scale scale in the image pixel coordinate system and the world coordinate system, as well as the optimal solution projection transformation coefficient:

[0042]

[0043] When marking the water gauge contour in the image, the boundary between the water gauge and the water surface should be marked along the water surface boundary, and the water gauge covered by silt or debris should also be marked; the water gauge contour in the image should be marked to fit the water surface line.

Claims

1. A method for calculating water level by image recognition based on a dual-region convolutional neural network, characterized in that: The following steps are involved: (1) Capture and collect water gauge images in the water conservancy field and perform image preprocessing. Then, label the water gauge contours in the images to produce the first-stage Mask R-CNN model training set. (2) Construct the Mask R-CNN mask recurrent convolutional neural network model, set the training parameters, and then define a multi-task loss function for each RoI, and use the training set to train the Mask R-CNN; at the same time, determine whether the model is trained correctly, whether it is trained sufficiently, whether it is overfitting, and whether the initialization weight setting is correct based on the loss function and form the first validation set. When the loss function Loss value reaches the optimal value, the training is completed; (3) After the Mask R-CNN training is completed, the Mask R-CNN model is verified using the first-stage Mask R-CNN model training set and the first validation set, and the water gauge contour mask is instance segmented, and the segmented and cropped water gauge image is output; then the scale numbers on the cropped water gauge image are labeled to produce the second-stage Faster R-CNN model training set; (4) Perform image standardization, build a Faster R-CNN convolutional neural network model, set training parameters, define a multi-task loss function for each RoI, train Faster R-CNN using the second-stage Faster R-CNN model training set, and determine whether the model is trained correctly, whether it is sufficiently trained, whether it is overfitted, and whether the initialization weight setting is correct based on the loss function, and form a second validation set. When the loss function Loss value reaches the optimal value, the training is completed; (5) Test the Faster R-CNN model and output the coordinates of the four corner pixels of the maximum scale of the model test result. Then, calculate the real water level through the projection transformation relationship between the water ruler scale in the image pixel coordinate system and the world coordinate system. The relationship equation between the water ruler scale and the actual water level is: in, ΔH'0 is the height of the character scale in the image, ΔH' is the height from the ruler scale reading to the water surface line in the image, ΔH0 is the actual height of the ruler scale, ΔH is the actual height from the ruler scale reading to the water surface line, H is the actual water level, the angular coordinates of the two scales are (x1, y1), (x2, y2), κ is the projection transformation coefficient between the pixel coordinate system and the world coordinate system; Ignoring the coefficient κ, the relationship between the actual water level H and the water gauge scale reading N identified by the image can be derived, that is, the water level calculation formula: (6) According to the real water level value and the calculated water level value, a fitting calculation is performed to obtain the optimal solution projection transformation coefficient κ; then, the final water level calculation formula is obtained through the projection transformation relationship between the water ruler scale in the image pixel coordinate system and the world coordinate system, as well as the optimal solution projection transformation coefficient:

2. The method for calculating water level by image recognition based on a dual-region convolutional neural network according to claim 1, characterized in that: When marking the water gauge contour in the image, the boundary between the water gauge and the water surface should be marked along the water surface boundary, and the water gauge covered by silt or debris should also be marked; the water gauge contour in the image should be marked to fit the water surface line.

Images