Water level identification method based on image processing

Abstract

The invention discloses a water level identification method based on image processing. The method specifically comprises the four steps of image visual angle correction, water gauge area semantic segmentation, digital detection and identification and graduated scale reading. According to the invention, the water level is intelligently measured based on the real-time back-transmission video picture of the actual camera and the identification and detection of the scale of the water gauge, the requirement of water gauge reading precision is met, the problems existing in the traditional water level monitoring method are solved by an automatic water level measurement technology, and the method has the advantages of strong applicability, wide positioning range, high positioning precision, real-time back-transmission of data and the like. Real-time acquisition of water level data is of great significance to flood prevention and protection of life and property safety of people.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and specifically relates to a method for realizing accurate recognition and measurement of water level by combining UNet++ and CRNN OCR technology through steps such as image perspective correction, water gauge area segmentation, digital detection and recognition, and scale reading. Background Art

[0002] Water level is an important data for hydrological observation. Real-time acquisition of water level data is of great significance for flood control and protection of people's lives and property. At present, manual reading of water level gauges is still the main way to obtain water level data, which is inefficient and difficult to achieve all-weather monitoring; automated water level measurement includes float-based methods, pressure-based methods, ultrasonic, radar wave-based methods, and fiber-optic water level gauges, but there are problems such as high installation cost, difficult operation and maintenance, and poor environmental adaptability. Water level monitoring based on image processing is a new hotspot in the field of water level monitoring technology. Many achievements have been made at home and abroad. Most of them are obtained by intercepting water level video images containing water gauges, grayscale conversion, median filtering, edge detection, and K-means clustering to obtain scale lines and then calculating water levels. Therefore, it is of great significance to invent a method for accurately identifying water levels. This not only helps to improve the accuracy and real-time performance of hydrological observations, but also can provide more reliable data support for key applications such as flood control and prevention. The development of new methods is expected to overcome the shortcomings of traditional methods and bring more efficient and reliable solutions to the field of water level monitoring. Summary of the invention

[0003] Purpose of the invention: The present invention aims to overcome the many limitations of traditional water level monitoring, such as sensitivity to viewing angle differences and scale digital recognition, and to provide a comprehensive and accurate water level identification solution that can provide a more reliable, economical and manageable water level identification solution for various application scenarios, and provide better services for the needs of flood control, resource management and other fields.

[0004] Technical solution: A water level recognition method based on image processing of the present invention comprises the following steps:

[0005] Step 1: Image perspective correction: Calculate the perspective transformation matrix through no less than four pairs of pixels, and project the original water level image collected by the corresponding camera position onto a new viewing plane through projection mapping;

[0006] Step 2: Build a semantic segmentation model for the water gauge area: Create a water gauge-non-water gauge label image, extract the water gauge area from the water level image after perspective correction based on the UNet++ neural network, and obtain the water gauge mask;

[0007] Step 3: Build a digital detection and recognition model: Use CRNN OCR technology, train the SVHN public dataset based on the CRNN model, detect the y coordinate of the bottom edge of the bounding box of the water level image after perspective correction that is closest to the digital scale on the water surface, combine the y coordinate with the water gauge mask to crop the image, and obtain the water gauge reading image corresponding to the current water level;

[0008] Step 4, scale reading: Perform pixel-level operations on the cropped water-scale image to get the water-scale matrix, divide the water-scale into E-type blocks and digital blocks, judge the rationality of the water-scale matrix based on prior knowledge, and determine the water-scale reading.

[0009] Furthermore, in step 1, image perspective correction requires no less than four pairs of pixels to meet the requirements of solving unknown numbers. The general transformation formula is shown in formula (1):

[0010]

[0011] x, y are the original image coordinates, corresponding to the transformed image coordinates (X', Y', Z').

[0012]

[0013] Furthermore, in step 2, the specific steps of constructing the water gauge area semantic segmentation model are as follows:

[0014] (2.1) Label the data, create a label image, outline the water gauge area in the water level image, and classify the outlined image into binary pixels, with the pixels in the water gauge area marked as 255 and the pixels in the non-water gauge area marked as 0;

[0015] (2.2) Training data: input the image and the corresponding label image as a data set into the UNet++ neural network for training to generate an image semantic segmentation model of the water gauge area and the non-water gauge area;

[0016] (2.3) Test, by inputting a new water level image, we can get the mask of the water gauge.

[0017] Furthermore, in step 3, the specific steps of constructing a digital detection and recognition model are as follows:

[0018] (3.1) Training data,Based on the SVHN dataset, CRNN OCR technology is used to train the digital detection and recognition model to detect and recognize numbers.

[0019] (3.2) Testing: by inputting the water level image after perspective correction, the numbers next to the scale are located and recognized;

[0020] (3.3) Filter the number closest to the water surface, and determine the y coordinate of the bottom edge of the bounding box of the recognized number and the digital scale value a of the water level digital scale closest to the water surface by extracting the y coordinate information of the bottom edge of the bounding box of the recognized number;

[0021] (3.4) Cropping: Cropping the image by combining the bounding box and the water gauge mask to obtain the water gauge reading image corresponding to the current water level.

[0022] Furthermore, in step 4, the specific steps of reading the scale are as follows:

[0023] (4.1) Partition the water gauge image and use the image matrix A composed of the grayscale values ​​of the corresponding pixels in the image to represent the water gauge:

[0024]

[0025] Sum by row i and its difference Δ i The image matrix is ​​divided into E-type block row vectors I i and the digital block row vector I i+1 , d i is the difference between adjacent elements of the row vector I, and the pixel-level width set D of the scale in the image is obtained:

[0026]

[0027] Δ i =|sum i -sum i+1 |,i=1,2,3,…n-1 Formula (5)

[0028] D=(d1,d2,…,d n'-1 ) Formula (6)

[0029] (4.2) Calculate the width of each area of ​​the water gauge. The water gauge is composed of alternating white E-shaped blocks and red digital blocks of equal width. The average width of the digital blocks is w, and the average width of the scale lines of the E-shaped blocks is b. The number of digital blocks is c1, and the number of scale lines of the E-shaped blocks is c2. Both are initially 0. When d i When the conditions are met, c1=c1+1, c2=c2+1, σ is an empirical number, 0<σ< <d i :

[0030]

[0031] (4.3) To judge the rationality of the element, the element should meet the following two conditions: the E-type block and the digital block d of the water level gauge 6i+1 The width is equal, and the five scale lines of the E-type block are d 6i+2 ,…,d6i+6 Equal width

[0032]

[0033] d 6i+2 ≈d 6i+3 ≈…≈d 6i+6

[0034] (4.4) Correct the unreasonable elements. In the scale matrix D, two adjacent elements representing digital blocks are replaced by d i and d j Indicates that when the tick marks are missing, it behaves as follows:

[0035]

[0036] If d i >w+b, that is, digital block d i If the width is unreasonable and contains one or more scale line widths, the scale matrix D is corrected: d i＝ d i -b, and insert d i+1 =b.

[0037] Based on the corrected scale data, the proportion of the scale section intercepted by the water surface to the digital block or E-type block c3 can be determined:

[0038]

[0039] (4.5) Reading: According to the number of digital blocks c1 and the number of scale lines c2 and c3 of the E-type block, the scale reading is determined, that is:

[0040] Water level = a-(5c1+c2+c3)

[0041] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0042] (1) The present invention processes images through image perspective correction, solves the problem under different perspectives, ensures robustness under different conditions, and is suitable for a variety of monitoring needs;

[0043] (2) The present invention adopts the UNet++ model to improve the accuracy of water-non-water body semantic segmentation, enhance the method's adaptability to complex scenes, and use OCR technology to accurately identify water level scale numbers to achieve the positioning of the scale closest to the water surface, thereby improving the accuracy and stability of water level recognition. It has the advantages of lower cost, simpler operation and maintenance, and stronger adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Flow chart of the present invention

[0045] Figure 2 Water level pictures before and after perspective correction

[0046] Figure 3 Water gauge mask

[0047] Figure 4 For reading detection and identification and water gauge cutting

[0048] Figure 5 Changes in training accuracy and loss rate DETAILED DESCRIPTION

[0049] The technical solution of the present invention is described in detail below, but the protection scope of the present invention is not limited to the embodiments.

[0050] like Figure 1 As shown, a water level recognition method based on image processing in this embodiment, by constructing a model for segmenting the water gauge and detecting and identifying scale numbers, outputs a corresponding cropped water gauge image, and accordingly realizes the partitioning and reading of the graphic matrix, specifically includes the following steps:

[0051] Step 1: Image perspective correction: Calculate the perspective transformation matrix through at least four pairs of pixels, and project the original water level image collected by the corresponding camera position to a new viewing plane through projection mapping. At least four pairs of pixels are required to meet the requirements of solving unknown numbers. The general transformation formula is shown in formula (1):

[0052]

[0053] x, y are the original image coordinates, corresponding to the transformed image coordinates (X', Y', Z').

[0054]

[0055] like Figure 2 The images are shown before and after perspective correction. It can be seen that some pixels are lost after perspective correction, but it has no effect on the water level reading.

[0056] Step 2: Build a semantic segmentation model for the water gauge area: Create a water gauge-non-water gauge label image, extract the water gauge area from the water level image after perspective correction based on the UNet++ neural network, and obtain the water gauge mask. The specific steps are as follows:

[0057] (2.1) Label the data, create a label image, outline the water gauge area in the water level image, and classify the outlined image into binary pixels, with the pixels in the water gauge area marked as 255 and the pixels in the non-water gauge area marked as 0;

[0058] (2.2) Training data: input the image and the corresponding label image as a data set into the UNet++ neural network for training to generate an image semantic segmentation model of the water gauge area and the non-water gauge area;

[0059] (2.3) Test, by inputting a new water level image, we can get the mask of the water gauge.

[0060] like Figure 3 The figure shows the water gauge mask image after segmentation, which identifies the water gauge more accurately.

[0061] Step 3: Build a digital detection and recognition model: Use CRNN OCR technology, train the SVHN public dataset based on the CRNN model, detect the y coordinate of the bottom edge of the bounding box of the water level image closest to the digital scale on the water surface after perspective correction, combine the y coordinate with the water gauge mask to crop the image, and obtain the water gauge reading image corresponding to the current water level. The specific steps are as follows:

[0062] (3.1) Training data: Based on the SVHN dataset, the CRNN OCR technology is used to train the digital detection and recognition model to detect and recognize numbers;

[0063] (3.2) Testing: by inputting the water level image after perspective correction, the numbers next to the scale are located and recognized;

[0064] (3.3) Filter the number closest to the water surface, and determine the y coordinate of the bottom edge of the bounding box of the recognized number and the digital scale value a of the water level digital scale closest to the water surface by extracting the y coordinate information of the bottom edge of the bounding box of the recognized number.

[0065] like Figure 4 The figure shows the detection and recognition of the 1-12 integer scales beside the water gauge, and the water gauge image is cropped according to the y coordinate at the bottom of the digital scale bounding box closest to the water level line.

[0066] Step 4, scale reading: Perform pixel-level operations on the cropped scale image to get the scale matrix, divide the scale into E-type blocks and digital blocks, judge the rationality of the scale matrix based on prior knowledge, and determine the scale reading. The specific steps are as follows:

[0067] (4.1) Partition the water gauge image and use the image matrix A composed of the grayscale values ​​of the corresponding pixels in the image to represent the water gauge:

[0068]

[0069] Sum by row i and its difference Δ i The image matrix is ​​divided into E-type block row vectors I i and the digital block row vector I i+1 , di is the difference between adjacent elements of the row vector I, and the pixel-level width set D of the scale in the image is obtained:

[0070]

[0071] Δ i =|sum i -sum i+1 |,i=1,2,3,…n-1 Formula (5)

[0072] D=(d1,d2,…,d n'-1 ) Formula (6)

[0073] (4.2) Calculate the width of each area of ​​the water gauge. The water gauge is composed of alternating white E-shaped blocks and red digital blocks of equal width. The average width of the digital blocks is w, and the average width of the scale lines of the E-shaped blocks is b. The number of digital blocks is c1, and the number of scale lines of the E-shaped blocks is c2. Both are initially 0. When d i When the conditions are met, c1=c1+1, c2=c2+1, σ is an empirical number, 0<σ<<d i :

[0074]

[0075] (4.3) Determine the rationality of the element. The element should meet the following two conditions: the E-type block and the digital block d of the water level gauge 6i+1 The width is equal, and the five scale lines of the E-type block are d 6i+2 ,…,d 6i+6 Equal width

[0076]

[0077] d 6i+2 ≈d 6i+3 ≈…≈d 6i+6

[0078] (4.4) Correct the unreasonable elements. In the scale matrix D, two adjacent elements representing digital blocks are replaced by d i and d j Indicates that when the tick marks are missing, it behaves as follows:

[0079]

[0080] If d i >w+b, that is, digital block d i If the width is unreasonable and contains one or more scale line widths, the scale matrix D is corrected: d i＝ d i -b, and insert d i+1 =b.

[0081] Based on the corrected scale data, the proportion of the scale section intercepted by the water surface to the digital block or E-type block c3 can be determined:

[0082]

[0083] (4.5) Reading: According to the number of digital blocks c1 and the number of scale lines c2 and c3 of the E-type block, the scale reading is determined, that is:

[0084] Water level = a-(5c1+c2+c3)

[0085] Table 1 shows the water level estimation results of 20 river channel images. It can be seen from Table 1 that the water level recognition method based on image processing proposed in the present invention can accurately estimate the water level and obtain good experimental results.

[0086] Table 1 Water level reading experimental results

[0087]

[0088]

[0089] This model training was iterated 300 times, achieving high accuracy and low loss rate. Figure 5 It can be seen that the iterative intersection-over-union ratio of the training set is stable at more than 70%, and the loss rate is stable at less than 1%. Therefore, segmenting the water gauge through the U-net++ neural network model can reflect the true situation of the water level.

[0090] To sum up, the present invention is based on the real-time feedback of video images from the actual camera, constructs a model for segmenting the water gauge and detecting and identifying scale numbers, outputs the corresponding cropped water gauge image, and realizes the partitioning and reading of the graphic matrix based on this, thereby realizing intelligent water level measurement, meeting the accuracy requirements of water gauge readings, and using automated water level measurement technology to solve the problems existing in traditional water level monitoring methods. It has the advantages of strong applicability, wide positioning range, high positioning accuracy, and real-time data feedback. It can be used for water level monitoring and management in various water environments such as reservoirs, rivers, and lakes. Real-time acquisition of water level data is of great significance to flood control and protection of people’s lives and property.

Claims

1. A water level recognition method based on image processing, characterized in that: This method is based on image processing technology, integrating deep learning models and OCR technology. It builds a water gauge regional segmentation model and a digital detection and recognition model through calibration data, and finally obtains the water gauge reading through pixel-level image processing. The specific steps are as follows: Step 1: Image perspective correction: Calculate the perspective transformation matrix through four pairs of pixels. Establish no less than four pairs of pixel points from the original viewing plane to the new viewing plane, calculate the perspective transformation matrix, and project the original water level image collected by the corresponding camera position to a new viewing plane through projection mapping; Step 2: Build a semantic segmentation model for the water gauge area: Create a water gauge-non-water gauge label image, extract the water gauge area from the water level image after perspective correction based on a deep learning neural network, and obtain a water gauge mask; Step 3: Build a digital detection and recognition model: Use OCR technology to train the Arabic numerals dataset based on a deep learning model, detect the y coordinate of the bottom edge of the bounding box of the water level image after perspective correction that is closest to the digital scale on the water surface, combine the y coordinate with the water gauge mask to crop the image, and obtain the water gauge reading image corresponding to the current water level; Step 4, scale reading: Perform pixel-level operations on the cropped water-scale image to get the water-scale matrix, divide the water-scale into E-type blocks and digital blocks, judge the rationality of the water-scale matrix based on prior knowledge, and determine the water-scale reading.

2. The water level recognition method based on image processing according to claim 1 is characterized in that: In step 1, image perspective correction: requires no less than four pairs of pixels to meet the requirements of solving unknown numbers. The general transformation formula is shown in formula (1): x, y are the original image coordinates, corresponding to the transformed image coordinates (X', Y', Z').

3. The water level recognition method based on image processing according to claim 1, characterized in that: In step 2, the specific steps of constructing the water gauge area semantic segmentation model are as follows: (3.1) Label the data, create a label image, outline the water gauge area in the water level image, and classify the outlined image into binary pixels, with the pixels in the water gauge area marked as 255 and the pixels in the non-water gauge area marked as 0; (3.2) Training data: input the image and the corresponding label image as a data set into the deep learning neural network for training to generate an image semantic segmentation model of the water gauge area and the non-water gauge area; (3.3) Test, by inputting a new water level image, we can get the mask of the water gauge.

4. The water level recognition method based on image processing according to claim 1, characterized in that: In step 3, the specific steps of constructing a digital detection and recognition model are as follows: (4.1) Training data: Based on the Arabic numerals dataset, OCR technology is used to train the numeral detection and recognition model to detect and recognize numerals; (4.2) Testing: by inputting the water level image after perspective correction, the numbers next to the scale are located and recognized; (4.3) Filter the number closest to the water surface, and determine the y coordinate of the bottom edge of the bounding box of the recognized number and the digital scale value a of the water level digital scale closest to the water surface by extracting the y coordinate information of the bottom edge of the bounding box of the recognized number; (4.4) Cropping: Cropping the image with the bounding box and the water gauge mask to obtain the water gauge reading image corresponding to the current water level.

5. The water level recognition method based on image processing according to claim 1, characterized in that: In step 4, the specific steps of scale reading are as follows: (5.1) Partition the water gauge image and use the image matrix A composed of the grayscale values ​​of the corresponding pixels in the image to represent the water gauge: Sum by row i and its difference Δ i The image matrix is ​​divided into E-type block row vectors I i and the digital block row vector I i+1 , d i is the difference between adjacent elements of the row vector I, and the pixel-level width set D of the scale in the image is obtained: Δ i = |sum i - sum i+1 |, i = 1, 2, 3, … n - 1 Equation (5) D=(d1,d2,…,d n'-1 ) Formula (6) (5.2) Calculate the width of each area of ​​the water gauge. The water gauge is composed of alternating white E-shaped blocks and red digital blocks of equal width. The average width of the digital blocks is w, and the average width of the scale lines of the E-shaped blocks is b. The number of digital blocks is c1, and the number of scale lines of the E-shaped blocks is c2. Both are initially 0. When d i When the conditions are met, c1=c1+1, c2=c2+1, σ is an empirical number, 0<σ<<d i : (5.3) To judge the rationality of an element, the element should satisfy the following two conditions: the E-type block and the digital block d of the water level gauge 6i+1 The width is equal, and the five scale lines of the E-type block are d 6i+2 ,…,d 6i+6 Equal width d 6i+2 ≈d 6i+3 ≈…≈d 6i+6 (5.4) Correct the unreasonable elements. In the scale matrix D, two adjacent elements representing digital blocks are replaced by d i and d j Indicates that when the tick marks are missing, it behaves as follows: If d i >w+b, that is, digital block d i If the width is unreasonable and contains one or more scale line widths, the scale matrix D is corrected: d i＝ d i -b, and insert d i+1 = b; Based on the corrected scale data, the proportion of the scale section intercepted by the water surface to the digital block or E-type block c3 can be determined: (5.5) Reading: According to the number of digital blocks c1 and the number of scale lines c2 and c3 of the E-type block, the scale reading is determined, that is: Water level = a-(5c1+c2+c3)

Images