A water level detection method with a water gauge linear scale with left and right positions

Abstract

An embodiment of the present invention provides a water level detection method for a water gauge linear scale with left and right positions. The method uses a deep learning target detection algorithm to intercept the approximate area where the water gauge is located, and simultaneously identifies the digital characters on the water gauge. Then, the intercepted approximate area of ​​the water gauge is perspective transformed to obtain the left and right half images of the water gauge. The left and right half images of the water gauge are grayscaled, and the processed images are Gaussian filtered and Canny edge detected. Then, the edge detected image is Hough transformed, and the number of linear scales in the water gauge after the transformation is accumulated. The accumulated value is judged and discussed according to different situations to obtain the accumulated value under different situations. The accumulated value is substituted into the corresponding formula to calculate the water level value, and finally compared with the recognized water gauge digital characters to verify the correctness of the water level value. The present invention compares the recognized water gauge digital with the calculated water level value to verify its accuracy, thereby improving the accuracy of the water gauge water level reading.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning and image processing, and is a method for detecting water levels on a water gauge linear scale with left and right positions. Background Art

[0002] Water level is a fundamental hydrological element of rivers, lakes, and reservoirs. Water level monitoring is a crucial indicator for monitoring water bodies such as rivers, lakes, and reservoirs, and is extremely important for water resource scheduling and flood control and irrigation. Traditional water level acquisition methods rely on manual reading of a water gauge to record the water level. However, this requires regular manual reading, resulting in low efficiency and automation, high labor intensity, and the potential for errors. Existing water level sensors include capacitive, ultrasonic, photoelectric encoding, and radar types, but are susceptible to environmental factors such as water quality and sediment content in actual use. These sensors also have high equipment and installation costs and present limitations in terms of use and routine maintenance.

[0003] Currently, many important water level observation points in China are equipped with video surveillance systems and standard water gauges. These cameras monitor water levels in real time, providing favorable conditions for water gauge and water level measurement based on video image processing. Image-based methods utilize image sensors, replacing the human eye, to capture water gauge images. Using image processing techniques, they detect the corresponding readings of the water level line, thereby automatically acquiring water level information. However, traditional water gauge scale recognition methods have certain limitations in practical applications and are unable to adapt well to changes in the external environment. Therefore, improving the accuracy of water gauge scale recognition and accurately sensing water level changes have become key concerns in current image-based water level monitoring.

[0004] Existing image-based water level detection methods still have significant limitations in measurement accuracy and reliability, as reflected in the following patents: 1) A method for detecting water level values ​​by identifying water gauge scale lines and water gauge characters. Patents include: 1) A method for recognizing water gauge E-shaped scales based on deep learning (publication number: CN110472636A). This method only recognizes the "E" character on the water gauge and does not calculate the scale value, resulting in relatively low calculation accuracy. 2) A water level measurement method (publication number: CN108759973A) directly uses the Hough line detection algorithm to detect lines on a local area image of a water gauge. This method is easily interfered with by water gauge numbers, significantly affecting the statistics of the line scale and thus the accuracy of the water gauge water level calculation. Summary of the Invention

[0005] The purpose of the present invention is to provide a water level detection method for a water gauge linear scale with left and right positions. The method performs perspective transformation on the collected picture to obtain the left and right half images of the water gauge, and then processes the transmission transformed picture. Thereafter, the water gauge image is scaled and counted in the left and right situations, and then the water level value of the water gauge is calculated using the statistical results. Finally, the water level value of the water gauge is checked according to the recognized number, thereby improving the detection accuracy and detection efficiency.

[0006] The technical solutions adopted in the present invention are as follows.

[0007] In one aspect, the present invention provides a method for detecting a water level on a water gauge linear scale with left and right positions, comprising:

[0008] Obtain a water gauge image sample at a certain moment from the real-time surveillance video;

[0009] The pre-trained YOLO-v5 object detection network is used to detect the approximate area of ​​the water gauge and recognize the digital characters of the water gauge. The approximate area image of the water gauge is obtained based on the detection results.

[0010] Manually mark the approximate area of ​​the water gauge image, and then use the position coordinates of the marked points to perform perspective transformation on the water gauge area to obtain the images of the left and right halves of the corrected water gauge.

[0011] Grayscale processing is performed on the left half and the right half of the water ruler to obtain grayscale images of the left half and the right half of the water ruler;

[0012] Performing Gaussian filtering on the grayscale images of the left and right halves of the water gauge to suppress noise and smooth the image;

[0013] The filtered images are processed using a Canny operator to detect the edge of the water gauge image, thereby obtaining two images of the left and right halves of the water gauge edge;

[0014] Hough transform is used to detect the linear scale of the right half of the water ruler edge image;

[0015] Draw the right half of the straight line scale diagram in the image domain based on the detected straight line scale results. These detected straight lines correspond to the "E" character on the water ruler.

[0016] Performing thinning processing on the right half of the drawn straight line graph, thinning the straight lines in the graph to 1 pixel;

[0017] According to the size of the digital characters on the water gauge, the integrity of the water gauge is judged, and the number of straight lines on the right half in different situations is accumulated using the corresponding mathematical relationship;

[0018] According to the relationship between the digital characters on the water gauge, the integrity of the water gauge is judged, and the number of straight lines on the right half in different situations is accumulated using the corresponding mathematical relationship;

[0019] The six sides of the "E" character on the water gauge correspond to the six scale lines in the straight line diagram. The cumulative number of straight lines is judged and discussed in different situations to obtain the cumulative value under different situations. The accumulated value is substituted into the corresponding formula to calculate the water level value.

[0020] According to the digital characters of the water gauge recognized in advance by the YOLO-v5 target detection network, the calculated water level value of the water gauge is checked. The water level value cannot be greater than the second-to-last smallest digital value recognized, because when the water level value is greater than this value, it means that the water level has passed the digital position. When the water level passes this position, the water gauge number cannot be recognized. Therefore, when the water gauge water level value is greater than the second-to-last smallest digital value recognized, the water level value calculation is unreasonable and needs to be recalculated. When the water gauge water level value is less than the second-to-last smallest digital value recognized, the water level value calculation is reasonable and the water gauge water level value is output.

[0021] The present invention utilizes a pre-trained YOLO-v5 object detection network to automatically detect the water gauge area and the digital characters on the water gauge, enabling rapid location of the approximate area where the water gauge is located and identification of the numbers on the water gauge. The specific training of YOLO-v5 can be based on existing technologies.

[0022] Optionally, label the acquired water gauge images using labelimg to obtain multiple water gauge sample datasets with labeled areas for the water gauge, background, and digital characters under different weather, lighting, and water flow conditions. These datasets are then divided into a validation set and a training set according to a certain ratio.

[0023] Optionally, the validation set and training set are used to pre-train the existing YOLO-v5 model framework according to the set weights to obtain a water gauge digit and water gauge area recognition training model based on the YOLO-v5 object detection network;

[0024] Optionally, the water ruler image to be detected is placed into the above-mentioned trained model for prediction, and the YOLO-v5 target detection network obtains the prediction box [X1, Y1, L1, W1] of the approximate area of ​​the water ruler and the prediction box [X2, Y2, L2, W2] of the digital characters of the water ruler, where X1 and Y1 are the coordinates of the upper left corner of the prediction box of the approximate area of ​​the water ruler, and L1 and W1 are the length and width of the prediction box of the approximate area of ​​the water ruler, respectively; X2 and Y2 are the coordinates of the upper left corner of the prediction box of the digital characters of the water ruler, and L2 and W2 are the length and width of the prediction box of the approximate area of ​​the water ruler.

[0025] Optionally, the method of performing perspective transformation on the image of the approximate area of ​​the water gauge includes:

[0026] Mark six coplanar but non-collinear marking points on the edge of the target water gauge and determine the coordinates of the six marking points in the world coordinate system: A: (X1, Y1), B: (X2, Y2), C: (X3, Y3), D: (X4, Y4), E: (X5, Y5), F: (X6, Y6);

[0027] Use the annotated target water gauge image as the annotation reference image and obtain the coordinates of the six marking points in the image pixel coordinate system: A: (x1, y1), B: (x2, y2), C: (x3, y3), D: (x4, y4), E: (x5, y5), F: (x6, y6)

[0028] According to the coordinates of the six identification points in the world coordinate system and the coordinates in the image pixel coordinate system, the coordinate transformation relationship between the calibration image and the template image is described using the perspective projection model between planes to calculate two sets of perspective transformation matrices T1 and T2:

[0029]

[0030] Where a~h are transformation coefficients. Multiply both sides of the equation by the denominator and match the six pairs of matching identification points to obtain the coordinates of two groups of four pairs of matching identification points. Substituting them into the above equation, we get the following matrix linear equation system:

[0031]

[0032] The calculated perspective transformation matrices T1 and T2 are:

[0033]

[0034] Using the principle that perspective transformation can maintain "linearity", the water ruler image is projected onto the standard water ruler image in the world coordinate system through the perspective transformation matrices T1 and T2 to obtain two images of the left and right halves of the water ruler.

[0035] Optionally, the method further includes: gray-scaling the left and right halves of the water gauge image, thereby reducing the amount of original image data and facilitating calculations during subsequent processing. The water gauge image adopts the RGB color mode and can be converted into a grayscale image with pixel values ​​of 0 to 255 after gray-scaling. The formula is as follows:

[0036]

[0037] Gaussian blur processing is performed on the left and right halves of the water ruler image after the grayscale processing, so that the image becomes blurred and smooth, reducing image noise and lowering the level of detail.

[0038] The filtered images are processed using the Canny operator to detect the edges of the water gauge image. In this process, a Gaussian filter is used to smooth the image to eliminate noise interference, and the image gradient is calculated. The image edge amplitude and angle are calculated based on the obtained gradient value. At the same time, non-maximum suppression is used to eliminate the stray response caused by edge detection. Finally, the double-threshold edge connection method is used to process and output the edge image of the water gauge in a binary form. This method is not easily affected by noise and can detect the actual edge of the image as much as possible, which is more critical for subsequent processing.

[0039] Perform Hough transform on the right half of the water ruler edge image, and store the transform result in the Hough transform accumulator. The Hough transform principle formula is as follows:

[0040] r=xcosθ+ysinθ (6)

[0041] The Hough transform uses the point-line duality relationship between the image space and the Hough parameter space to convert the line detection problem in the image space into the parameter space, performs cumulative statistics in the parameter space, and then detects the line in the image by finding the peak value of the accumulator.

[0042] Find the maximum point of the accumulated value in the Hough transform accumulator, record the point and clear it to zero, continue to find and record the maximum value of the next accumulator, and repeat the above operation until all the accumulated values ​​in the accumulator are cleared. These recorded points correspond to the straight lines detected in the corresponding image.

[0043] Draw a straight line graph of the right half of the water ruler edge in the image domain according to the zero point of the right half of the water ruler edge image, and these straight lines correspond to the "E" character detected on the water ruler;

[0044] Refine the right half of the straight line graph to reduce the width of the straight line in the graph to 1 pixel;

[0045] The identified numbers are judged:

[0046] 1) When the maximum value of the recognized number is 9, it means that the water gauge in the obtained photo is complete;

[0047] Performing horizontal grayscale value statistics on the thinned right half image, when the grayscale statistical value is greater than a set value, it is recorded as one straight line, and the total number of straight lines in the right half is recorded as N1;

[0048] 2) If the maximum value of the recognized digits is not 9, it means that the water gauge in the acquired photo is incomplete;

[0049] Subtract the maximum value identified from 9 and multiply by 6 to calculate the number of straight lines not obtained in the incomplete water gauge image, which is recorded as N2

[0050] Performing horizontal grayscale value statistics on the thinned right half image, when the grayscale statistical value is greater than a set value, it is recorded as one straight line, and the total number of straight lines in the right half is recorded as N3;

[0051] The total number of straight lines on the right half is N1=N2+N3

[0052] In the straight line graph, one "E" character corresponds to six straight lines. Since there is a certain positional relationship between the "E" characters and the numeric characters on the left and right sides of the water gauge, the number of straight lines can be counted and the water level value of the water gauge can be calculated using this relationship according to the corresponding expression.

[0053] Define {.} as the rounding function;

[0054] Determine whether the cumulative number of straight lines on the right half is divisible by 6:

[0055] 1) When the number of straight lines N1 represented by the letter E on the right half of the water gauge is not divisible by 6;

[0056] According to the number of straight lines on the right half of the scale that has been counted in advance, the water level value of the water gauge is calculated using the following formula, which is recorded as Z i

[0057]

[0058]

[0059] 2) When the number of lines N1 represented by the character E on the right half of the water gauge is divisible by 6, the following processing needs to be performed on the left half of the image:

[0060] According to the corresponding position relationship between the E character on the right half of the water ruler and the digital characters on the left half, the left half of the image corresponding to the position of the last "E" character on the right half of the image is intercepted;

[0061] Perform the above Hough transform on the intercepted image, and record the cumulative number of straight lines in the intercepted image as N4. Then calculate the water level value of the water gauge according to the following formula, which is recorded as Z i ;

[0062]

[0063] According to the digital characters of the water gauge recognized in advance by the YOLO-v5 target detection network, the calculated water level value of the water gauge is checked. If the water level value of the water gauge is greater than the second-to-last smallest digital value recognized, the water level value calculation is unreasonable and needs to be recalculated. If the water level value of the water gauge is less than the second-to-last smallest digital value recognized, the water level value calculation is reasonable and the water level value of the water gauge is output.

[0064] Beneficial effects

[0065] The present invention utilizes the YOLO-v5 target detection algorithm based on deep learning to intercept the approximate area where the water gauge is located, and simultaneously identifies the digital characters on the water gauge. Then, perspective transformation is performed on the intercepted approximate area of ​​the water gauge to obtain left and right half images of the water gauge. The left and right half images of the water gauge are grayscaled, and the processed images are subjected to Gaussian filtering and Canny edge detection. Thereafter, Hough transformation is performed on the edge detection image to accumulate the number of straight lines. The water level value of the water gauge is calculated using the positional relationship between the characters and the numbers on the left and right sides. Finally, the obtained water level value of the water gauge is verified using the identified digital characters on the water gauge. The present invention proposes a water level detection method for a water gauge linear scale with left and right positions. By using the YOLO-v5 target detection algorithm, the approximate area of ​​the water gauge can be accurately located. The water gauge image is divided into two images, the left and right halves, using Hough. The number of water gauge linear scale bars is accumulated according to different situations, which can effectively reduce the cumulative interference of numbers, stains, etc. on the water gauge scale. At the same time, the recognized water gauge numbers are compared with the calculated water level value to verify their accuracy, thereby improving the accuracy of the water gauge water level reading and having good environmental adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a flowchart of a specific implementation method of a water level detection method of a water gauge linear scale with left and right positions;

[0067] Figure 2 This is a flowchart of the YOLO-v5 target detection algorithm;

[0068] Figure 3 Schematic diagram of the approximate area of ​​the water gauge and the recognized digital characters detected by the YOLO-v5 target detection network in two cases;

[0069] Figure 4 Schematic diagrams of six identification points in a target reference image in two specific embodiments of the present invention;

[0070] Figure 5 Schematic diagrams of the left and right halves of the water ruler image obtained by perspective transformation in two specific embodiments of the present invention;

[0071] Figure 6Schematic diagrams of grayscale images of the left and right halves of a water ruler in two specific embodiments of the present invention;

[0072] Figure 7 Schematic diagrams of Gaussian blurred images of the left and right halves of a water ruler in two specific embodiments of the present invention;

[0073] Figure 8 Schematic diagrams of the left and right half images of the water ruler after Canny edge detection in two specific embodiments of the present invention;

[0074] Figure 9 Schematic diagrams of straight line graphs detected in the original image after Hough transform in specific embodiments of the present invention under two conditions;

[0075] Figure 10 Schematic diagrams of straight line graphs drawn in the image domain after Hough transform in specific embodiments of the present invention under two conditions; DETAILED DESCRIPTION

[0076] The following is a further description with reference to the accompanying drawings and specific embodiments.

[0077] Example 1

[0078] This embodiment introduces a water level detection method with a water gauge linear scale with left and right positions. Figure 1 As shown, two cases are discussed separately, including:

[0079] Obtain a water gauge image sample at a certain moment from the real-time surveillance video;

[0080] The pre-trained YOLO-v5 target detection network is used to detect the approximate area of ​​the water gauge and recognize the digital characters of the water gauge. The approximate area image of the water gauge is obtained according to the detection results. The program flow of the YOLO-v5 target detection algorithm is as follows Figure 2 As shown;

[0081] Manually mark the approximate area of ​​the water gauge image, and then use the position coordinates of the marked points to perform perspective transformation on the water gauge area to obtain the images of the left and right halves of the corrected water gauge.

[0082] Grayscale processing is performed on the left half and the right half of the water ruler to obtain grayscale images of the left half and the right half of the water ruler;

[0083] Performing Gaussian filtering on the grayscale images of the left and right halves of the water gauge to suppress noise and smooth the image;

[0084] The filtered images are processed using a Canny operator to detect the edge of the water gauge image, thereby obtaining two images of the left and right halves of the water gauge edge;

[0085] Hough transform is used to detect the linear scale of the right half of the water ruler edge image;

[0086] Draw the right half of the straight line scale diagram in the image domain based on the detected straight line scale results. These detected straight lines correspond to the "E" character on the water ruler.

[0087] Performing thinning processing on the right half of the drawn straight line graph, thinning the straight lines in the graph to 1 pixel;

[0088] According to the size of the digital characters on the water gauge, the integrity of the water gauge is judged, and the number of straight lines on the right half in different situations is accumulated using the corresponding mathematical relationship;

[0089] According to the relationship between the digital characters on the water gauge, the integrity of the water gauge is judged, and the number of straight lines on the right half in different situations is accumulated using the corresponding mathematical relationship;

[0090] The six sides of the "E" character on the water gauge correspond to the six scale lines in the straight line diagram. The cumulative number of straight lines is judged and discussed in different situations to obtain the cumulative value under different situations. The accumulated value is substituted into the corresponding formula to calculate the water level value.

[0091] According to the digital characters of the water gauge recognized in advance by the YOLO-v5 target detection network, the calculated water level value of the water gauge is checked. The water level value cannot be greater than the second-to-last smallest digital value recognized, because when the water level value is greater than this value, it means that the water level has passed the digital position. When the water level passes this position, the water gauge number cannot be recognized. Therefore, when the water gauge water level value is greater than the second-to-last smallest digital value recognized, the water level value calculation is unreasonable and needs to be recalculated. When the water gauge water level value is less than the second-to-last smallest digital value recognized, the water level value calculation is reasonable and the water gauge water level value is output.

[0092] This embodiment provides a method for detecting water level on a water gauge linear scale with left and right positions, specifically involving the following contents.

[0093] 1. Preliminarily determine the approximate area of ​​the water gauge and identify the water gauge digital characters

[0094] The present invention utilizes a pre-trained YOLO-v5 object detection network to automatically detect the water gauge area and the digital characters on the water gauge, enabling rapid location of the approximate area where the water gauge is located and identification of the numbers on the water gauge. The specific training of YOLO-v5 can be based on existing technologies.

[0095] Optionally, label the acquired water gauge images using labelimg to obtain multiple water gauge sample datasets with labeled areas for the water gauge, background, and digital characters under different weather, lighting, and water flow conditions. These datasets are then divided into a validation set and a training set according to a certain ratio.

[0096] Optionally, the validation set and training set are used to pre-train the existing YOLO-v5 model framework according to the set weights to obtain a water gauge digit and water gauge area recognition training model based on the YOLO-v5 object detection network;

[0097] Optionally, the water ruler image to be detected is placed into the above-mentioned trained model for prediction, and the YOLO-v5 target detection network obtains the prediction box [X1, Y1, L1, W1] of the approximate area of ​​the water ruler and the prediction box [X2, Y2, L2, W2] of the digital characters of the water ruler, where X1 and Y1 are the coordinates of the upper left corner of the prediction box of the approximate area of ​​the water ruler, and L1 and W1 are the length and width of the prediction box of the approximate area of ​​the water ruler, respectively; X2 and Y2 are the coordinates of the upper left corner of the prediction box of the digital characters of the water ruler, and L2 and W2 are the length and width of the prediction box of the approximate area of ​​the water ruler.

[0098] 2. Obtain the left and right half images of the water ruler through transmission transformation

[0099] For the water gauge target to be detected, before the actual detection, the YOLO-v5 target detection network needs to intercept the approximate area of ​​the water gauge as a reference image, mark the corresponding position, and obtain the reference marking coordinates required for actual detection. The markings for Case 1 and Case 2 are as follows Figure 3 shown.

[0100] 2.1 First, mark the target water level. Six marking points need to be marked on the edge of the water level, such as Figure 3 As shown, it is necessary to pay attention to the following when marking: in order to avoid the influence of the water gauge character color on the marked points, the color of the marking points needs to be distinguished from the color of the water gauge characters. For example: when the color of the water gauge is blue, the color of the 6 marking points can be set to red; when the color of the water gauge is red, the color of the 6 marking points can be set to blue.

[0101] 2.2 Case 1:

[0102] The left pair of markers and the middle pair of markers are grouped together, and the middle pair of markers and the right pair of markers are grouped together to form two groups of four markers. According to the perspective transformation formula and the calculation principle of the transmission coefficient, the coordinate positions of the first group of four markers in the world coordinate system are set to (0,0), (0,400), (40,400), (40,0); according to the coordinate positions of the four markers of the annotated reference image in the image coordinate system are (594,65), (585,542), (644,542), (655,66); according to the above formulas (3) and (4), the perspective projection matrix T1 can be calculated as:

[0103]

[0104] Perform the above operation on the second group of four marker points, setting the coordinate positions of the four marker points in the world coordinate system to (0,0), (0,400), (40,400), (40,0); according to the coordinate positions of the four marker points in the annotated reference image in the image coordinate system to (655,66), (644,542), (703,542), (715,67); according to the above formulas (2) and (3), the perspective projection matrix T2 can be calculated as:

[0105]

[0106] Case 2:

[0107] The left pair of markers and the middle pair of markers are grouped together, and the middle pair of markers and the right pair of markers are grouped together to form two groups of four markers. According to the perspective transformation formula and the calculation principle of the transmission coefficient, the coordinate positions of the first group of four markers in the world coordinate system are set to (0,0), (0,200), (40,200), (40,0); according to the coordinate positions of the four markers of the annotated reference image in the image coordinate system are (42,71), (39,308), (97,308), (102,72); according to the above formulas (3) and (4), the perspective projection matrix T1 can be calculated as:

[0108]

[0109] Perform the above operation on the second group of four marker points, setting the coordinate positions of the four marker points in the world coordinate system to (0,0), (0,200), (40,200), (40,0); according to the coordinate positions of the four marker points in the annotated reference image in the image coordinate system to (102,72), (97,308), (155,310), (162,73); according to the above formulas (3) and (4), the perspective projection matrix T2 can be calculated as:

[0110]

[0111] Using the principle that perspective transformation can maintain "linearity", the water ruler image is projected onto the standard water ruler image in the world coordinate system through the perspective transformation matrix T1, T2, and the left and right half images of the water ruler are obtained. Case 1 and Case 2 are as follows Figure 4 shown.

[0112] 3. Preprocessing of the left and right half images of the water ruler

[0113] 3.1 This embodiment performs grayscale processing on the left and right half images of the water ruler. The results of Case 1 and Case 2 are as follows: Figure 5 shown.

[0114] After processing, the amount of original image data is reduced, which is beneficial for subsequent calculations. After grayscale processing, the three-channel image can be converted into a grayscale image with pixel values ​​of 0 to 255. The formula is as follows:

[0115]

[0116] 3.2 Gaussian blur operation is performed on the left and right halves of the grayscale ruler image to make the image blurred and smooth, reduce image noise and reduce the level of detail. In case 2, it is necessary to intercept the left half of the image corresponding to the position of the last "E" character on the right half of the ruler according to the corresponding position relationship between the E character on the right half of the ruler and the digital characters on the left half. The processing results of case 1 and case 2 are as follows: Figure 6 As shown;

[0117] 3.3 The filtered images are processed using the Canny operator to detect the edge of the water gauge image, and two images of the left and right halves of the water gauge edge are obtained. The results of case 1 and case 2 are as follows: Figure 7 shown.

[0118] The filtered images are processed using the Canny operator to detect the edges of the water gauge image. In this process, a Gaussian filter is used to smooth the image to eliminate noise interference, and the image gradient is calculated. The image edge amplitude and angle are calculated based on the obtained gradient value. At the same time, non-maximum suppression is used to eliminate the stray response caused by edge detection. Finally, the image is processed using a double-threshold edge connection method to output the edge image of the water gauge in a binary form.

[0119] 4. Determination of water level value of water gauge

[0120] The right half of the water gauge edge image is transformed by Hough transform. The straight lines detected in the original image after transformation in case 1 and case 2 are as follows: Figure 8As shown, the result of the transformation is stored in the accumulator of the Hough transform. The principle formula of the Hough transform is as follows:

[0121] r=xcosθ+ysinθ

[0122] The Hough transform uses the point-line duality relationship between the image space and the Hough parameter space to convert the line detection problem in the image space into the parameter space, performs cumulative statistics in the parameter space, and then detects the line in the image by finding the peak value of the accumulator.

[0123] Find the maximum point of the accumulated value in the Hough transform accumulator, record the point and clear it to zero, continue to find and record the maximum value of the next accumulator, and repeat the above operation until all the accumulated values ​​in the accumulator are cleared. These recorded points correspond to the straight lines detected in the corresponding image.

[0124] According to the zero point of the right half of the water ruler edge image, a straight line graph of the right half is drawn in the image domain. The results of case 1 and case 2 are as follows: Figure 9 As shown, these straight lines correspond to the detected E characters on the water gauge;

[0125] Performing thinning processing on the right half of the drawn straight line graph, thinning the straight lines in the graph to 1 pixel;

[0126] The identified numbers are judged:

[0127] 1) When the maximum value of the recognized number is 9, it means that the water gauge in the obtained photo is complete;

[0128] Performing horizontal grayscale value statistics on the thinned right half image, when the grayscale statistical value is greater than a set value, it is recorded as one straight line, and the total number of straight lines in the right half is recorded as N1;

[0129] 2) If the maximum value of the recognized digits is not 9, it means that the water gauge in the acquired photo is incomplete;

[0130] Subtract the maximum value identified from 9 and multiply by 6 to calculate the number of straight lines not obtained in the incomplete water gauge image, which is recorded as N2

[0131] Performing horizontal grayscale value statistics on the thinned right half image, when the grayscale statistical value is greater than a set value, it is recorded as one straight line, and the total number of straight lines in the right half is recorded as N3;

[0132] The total number of straight lines on the right half is N1=N2+N3

[0133] In the example shown in the figure, the total number of straight lines in the right half of Case 1 and Case 2 are N1=40 and N1=42 respectively.

[0134] In the straight line graph, one E character corresponds to six straight lines. Since there is a certain positional relationship between the E characters and the numeric characters on the left and right sides of the water gauge, the number of straight lines can be counted and the water level value of the water gauge can be calculated using the corresponding expression using this relationship.

[0135] Define {.} as the rounding function;

[0136] Determine whether the cumulative number of straight lines on the right half is divisible by 6:

[0137] Case 1:

[0138] 1) When the number of straight lines N1 represented by the letter E on the right half of the water gauge is not divisible by 6;

[0139] According to the number of straight lines on the right half of the scale that has been counted in advance, the water level value of the water gauge is calculated using the following formula, which is recorded as Z i

[0140]

[0141] In the example shown in the figure, substituting the accumulated N1 in case 1 into the above formula, we can get Z i =31.667

[0142] Case 2:

[0143] 2) When the number of lines N1 represented by the character E on the right half of the water gauge is divisible by 6, the following processing needs to be performed on the left half of the image:

[0144] According to the corresponding position relationship between the E character on the right half of the water ruler and the digital characters on the left half, the left half of the image corresponding to the position of the last "E" character on the right half of the image is intercepted;

[0145] Perform the above Hough transform on the intercepted image, and record the cumulative number of straight lines in the intercepted image as N4. Then calculate the water level value of the water gauge according to the following formula, which is recorded as Z i ;

[0146]

[0147] In the example shown in the figure, the cumulative N4 in case 2 is 2

[0148] In the example shown in the figure, the accumulated N1 and N4 in case 2 are substituted into the above formula to obtain Z i =28.333

[0149] According to the digital characters of the water gauge recognized in advance by the YOLO-v5 target detection network, the calculated water level value of the water gauge is checked. The water level value cannot be greater than the second-to-last smallest digital value recognized, because when the water level value is greater than this value, it means that the water level has passed the digital position. When the water level passes this position, the water gauge number cannot be recognized. Therefore, when the water gauge water level value is greater than the second-to-last smallest digital value recognized, the water level value calculation is unreasonable and needs to be recalculated. When the water gauge water level value is less than the second-to-last smallest digital value recognized, the water level value calculation is reasonable and the water gauge water level value is output.

[0150] In summary, the present invention has the following advantages and improvements:

[0151] 1) Strong environmental adaptability. Using the YOLO-v5 target detection algorithm, the approximate area of ​​the water gauge can be accurately located and the number of linear scales of the water gauge can be accumulated according to the situation. This can effectively reduce the cumulative interference of numbers, stains, etc. on the water gauge scale, and has a certain adaptability to water gauge water level detection in complex environments;

[0152] 2) High detection accuracy: The present invention verifies the accuracy of the water level value of the water gauge by comparing the size relationship between the water gauge digit and the water gauge value, thereby improving the detection accuracy of the water gauge water level.

[0153] 3) The water gauge has strong adaptability. It can effectively count the number of scale bars on both complete and incomplete water gauges, and then accurately calculate the water level value of the water gauge.

[0154] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which are all protected by the present invention.

Claims

1. A water level detection method with a water gauge linear scale divided into left and right positions, characterized in that: The following steps are involved: 1) Obtain a water gauge image sample at a certain moment from the real-time surveillance video; 2) Use the pre-trained YOLO-v5 object detection network to detect the approximate area of ​​the water gauge and recognize the digital characters on the water gauge. Based on the detection results, an image of the approximate area of ​​the water gauge is obtained; 3) Manually annotate the approximate area of ​​the water gauge image, and then use the position coordinates of the annotated points to perform a perspective transformation on the water gauge area to obtain the images of the left and right halves of the corrected water gauge; 4) performing image preprocessing on the left and right halves of the water ruler images respectively; 5) Use Hough transform to detect the linear scale of the right half of the water ruler edge image; 6) Draw the right half of the linear scale diagram in the image domain based on the detected linear scale results. These detected straight lines correspond to the "E" character on the water ruler. 7) performing a thinning process on the right half of the drawn linear scale graph, and thinning the width of the straight lines in the graph to 1 pixel; 8) judging the integrity of the water gauge according to the size of the digital characters on the water gauge, and accumulating the number of straight lines on the right half in different situations using the corresponding mathematical relationship; 9) The six sides of the "E" character on the water gauge correspond to the six scale lines in the straight line diagram. The cumulative number of lines is determined and discussed in different situations to obtain the cumulative value under different circumstances. The accumulated value is then substituted into the corresponding formula to calculate the water level value. 10) Compare the recognized water gauge number with the water level value to verify the accuracy of the calculated water gauge water level value; The six sides of the "E" character on the water gauge correspond to the six scale lines in the straight line diagram. The cumulative number of straight lines is judged and discussed in different situations to obtain the cumulative values ​​in different situations. The values ​​are substituted into the corresponding formulas to calculate the water level values, including: Refine the right half of the straight line graph to reduce the width of the straight line in the graph to 1 pixel; The identified numbers are judged: 1) When the maximum value of the recognized number is 9, it means that the water gauge in the obtained photo is complete; Performing horizontal grayscale value statistics on the thinned right half image, when the grayscale statistical value is greater than a set value, it is recorded as one straight line, and the total number of straight lines in the right half is recorded as N1; 2) If the maximum value of the recognized digits is not 9, it means that the water gauge in the acquired photo is incomplete; Subtract the maximum value identified from 9 and multiply by 6 to calculate the number of straight lines not obtained in the incomplete water gauge image, which is recorded as N2; Performing horizontal grayscale value statistics on the thinned right half image, when the grayscale statistical value is greater than a set value, it is recorded as one straight line, and the total number of straight lines in the right half is recorded as N3; The total number of straight lines on the right half is N1=N2+N3; The six sides of the "E" character on the water gauge correspond to the six scale lines in the linear graph. Since the "E" characters on the left and right sides of the water gauge have a certain positional relationship with the numeric characters, the number of lines can be counted and this relationship can be used to calculate the water level value of the water gauge according to the corresponding expression. Define {.} as the rounding function; Determine whether the cumulative number of straight lines on the right half is divisible by 6: 1) When the number of straight lines N1 represented by the "E" character on the right half of the water gauge is not divisible by 6; According to the number of straight lines on the right half of the scale that has been counted in advance, the water level value of the water gauge is calculated using the following formula, which is recorded as Z i 2) When the number of lines N1 represented by the "E" character on the right half of the water gauge is divisible by 6, the following processing needs to be performed on the left half of the image: According to the corresponding position relationship between the "E" character on the right half of the water ruler and the digital characters on the left half, the left half of the image corresponding to the position of the last "E" character on the right half of the image is intercepted; Perform the above Hough transform on the intercepted image, and record the cumulative number of straight lines in the intercepted image as N4. Then calculate the water level value of the water gauge according to the following formula, which is recorded as Z i ; 2. The method according to claim 1, wherein: The process of pre-training the YOLO-v5 object detection network includes: 2-1) Annotate the acquired water gauge images to obtain multiple water gauge sample datasets with annotated areas for the water gauge, background, and digital characters under different weather, lighting, and water flow conditions. Divide the dataset into a validation set and a training set according to a certain ratio. 2-2) Using the above validation and training sets, pre-train the existing YOLO-v5 model framework according to the set weights to obtain a water gauge digit and water gauge area recognition training model based on the YOLO-v5 object detection network; 2-3) The water ruler image to be detected is placed into the above-mentioned trained model for prediction. The YOLO-v5 target detection network obtains the prediction box [X1, Y1, L1, W1] of the approximate area of ​​the water ruler and the prediction box [X2, Y2, L2, W2] of the digital characters of the water ruler, where X1 and Y1 are the coordinates of the upper left corner of the prediction box of the approximate area of ​​the water ruler, and L1 and W1 are the length and width of the prediction box of the approximate area of ​​the water ruler, respectively. X2 and Y2 are the coordinates of the upper left corner of the prediction box of the digital characters of the water ruler, and L2 and W2 are the length and width of the prediction box of the approximate area of ​​the water ruler.

3. The method according to claim 1, wherein include: Using the principle that perspective transformation can maintain "linearity", the water ruler image is projected onto the standard water ruler image in the world coordinate system through the perspective transformation matrices T1 and T2 to obtain the left and right half images of the water ruler.

4. The method according to claim 1, wherein the image preprocessing process is as follows: 4-1) grayscale processing is performed on the left and right halves of the water gauge image. The water gauge image adopts the RGB color mode and can be converted into a grayscale image with a pixel value of 0 to 255 after grayscale processing; 4-2) performing a Gaussian filter operation on the grayscale images of the left and right halves of the water gauge to suppress noise and smooth the images, thereby obtaining smoothed filtered images of the left and right halves of the water gauge; 4-3) The filtered images are processed using the Canny operator to detect the edge of the water gauge image, thereby obtaining the left and right half images of the water gauge edge.

5. The method according to claim 1, wherein: Performing Hough transform on the right half image of the water ruler edge; Hough transform uses the point-line duality relationship between image space and Hough parameter space to convert the line detection problem in image space to parameter space, performs cumulative statistics in parameter space, and then detects the line in the image by finding the peak value of the accumulator; Find the maximum point of the accumulated value in the Hough transform accumulator, record the point and clear it to zero, continue to find and record the maximum value of the next accumulator, and repeat the above operation until all the accumulated values ​​in the accumulator are cleared. These recorded points correspond to the straight lines detected in the corresponding image; According to the zero point of the right half of the water ruler edge image, a straight line graph of the right half is drawn in the image domain, and these straight lines correspond to the "E" character detected on the water ruler.

6. The method according to claim 1, wherein: According to the digital characters of the water gauge recognized in advance by the YOLO-v5 target detection network, the calculated water level value of the water gauge is checked. The water level value cannot be greater than the second-to-last smallest digital value recognized, because when the water level value is greater than this value, it means that the water level has passed the digital position. When the water level passes this position, the water gauge number cannot be recognized. Therefore, when the water gauge water level value is greater than the second-to-last smallest digital value recognized, the water level value calculation is unreasonable and needs to be recalculated. When the water gauge water level value is less than the second-to-last smallest digital value recognized, the water level value calculation is reasonable and the water gauge water level value is output.

Images