A method for realizing water level recognition by virtual water gauge through image recognition technology
Through image recognition technology and CNN model, combined with virtual water ruler, the existing water level recognition ability and water ruler installation requirements are solved, and high-precision water level recognition in reflective and night infrared modes are achieved. The error is within 1cm, meeting the actual monitoring needs.
Patent Information
- Application Number
- CN202110807448.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-16
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2041-07-16
AI Technical Summary
The existing water level recognition methods have poor recognition capabilities in complex background, reflective, and night infrared modes, and rely on the installation requirements of the physical water ruler, making it difficult to achieve high-precision recognition.
Through image recognition technology, the water level scene image is obtained using the camera, the water surface position is marked through multi-sample learning, combined with the convolutional neural network (CNN) training model, and the virtual water ruler is set to achieve water level recognition in complex backgrounds, with strong anti-interference ability , adapt to any inclination angle.
It improves the accuracy of water level recognition, solves the problem of difficulty in installing the water ruler, and realizes high-precision recognition in the case of reflective and faint visible, with an error of less than 1cm, meeting the actual monitoring needs.
Smart Images

Figure CN113516081B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of deep learning and water level monitoring, and in particular to a method for realizing water level recognition by a virtual water gauge through image recognition technology. Background Art
[0002] Existing water level identification methods are mainly based on
[0003] 1. The edge segmentation algorithm can be used to recognize the scale of the physical water gauge. For example, grayscale images are used for image processing, and then the boundary recognition of the water level and water gauge is realized through the edge segmentation algorithm. However, the segmentation effect is easily affected by complex background factors such as lighting and shoreline, so it is impossible to simply and effectively segment the water level. In addition, different degrees of camera tilt have a greater impact on the recognition accuracy.
[0004] 2. The scale lines of the water ruler are extracted through projection analysis, K-means cluster analysis, etc. to obtain the water ruler readings, and the characteristic values of the visible physical water ruler scale value "E" and the numbers on the water ruler are identified to identify the water level. This method cannot accurately identify the water level in reflective and weakly visible conditions (night vision infrared mode).
[0005] In summary, the existing water gauge image recognition methods have the following problems:
[0006] 1. Water level identification usually relies on a physical water gauge, which has installation requirements for the position and inclination angle of the water gauge and camera. It is difficult to install in actual work (for safety reasons, cameras cannot be installed on the dam, etc., which will damage the dam body), resulting in no guarantee of identification accuracy.
[0007] 2. The general edge segmentation algorithm is usually used to identify the water level. The anti-interference ability is weak, resulting in poor recognition ability in scenes such as reflection (night fill light), night (infrared mode), and complex background. Summary of the invention
[0008] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide a method for realizing water level recognition by a virtual water gauge through image recognition technology. The method has strong anti-interference ability and can recognize the water level of any complex background under reflective and weakly visible (infrared mode or night fill light) conditions; at the same time, it does not rely on the installation tilt angle and can achieve a high-precision recognition effect at any tilt angle.
[0009] The purpose of the present invention is achieved through the following technical solutions:
[0010] A method for realizing water level recognition by virtual water gauge through image recognition technology, comprising the following steps: S1, selecting a scene: determining the intersection of the water surface and a characteristic object, the characteristic object includes a water gauge or other characteristic objects such as a wall, and the camera can be guaranteed that the maximum water surface and the lowest water surface intersect with the characteristic object (which can be achieved by enlarging and reducing the camera focal length). Special note: After the camera selects a scene, there will be no change in angle and resolution.
[0011] S2, virtual water level setting: After selecting the scene, the feature object is obtained (the feature object refers to the object that intersects with the water surface). According to the position of the feature object and the scene, it is mainly determined whether it is a horizontal or vertical virtual water level, and the central axis of the virtual water level is determined. After the central axis is obtained, the highest water level direction K is set. If it is a vertical ('|') virtual water level, the direction of the highest water level is on the top, and if it is a horizontal ('-') virtual water level, the direction of the highest water level is on the left or right. After obtaining the direction of the highest water level, set the baseline (the pixel distance between the direction on the picture and the highest water level) and set the highest water level value. And set the stride according to the required water level accuracy (the stride is set to 0.1m, 0.5m, 1m according to actual experience), set the water level of each scale downward from the highest water level, and after setting several scales, directly calculate the water level value below through RNN (recurrent neural network) until the end of the virtual water level, that is, the lowest water level value. Through the above operations, we can obtain the array R[(V1, R1(Rx1, Ry1)),(V2, R2(Rx2, Ry2)),…,(Vn, Rn(Rxn, Ryn))] of virtual water scale scale values and corresponding pixel values (V1 is the highest water level value, R1 is the corresponding pixel point, Vn is the lowest water level value, Rn is the corresponding pixel value).
[0012] S3, water level labeling: extract frames from the camera’s video stream to obtain 24-hour scene images, select images from different time periods and images under different weather conditions (at least 100 images) and import them into the training data set. Set the intersection point of the virtual water gauge and the water level to complete the water level labeling and obtain the water level labeling data set D0 and labeling set A0.
[0013] S4, model training: First, crop the image training data set D0 (for the sake of example, the image pixel is 1920*1080), according to the first pixel value (width, height) (Rx1, Ry1) in the virtual water ruler array R[(V1, R1(Rx1, Ry1)), (V2, R2(Rx2, Ry2)), ..., (Vn, Rn(Rxn, Ryn))] obtained earlier, and also know whether the virtual water ruler is horizontal or vertical according to the highest water level K. If the virtual water ruler is horizontal ('-'), then the point at the upper left corner of the cropped position is (0, Ry1-25), and the point at the lower right corner is (1920, Ry1+25). If the virtual water gauge is vertical ('|'), then the point at the upper left corner of the clipping position is (Rx1-25, 0), and the point at the lower right corner is (Rx1+25, 1080). The clipping data set D1 is obtained by clipping through this rectangle. Then, the annotation set A0 of each image is transformed into a rectangle to obtain the annotation set A1. The rectangle transformation rule is based on the annotation point (x, y). The point at the upper left corner is (x-25, y-25), and the point at the lower right corner is (x+25, y+25). Finally, CNN (convolutional neural network) is used to train the model of the annotated data set D1 and the annotation set A1 to obtain the object detection model P1 of the water level area.
[0014] S5, calculate the water level through the model: use the picture C1 of the water level recognition scene to calculate the water level. First, through the cropping step of S4, get the picture C2, use the object detection model P1 of the water level area of C2 to calculate the water level area B[(Bx1, By1), (Bx2, By2)] (upper left point B1, lower right point B2). Then combine the virtual water ruler array R[(V1, R1(Rx1, Ry1)), (V2, R2(Rx2, Ry2)), ..., (Vn, Rn(Rxn, Ryn))] (V1 is the maximum water level value, R1 is the corresponding pixel point) and the highest water level direction K of the previous virtual water ruler: if it is on the top, then it is the vertical water ruler ('|'), calculate the water level position By = By1 + (By2-By1) / 2. Substitute By into the virtual water gauge R, and calculate from the high to the low. If By is between Ry1 and Ry2, then the water level value = V2 + (V1-V2) * ((Ry2-By) / (Ry2-Ry1)); if it is on the left or right, then it is a horizontal water gauge ('-'), and the water level position Bx = Bx1 + (Bx2-Bx1) / 2 is calculated. Substitute Bx into the virtual water gauge R, and calculate from the high to the low. If Bx is between Rx1 and Rx2, then the water level value = V2 + (V1-V2) * ((Rx2-Bx) / (Rx2-Rx1)); the final water level value is obtained.
[0015] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention effectively solves the problem of difficulty in water gauge recognition under complex background conditions through a deep learning method, and improves the recognition accuracy. (2) The present invention can identify the water level through a virtual water gauge in the absence of a water gauge, solving the problem that the water gauge is difficult to install and observe; (3) The water gauge reading accuracy of the present invention is high. When set with a step of 0.1m, the maximum error of the water level calculation result is within 1cm, which meets the actual water level monitoring needs. As long as the resolution supports it, the step can be changed to 0.01m if higher accuracy is required. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 It is the overall flow chart of the present invention;
[0017] Figure 2 It is a schematic diagram of the configuration of the longitudinal virtual water gauge of the present invention;
[0018] Figure 3 It is a schematic diagram of water level marking of the present invention;
[0019] Figure 4 This is a schematic diagram of image clipping according to the present invention;
[0020] Figure 5 This is a schematic diagram of the verification result F1-score of the present invention;
[0021] Figure 6 This is a schematic diagram of the test result observation of the present invention; DETAILED DESCRIPTION
[0022] The present invention is further described in detail below in conjunction with implementation cases and drawings, but the implementation methods of the present invention are not limited thereto.
[0023] The basic principle of the present invention is to use a camera to obtain a water level scene image, mark the water surface position through multi-sample learning, and obtain a model through CNN (convolutional neural network) training. The current water level is calculated by identifying the pixel position of the water surface feature and combining it with the pixel position of the virtual water gauge set in the scene image. The method has strong anti-interference ability and can identify the water level of any complex background under reflective and weakly visible (infrared mode or night fill light) conditions; at the same time, it does not rely on the installation tilt angle, and can achieve a high-precision recognition effect at any tilt angle.
[0024] The specific implementation process is as follows: Figures 1 to 6 As shown, the following steps are included:
[0025] (1) Video access: Access the video by connecting to the camera stream address or national standard protocol.
[0026] (2) Select the scene: determine the intersection of the water surface and the characteristic object. The characteristic object includes other characteristic objects such as water gauges or walls. The camera can ensure that the maximum water surface and the lowest water surface intersect with the characteristic object (this can be achieved by zooming in and out the camera focal length). Special note: After the camera selects the scene, the angle and resolution will not change.
[0027] (3) Setting of virtual water level: After selecting a scene, we get the characteristic object (characteristic object refers to the object that intersects with the water surface). According to the position of the characteristic object and the scene, we determine whether it is a horizontal or vertical virtual water level and the central axis of the virtual water level for the convenience of observation and measurement. After we have the central axis, we set the highest water level K. If it is a vertical ('|') virtual water level, the highest water level is at the top. If it is a horizontal ('-') virtual water level, the highest water level is at the left or right. After we get the highest water level, we set the baseline (the pixel distance between the orientation on the image and the highest water level) and the highest water level value. We set the stride according to the required water level accuracy (the stride is set to 0.1m, 0.5m, 1m according to actual experience). We set the water level of each scale downward from the highest water level. After setting several scales, we directly calculate the water level value below through RNN (recurrent neural network) until the end of the virtual water level, which is the lowest water level value. Through the above operations, we can obtain the array R[(V1, R1(Rx1, Ry1)),(V2, R2(Rx2, Ry2)),…,(Vn, Rn(Rxn, Ryn))] of virtual water scale scale values and corresponding pixel values (V1 is the highest water level value, R1 is the corresponding pixel point, Vn is the lowest water level value, Rn is the corresponding pixel value).
[0028] (4) Dataset collection: By extracting frames from the camera’s video stream, we obtain pictures of 24-hour scenes. We select pictures from different time periods, different weather conditions, and different camera modes (at least 100 pictures) to obtain the dataset Q.
[0029] (5) Water level labeling: 80% of the data set Q is selected as the labeling data set D0. By setting the intersection point of the virtual water gauge and the water level, the water level labeling is completed to obtain the water level labeling data set D0 and the labeling set A0.
[0030] (6) Model training: First, crop the image training data set D0 (for the sake of example, the image pixels are 1920*1080) according to the first pixel value (width, height) (Rx1, Ry1) in the virtual water ruler array R[(V1, R1(Rx1, Ry1)), (V2, R2(Rx2, Ry2)), …, (Vn, Rn(Rxn, Ryn))] obtained earlier. At the same time, according to the highest water level orientation K, we also know whether the virtual water ruler is horizontal or vertical. If the virtual water ruler is horizontal ('-'), then the point at the upper left corner of the cropped position is (0, Ry1-25), and the point at the lower right corner is (1920, Ry1+25). If the virtual water gauge is vertical ('|'), then the point at the upper left corner of the clipping position is (Rx1-25, 0), and the point at the lower right corner is (Rx1+25, 1080). The clipping data set D1 is obtained by clipping through this rectangle. Then, the annotation set A0 of each image is transformed into a rectangle to obtain the annotation set A1. The rectangle transformation rule is based on the annotation point (x, y). The point at the upper left corner is (x-25, y-25), and the point at the lower right corner is (x+25, y+25). Finally, CNN (convolutional neural network) is used to train the model of the annotated data set D1 and the annotation set A1 to obtain the object detection model P1 of the water level area.
[0031] (7) Model verification: The remaining 20% of the data set Q is selected as the validation data set. The correctness of the model P1 is verified through the validation data set. The model's accuracy, regression rate, and F1-score are obtained.
[0032] Serial number Threshold F1-score 1 0 0.93 2 0.1 0.97 3 0.2 0.98 4 0.3 0.98 5 0.4 0.98 6 0.5 0.98 7 0.6 0.98 8 0.7 0.98 9 0.8 0.98 10 0.9 1
[0033] (8) Water level calculation: Calculate the water level through the model: Use the picture C1 of the water level recognition scene to calculate the water level. First, through the cropping step of S4, get the picture C2, and use the object detection model P1 of the water level area of C2 to calculate the water level area B[(Bx1, By1), (Bx2, By2)] (upper left point B1, lower right point B2). Then combine the virtual water scale array R[(V1, R1(Rx1, Ry1)), (V2, R2(Rx2, Ry2)), ..., (Vn, Rn(Rxn, Ryn))] (V1 is the maximum water level value, R1 is the corresponding pixel point) and the highest water level direction K of the previous virtual water scale: if it is on the top, then it is the vertical water scale ('|'), calculate the water level position By = By1 + (By2-By1) / 2. Substitute By into the virtual water gauge R, and calculate from the high to the low. If By is between Ry1 and Ry2, then the water level value = V2 + (V1-V2) * ((Ry2-By) / (Ry2-Ry1)); if it is on the left or right, then it is a horizontal water gauge ('-'), and the water level position Bx = Bx1 + (Bx2-Bx1) / 2 is calculated. Substitute Bx into the virtual water gauge R, and calculate from the high to the low. If Bx is between Rx1 and Rx2, then the water level value = V2 + (V1-V2) * ((Rx2-Bx) / (Rx2-Rx1)); the final water level value is obtained.
[0034] (9) Water level output: The calculated water level is recorded for result testing, the recognition effect is displayed through a curve graph, and the recognition reading and manual reading are compared for result comparison.
[0035] Results comparison
[0036] Serial number Manual reading / m Identification reading / m Error value / m Picture Time Confidence(%) 1 1.40 1.396 -0.004 2021-07-12 07:10:00 98.65 2 1.40 1.405 +0.005 2021-07-12 07:20:00 98.13 3 1.40 1.401 +0.001 2021-07-12 07:40:00 99.03 4 1.40 1.402 +0.002 2021-07-12 07:50:00 99.24 5 1.40 1.397 -0.003 2021-07-12 08:00:11 99.03 6 1.39 1.391 +0.001 2021-07-12 12:20:00 99.56 7 1.39 1.391 +0.001 2021-07-12 12:30:00 99.57 8 1.4 1.4 0 2021-07-12 12:40:00 99.09 9 1.39 1.39 0 2021-07-12 12:50:00 99.51 10 1.39 1.392 +0.002 2021-07-12 13:10:13 99.52
Claims
1. A method for realizing water level recognition by a virtual water gauge through image recognition technology, characterized in that: The method comprises the following steps: S1 selecting a scene, S2 setting a virtual water gauge, S3 marking a water level, S4 training a model, and S5 calculating a water level through a model; In S2, a feature object is obtained after selecting a scene, and the feature object refers to an object intersecting with the water surface. According to the position of the feature object and the scene, it is determined whether it is a horizontal or vertical virtual water ruler, and the central axis of the virtual water ruler is determined, and the highest water level orientation K is set. If it is a vertical virtual water ruler, the highest water level orientation K is on the top, and if it is a horizontal virtual water ruler, the highest water level orientation K is on the left or right; after obtaining the highest water level orientation K, a baseline is set, and the baseline is the pixel distance between the highest water level orientation K and the highest water level. , and set the highest water level value; and set the stride according to the required water level accuracy, set the water level of each scale downward through the highest water level, and after setting several scales, directly calculate the following water level value through RNN until the end of the virtual water ruler, and take the end of the virtual water ruler as the lowest water level value; obtain the array R of the virtual water ruler scale value and the corresponding pixel value, the array R is: (V1, R1 (Rx1Ry1)), (V2, R2 (Rx2, Ry2)), …, (Vn, Rn (Rxn, Ryn)); Among them, V1 is the highest water level value, R1 is the pixel point corresponding to V1, Vn is the lowest water level value, and Rn is the pixel value corresponding to Vn; In S3, the water level is marked by a point marking method, and the marked area is enlarged during marking; the water level marking is completed by setting the intersection point of the virtual water gauge and the water level, and the water level marking picture training data set D0 and the annotation set A0 of each picture are obtained; In S4, the image is transformed and cropped according to the location of the virtual water ruler before training and recognition, and the cropped image is provided for training and recognition; first, the image training data set D0 is cropped, where the image pixel is 1920*1080, and the virtual water ruler is obtained according to the first pixel value (Rx1, Ry1) in the array R obtained above and the highest water level orientation K. If the virtual water ruler is horizontal, the point in the upper left corner of the cropped position is (0, Ry1-25), and the point in the lower right corner is (1920, Ry1+25); if the virtual water ruler is Vertically, the point at the upper left corner of the cropping position is (Rx1-25, 0), and the point at the lower right corner is (Rx1+25, 1080). The cropped data set D1 is obtained by rectangle cropping; then the annotation set A0 of each image is transformed into the annotation set A1 by rectangle. The rectangle transformation rule is based on the annotation point (x, y). The point at the upper left corner is (x-25, y-25), and the point at the lower right corner is (x+25, y+25); finally, the convolutional neural network CNN is used to train the model of the labeled data set D1 and the annotation set A1 to obtain the object detection model P1 of the water level area; In S5, the water level is calculated using the picture C1 of the water level recognition scene. First, the picture C2 is obtained through the cropping step of S4, and the water level area B is calculated using the object detection model P1 of the water level area, where the upper left point B1 (Bx1, By1), the lower right point B2 (Bx2, By2); then combine the array R and the highest water level direction K, if the highest water level direction K is on the top, it is a longitudinal water gauge, and the water level position By is calculated as By1+(By2-By1) / 2; bring By into the array R, and calculate according to the water level from high to low. If By is between Ry1 and Ry2, then the water level value is V2+(V1-V2)*((Ry2-By) / (Ry2-Ry1)); if the highest water level direction K is on the left or right, it is a horizontal water gauge, and the water level position Bx is calculated as Bx1+(Bx2-Bx1) / 2; bring Bx into the array R, and calculate according to the water level from high to low. If Bx is between Rx1 and Rx2, then the water level value is V2+(V1-V2)*((Rx2-Bx) / (Rx2-Rx1)); finally get the water level value.
Citation Information
Patent Citations
A water level identification method and system based on a virtual water gauge
CN109815865A
Water level detecting method based on video recognition
CN110207783A
Water gauge identification method based on image semantic segmentation
CN110956172A
Water level identification method based on YOLOv3
CN112215125A
Apparatus and method for measuring water level
KR101284842B1
