Water level measurement method and device for the South-to-North Water Diversion Project Middle Route based on AI image recognition

Abstract

The present invention discloses a water level measurement method and device for the middle route project of the South-to-North Water Diversion Project based on AI image recognition. According to the red scale feature of the water level gauge, a suitable color space mode is selected to locate the specific position of the water level gauge in the captured image and the part of the water level gauge above the water level line is cropped; the scale of the water level gauge on the left side is cropped from the cropped water level gauge image in sequence, including numbers and the character "E" on the left side; the size of each character and number is identified by a template recognition method; the size of the smallest number and its location are determined, and whether the water level line position is in the middle of the character; the multi-scale method is used to estimate the size of the pixel value occupied by each millimeter, and the accurate position of the water level is calculated according to the size of the last number and the vertical pixel value of the water level gauge image. The present invention automatically selects the corresponding color space mode for intelligent fusion processing according to the characteristics of the image combined with the color change of the image, and automatically adjusts the size of the image area for recognition by a multi-scale method.

Description

Technical Field

[0001] The present invention belongs to the field of water level monitoring, and specifically relates to a water level measurement method and device for the South-to-North Water Diversion Middle Route Project based on AI image recognition. Background Art

[0002] The South-to-North Water Diversion Project is a strategic water diversion project that mainly solves the shortage of water resources in northern my country and realizes the allocation and rational allocation of water resources between the north and the south. The collection of hydrological data involved in the construction and operation of the project is a very important issue. Among them, water level data is a very critical data in the collection of hydrological information, and it is the most direct factor reflecting the water situation. The water level of rivers and lakes will be affected by natural factors such as rainfall, riverbed erosion and siltation, and strong wind direction. In addition, the change of reservoir water level will also be affected by the flow and water storage of upstream rivers and lakes. The measurement of water level is of great significance to the planning and implementation of surrounding water conservancy projects and the construction of waterway ports and other projects. At the same time, the measurement of water level is also an important data and basis for investigation and defense and formulation of measures such as drought and flood prevention, sand content detection, and ice conditions. In addition, the measurement of water level is also the basis for effectively understanding the water situation and making reasonable arrangements for the transportation of water resources. Therefore, it is necessary to find a calculation method suitable for the water level measurement of the South-to-North Water Diversion Project.

[0003] At present, there are some methods for measuring water level, among which manual measurement is the most primitive method. However, this method is time-consuming and labor-intensive. The safety factor of measuring in the river under bad weather conditions is low, which will threaten the personal safety of the staff. There are also large errors, and it is impossible to monitor the water level data in real time, so it is gradually replaced. With the development of technology, various water level meters have emerged. Among them, contact water level meters include pressure water level meters, bubble water level meters, electronic water gauges, etc., and non-contact ones include laser water level meters, radar water level meters, ultrasonic water level meters, etc. Among them, the contact water level meter takes the pressure type and bubble water level meter as examples. The pressure water level meter has a low accuracy, and the bubble water level meter is not suitable for water bodies with silt; the non-contact water level meter takes the ultrasonic water level meter as an example, and the ultrasonic water level meter is easily affected by strong winds and temperature. Due to different principles and manufacturing processes, these water level meters have their own specific application environments and limitations due to their own characteristics. In practice, if only one method is used to measure the data as the result, the result may not be convincing enough and it is difficult to avoid the errors. Therefore, in practice, a combination of multiple measurement methods is generally used to improve the accuracy of water level measurement.

[0004] With the development of science and technology, the equipment used for water level measurement has been improved and improved, but in practical applications, the instruments are constantly improved, lacking innovation in technical methods. Therefore, it is sometimes difficult to achieve good results by using a single method in water level measurement. Therefore, in the application of water level measurement in the South-to-North Water Diversion Project, scientific and technological personnel should select a reasonable combination of water level measurement methods for the South-to-North Water Diversion Project based on the environmental characteristics and measurement requirements of the site. The continuous development of computer technology has gradually made some work intelligent and mechanized. Among them, the rapid development of image recognition has made it increasingly mature and applied in many fields, such as face recognition, license plate recognition, and QR code recognition. Therefore, applying image recognition to water level recognition is a better way to solve water level recognition.

[0005] At present, ultrasonic level meters are generally installed on the Henan section of the middle route of the South-to-North Water Diversion Project to measure water levels. This method has the advantages of non-contact, stable and reliable, but ultrasonic waves are easily affected by temperature and interference from obstacles, and there is no more effective comparison and analysis. Although there are waves in the middle route of the South-to-North Water Diversion Project, the ripples are shallow, which provides great advantages for image processing to identify water levels. However, due to its good water quality, clear water flow, easy reflection, strong light, and complex background, general image recognition water level measurement is prone to large errors; affected by the flood season, the middle route of the South-to-North Water Diversion Project has higher requirements for water level monitoring, so the accuracy of water level measurement must be guaranteed. Summary of the invention

[0006] In order to solve the above problems, the present invention combines the actual situation of the South-to-North Water Diversion Project, analyzes the impact of the surrounding environment on water level image recognition, and combines image processing technology with AI technology in view of the water quality environment and water level characteristics of the middle line of the South-to-North Water Diversion Project to provide a new non-contact water level measurement method. The present invention is based on the AI ​​image recognition method, which can automatically select the corresponding color space mode for intelligent fusion processing according to the characteristics of the image combined with the image color change, and automatically adjust the image area size for recognition in a multi-scale manner. It has the advantages of non-contact, low cost, easy operation, and real-time performance, which liberates manpower and improves measurement accuracy.

[0007] The technical solution adopted by the present invention is: a water level measurement method for the South-to-North Water Diversion Middle Route Project based on AI image recognition, comprising the following steps:

[0008] Step 1: According to the red scale feature of the water level gauge, a suitable color space mode is selected to locate the specific position of the water level gauge in the captured image and the portion of the water level gauge above the water level line is cropped.

[0009] Step 2, from the cropped water level gauge image, crop out the scale of the water level gauge on the left, including the numbers and the character "E" on the left.

[0010] Step 3: Use template recognition method to identify the size of each character and number.

[0011] Step 4, determine the size and position of the minimum number, and determine whether the waterline is in the middle of the character.

[0012] Step 5: Use a multi-scale method to estimate the pixel value per millimeter, and calculate the exact water level position based on the size of the last digit and the vertical pixel value of the water level gauge image.

[0013] The present invention also provides a water level measuring device for the South-to-North Water Diversion Project Middle Route based on AI image recognition, comprising a solar panel, a monitoring host, a water level gauge, a simulated water pool and a camera, wherein a solar panel is installed on the top of the water level gauge, the solar panel is fixed on the water level gauge through a fixing device, a monitoring host is installed above the water level gauge, the solar panel is connected to the monitoring host through a wiring port, the bottom of the water level gauge is placed in a simulated water pool, the camera is aimed at the immersion of the water level gauge in the simulated water pool; the camera is connected to a video image receiving device through a wireless device, the video image receiving device uploads the video data to a cloud server for processing, and displays the processing results through a data display device.

[0014] Further, the monitoring host includes a single-chip microcomputer U1; the single-chip microcomputer U1 is respectively connected to a 4G communication module G1 and a FIFO memory module F1 connected to an image acquisition module A1, and the PC14 and PC15 serial ports of the single-chip microcomputer U1 are connected to an external low-speed clock module, which uses a crystal oscillator X2 with a frequency of 32.768KHZ, and the crystal oscillator X2 is connected in parallel with capacitors C7 and C8 connected in series; the VDD and VSS interfaces of the single-chip microcomputer U1 are respectively connected to VCC and a ground line; the single-chip microcomputer U1 is powered by a VBAT interface, which is respectively connected to a 1N4148 diode D1, a diode D2 and a capacitor C6, and a 3.3V battery B1 is used, and its two ends are connected between capacitor C6 and diode D1; the single-chip microcomputer U1 is connected to a set module through a RESET interface, and the module includes a resistor R3, a switch KEY1 and a capacitor C5, wherein the switch KEY1 and the capacitor C5 are connected in parallel, and the two ends of the resistor R3 are respectively connected to VCC and the capacitor C 5; the Vref-, OSC-IN and OSC-OUT interfaces of the single-chip computer U1 are connected to the external high-speed clock module, and the high-speed clock module adopts an 8MHZ crystal oscillator X1, the crystal oscillator X1 is connected in parallel with the resistor R2, and the two ends of the resistor R2 are respectively connected to the capacitor C3 and the capacitor C4; the Vref+, VDDA and VSSA interfaces of the single-chip computer U1 are connected to the analog voltage module, and the Vref+ and VDDA are connected to the VSSA interface in parallel with the capacitor C1 and the capacitor C2, and the two ends of the resistor R1 are respectively connected to VCC and the Vref+ and VDDA interfaces of the single-chip computer U1; on the image acquisition module A1, the camera VCC interface is connected to the positive pole of the VCC power supply; the camera PWDN interface is connected to GND, and is respectively connected to the camera VREFN and VREFH interfaces through capacitors C9 and C10; the camera RESET is connected to VCC through resistor R4, and is connected to the camera GND interface through capacitor C11.

[0015] The beneficial effects produced by the present invention are:

[0016] 1. The present invention performs intelligent fusion processing on two color space modes according to the color change of the image.

[0017] 2. It can find the optimal threshold for red pixel segmentation of the South-to-North Water Diversion water level image based on the RGB color space, as well as the optimal threshold grayscale T for background and foreground segmentation.

[0018] 3. The present invention uses a multi-scale approach to automatically identify image regions. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of the water level measurement method of the water level gauge based on image recognition of the present invention;

[0020] Figure 2This is a flow chart of a method for extracting red components according to an RGB model of the present invention;

[0021] Figure 3 Flow chart of the method for extracting red component according to the HSV model of the present invention

[0022] Figure 4 This is the H (hue) channel correspondence diagram after converting RGB to HSV in MATLAB;

[0023] Figure 5 A vertical cutaway view of the left side of the water level gauge of the present invention;

[0024] Figure 6 It is a red scale area extraction diagram of the water level gauge of the present invention;

[0025] Figure 7 It is a binary image of the red scale area of ​​the water level gauge of the present invention;

[0026] Figure 8 This is a cut-away view of the water level gauge above the water level line of the present invention;

[0027] Fig. 9 It is a character recognition template diagram of the present invention;

[0028] Fig.10 It is a multi-scale recognition flow chart of the present invention;

[0029] Fig.11 A schematic diagram of multi-scale changes of an image according to the present invention;

[0030] Fig.12 It is the recognition result diagram of the present invention;

[0031] Fig.13 This is a real-life installation diagram of the water level multi-scale measurement device of the present invention;

[0032] Fig.14 It is a device diagram of the water level multi-scale measurement device in the present invention;

[0033] Fig.15 It is a module schematic diagram of the water level multi-scale measurement device in the present invention;

[0034] In the figure: 1-solar panel, 2-fixture, 3-wiring port, 4-monitoring host, 5-water level gauge, 6-simulated water pool, 7-camera. DETAILED DESCRIPTION

[0035] The present invention will be further described below in conjunction with the accompanying drawings.

[0036] like Figure 1 As shown, the present invention is a water level measurement method for the South-to-North Water Diversion Middle Route Project based on AI image recognition, comprising the following steps:

[0037] Step 1: According to the red scale feature of the water level gauge, select the appropriate color space mode to locate the specific position of the water level gauge in the captured image and crop the part of the water level gauge above the water level line.

[0038] Step 2: From the cropped water level gauge image, crop out the scale of the water level gauge on the left, including the numbers and the character "E" on the left.

[0039] Step 3: Use template recognition method to identify the size of each character and number.

[0040] Step 4: Determine the size and position of the smallest number, and determine whether the waterline is in the middle of the character.

[0041] Step 5: Use a multi-scale method to estimate the pixel value per millimeter, and calculate the exact water level position based on the size of the last digit and the vertical pixel value of the water level gauge image.

[0042] Step 1 of the present invention, cutting the water level gauge, comprises the following specific steps:

[0043] Step 1-1: First, take the RGB mode of the water level gauge image, determine whether its Euclidean distance with the red image is less than 2.28 pixels or greater than 2.80 pixels, and use its corresponding color space to locate the red scale area of ​​the water level gauge.

[0044] The weighted red component Euclidean distance △C between the water gauge image and the standard red image in RGB space is calculated by the formula:

[0045]

[0046] △R=R1-R2 (2)

[0047] △G=G1-G2 (3)

[0048] △B=B1-B2 (4)

[0049]

[0050]

[0051] Among them, in formula (1), formula (2), formula (3), and formula (4), R1, G1, B1 are the component values ​​of the standard red image, and R2, G2, B2 are the component values ​​of the water level gauge image; in formula (5), △C is the weighted Euclidean distance between the two; formula (6) is a method for judging whether to use HSV color space or RGB color space according to the red component △C, or a method of weighted fusion of the two.

[0052] Step 1-1-1: If the weighted Euclidean distance △C of the red component is greater than 2.80 pixels, use its RGB color space image to locate the water level gauge. The flow chart for determining the position of the water level gauge using the RGB color space image is as follows: Figure 2 shown.

[0053] Step 1-1-2: If the weighted Euclidean distance △C of the red component is less than 2.28 pixels, use its HSV color space image to locate the water level gauge. The flow chart for determining the position of the water level gauge using the HSV color space image is as follows: Figure 3 shown.

[0054] Step 1-1-3: If the weighted Euclidean distance △C of the red component is greater than 2.28 pixels and less than 2.80 pixels, its RGB color space image or HSV color space image can be used to locate the water level gauge.

[0055] The step 1-1-1 of determining the position of the water gauge using the RGB color space image includes the following specific steps:

[0056] Step 1-1-1-1: If the RGB color space image is used to locate the water level gauge, first separate the red area component from other components according to the captured image and select a suitable threshold, and set the components that do not meet the threshold to white, such as Figure 8 shown.

[0057] Define its component graph as: P = (RG>level) && (RB>level), then

[0058] C=YP (7)

[0059] Set the pixel of component image C to 255. Among them, Y is the original water level gauge image, P is the red area component, C is the other color water level gauge image component, R is the red channel component of the water level gauge image, G is the green channel component of the water level gauge image, B is the blue channel component of the water level gauge image, level is the threshold, and the threshold level is finally selected as 30 pixels according to the environment and water quality characteristics of the South-to-North Water Diversion Pool.

[0060] Step 1-1-1-2: Binarize the image obtained in step 1-1-1-1. The maximum between-class variance method (OTSU) is used here to divide the image into foreground and background according to the grayscale characteristics of the image. After the threshold value obtained by this method is used for image binarization segmentation, the between-class variance of the foreground and background images is the largest and is not affected by the image brightness and contrast.

[0061] Let T be the segmentation threshold of foreground and background, the proportion of foreground points in the image is w0, and the average grayscale is u0; the proportion of background points in the image is w1, and the average grayscale is u1. The total average grayscale of the image is u, and the variance of the foreground and background images is:

[0062] w0+w1=1 (8)

[0063] u=w0×u0+w1×u1 (9)

[0064] g=w0×(u0-u) 2 +w1×(u1-u) 2 (10)

[0065] Combining the above two equations:

[0066] g=w0w1×(u0-u1) 2 (11)

[0067] When the variance g is the largest, it can be considered that the difference between the foreground and the background is the largest. At this time, the grayscale T is the best threshold of 0.7255 pixels. This threshold is used to binarize the water level gauge image, and morphological opening and closing operations are performed to remove small impurities in the image and smooth the boundaries to obtain an image containing only black and white pixels, where the black part is the water level gauge, as shown in Figure 7 shown.

[0068] Step 1-1-1-3: Determine and crop the boundary of the water level gauge based on the image obtained in step 1-1-1-2, loop from the top, bottom, left, and right positions to the middle, find the position where the pixel is 0, which is its boundary, record the boundary position and crop it.

[0069] The step 1-1-2 uses the HSV color space image to determine the water gauge position, and includes the following specific steps:

[0070] Step 1-1-2-1: If the HSV image is used to locate the water level gauge, first convert the color space of the captured image from RGB format to HSV format, and then limit the red pixels according to its color component table. The process is as follows: Figure 3 shown.

[0071] Convert RGB to HSV:

[0072] V=max(R,G,B) (12)

[0073]

[0074]

[0075] Where max is the largest value among R, G, and B, and min is the smallest value among R, G, and B;

[0076] According to the HSV component model, the range of red is: [0,43,46]~[10,255,255]∪[156,43,46]~[180,255,255]. Red is special and covers multiple ranges. Here, part of red is classified as purple. After RGB is converted to HSV in MATLAB, the value of the H channel is double data in the range of 0 to 1. However, H in the HSV color space is 0 to 360 degrees. Their corresponding relationship is as follows Figure 4 As shown. The value of H ranges from 0 to 1, corresponding to the position of the color on the color circle. As the hue increases from 0 to 1, the color transitions from red to yellow, green, cyan, blue, magenta, and finally back to red. Saturation S is a proportional value ranging from 0 to 1, which is expressed as the ratio between the purity of the selected color and the maximum purity of the color. When S = 0, there is only grayscale. V represents the brightness of the color, ranging from 0 to 1.

[0077] Since the South-to-North Water Diversion Project pool is located outdoors, there is generally sufficient fill light, so the red part of the water level gauge has less color distortion and higher saturation. Based on the experiment, we selected the following parameters as the limits of the water level gauge:

[0078] a=[0.833 0.4 1 0.3 1] (15)

[0079] P=H>(a(1)&&(S>=a(2)&&S<=a(3))&&(V>=a(4)&&V<=a(5)) (16)

[0080] Where P is the red area component, H is the image hue, S is the image saturation, and V is the image brightness.

[0081] Step 1-1-2-2: Count the number of red pixel values ​​of the image obtained in step 1-1-2-1, and count and record the row and column positions with the largest number of red pixels in the row and column directions respectively.

[0082] Step 1-1-2-3: Scan from the top, bottom, left, and right positions toward the center according to the position obtained in step 1-1-2-2. In the top and bottom directions, if it does not exceed the boundary and the number of red pixels is greater than 0, scan toward the center; in the left and right directions, scan from the outermost edge of the image toward the center to find the position where the number of red pixels is not 0 and record it.

[0083] Step 1-1-2-4: Cut the water gauge according to the water gauge boundary position obtained in step 1-1-2-3, such as Figure 5 shown.

[0084] Step 1-2: Cut according to the ruler position determined in step 1-1.

[0085] Step 2 of the present invention, water level gauge scale cutting, comprises the following specific steps:

[0086] Step 2-1: Determine and record the position of each pixel on the left half of the scale of the water level gauge based on the water level gauge image cropped in step 1.

[0087] Step 2-2: Cut out the characters, numbers and character "E" on the left scale from top to bottom according to the position obtained in step 2-1, such as Figure 6 shown.

[0088] Step 3 of the present invention, scale identification, includes the following specific steps:

[0089] Step 3-1: Create a standard character template library, normalize the character size and convert it into grayscale, such as Fig. 9 shown.

[0090] Step 3-2: Create a code table for automatic character recognition, using the character class array char. As follows: liccode = char (['0123456789''A''B''C''D''G''E''H']), where "A" represents the number "10"; "B" represents the number "11"; "C" represents the number "12"; "D" represents the number "13"; "H" represents the number "14"; "E" represents the symbol "E" on the left side of the scale.

[0091] Step 3-3: Adjust the format of the cropped character image to be consistent with the template image.

[0092] Step 3-4: Use the corr2 function to calculate the correlation coefficient between each segmented character and each character in the character template library, and save them in an array.

[0093] Step 3-5: Use the find function to find the location of the maximum correlation coefficient, so as to find the corresponding characters in the automatic recognition character code table in order and output them to achieve character recognition.

[0094] Step 3-6: Completion of obscured characters. Since the South-to-North Water Diversion Project pool is located outdoors, there may be natural factors such as dirt and other stains and artificial factors such as the covering of the water level gauge fixing clip, which will affect the recognition of characters. The boundary coordinate value of the character is determined by quadratic curve fitting through pixel coordinates, and the obscured number and character "E" can be fitted according to the size of the front and back digits in the recognized character array and the character arrangement.

[0095] The minimum scale number identification in step 4 of the present invention includes the following specific steps:

[0096] Step 4-1: Determine whether the last character is "E".

[0097] Step 4-1-1: Determine the value of the last character based on the template recognition result. Since the scale on the left side of the water level gauge is a cross arrangement of numbers and characters, the following judgment can be made based on the scale on the left side of the water level gauge: if the last character is "E", the second to last character is the smallest number above the water level line; if the last character is not "E", the last character is the smallest number above the water level line.

[0098] Step 4-2: Output the value c of the last numeric character and return the pixel position y where the last character ends.

[0099] The specific scale identification of step 5 of the present invention includes the following specific steps:

[0100] Step 5-1: The length of each grid of the character "E" is 1 cm. Calculate the pixel value corresponding to the water level scale scale, that is, the pixel value per centimeter.

[0101] m=h / s (17)

[0102] Among them, m is the pixel value of each centimeter of the water level gauge scale image, h is the total pixel value of the vertical length of the water level gauge, and s is the actual length of the water level gauge.

[0103] Step 5-2: Considering that the waterline may be in the middle of a character, estimate the length of the scale value above the waterline below the recognizable character based on the last character pixel value position returned in step 4-2.

[0104] n=(hy) / m / 10 (18)

[0105] Where h is the total pixel length of the water gauge in the vertical direction, and y is the pixel position of the last character. (The length of each character on the water gauge scale is 5cm, and each number plus the character "E" is 1dm, so the minimum number c returned is in dm, so divide by 10).

[0106] Step 5-3: Calculate the exact position of the water level based on the size of the last digit and the content and end position of the last character.

[0107] If the last character is "E", the water level gauge scale is calculated as follows:

[0108] result = c - 0.5 - n (19)

[0109] If the last character is not "E", the water level gauge scale is calculated as follows:

[0110] result = cn (20)

[0111] Among them, result is the final result of the water level position of the water level gauge, c is the size of the last number, that is, the smallest number above the water level line. Since the scale of the water level gauge starts from the character "0" and is arranged alternately with the character "E", if the last character is "E" instead of a number, 0.5dm should be subtracted.

[0112] Step 5-4: Use the multi-scale method to improve the accuracy of water level gauge water level scale recognition. The flow chart is as follows: Fig.10 As shown in the figure. The water quality of the South-to-North Water Diversion Project pool is relatively clear. Considering that the reflection of the water level gauge will cause errors in the clipping of the water level gauge above the water, a multi-scale algorithm is used to accurately identify the water level position scale according to the characteristics of the darker tone of the reflection part and the blurred characters. In the multi-scale algorithm, the optimal positioning frame size value is adaptively calculated for different data sets. During network training, there will be a positioning frame with an initial set length and width. The network outputs a predicted frame based on the initial positioning frame, and then compares it with the real frame, calculates the difference between the two, and then reversely updates and iterates the network parameters. The multi-scale algorithm generates a positioning frame based on the first input image. Thereafter, the positioning frame size will be updated according to the scale change, and batch inference produces real-time results, which can directly perform effective inference on single images, batch images, videos, and even webcam port inputs.

[0113] Step 5-3-1: Preprocess the scale image captured by the camera, and use the data enhancement strategy to expand the data set. Segment a character "E" and connect it to a number (1dm) and perform lossless spatial pixel expansion. The digital segmentation part is defined in order, and the character "E" segmentation part only has a sub-category definition. The segmented image is used as the model input, and its corresponding features are extracted. Select images different from the training to test the model, compare the test results with the true values, and adjust the model parameters according to the error to achieve the best effect.

[0114] Step 5-3-2: The water level line is located at a scale position where the upper and lower lengths of 1cm are intercepted and sent to the model for recognition. The result is accurate to the centimeter level. In order to further improve the recognition accuracy, the water level line segment image is further divided into ten equal parts after two times of lossless spatial pixel expansion, and then input into the network model for recognition. The image segment where the water level line is located is automatically identified, and the accuracy of the water level line is calculated according to the clipping segment value and the pixel expansion ratio. If the accuracy does not reach the millimeter level, the local image segment where the water level line is located is selected to further expand the spatial pixels. In this way, the water level line can be gradually concretized by iterating multiple feature extractions and superposition of pixel expansion. Finally, the specific position of the water level line can be calculated according to its pixel expansion multiples and its clipping sub-segmentation times. Because excessive pixel expansion can cause image distortion, excessive proportions and other problems, the recognition is considered to be completed when the calculated water level line accuracy reaches the millimeter level or the pixel expansion iteration exceeds 4 times, that is, the pixel expansion reaches 16 times. The following is the calculation formula for the water level line position after two pixel expansions.

[0115] x1=p / 10 (21)

[0116] x2=(x1×k) / 10 (22)

[0117] x n =(x n-1 ×k) / 10 0 <n<4 n∈Z (23)

[0118] d=x1×(a1-1)+x2×(a2-1)+(x2×k) / 10×(a3-1) (24)

[0119] s=h / p×d (25)

[0120] Where p is the vertical pixel size of the initial image, x1 is the vertical pixel size of each image segment after the first ten-equal division, x2 is the vertical pixel size of each image segment after the second ten-equal division after pixel expansion of the image segment identified by x1, n To x n-1 The vertical pixel size of each image segment after the identified image segment is divided into ten equal parts for the nth time after pixel expansion. a1 is the image segment number of the first identification, k is the multiple of pixel expansion, a2 is the image segment number of the second identification, a3 is the image segment number of the third identification, d is the vertical pixel position of the initial image where the water level line is located, h is the vertical length of the initial image, and s is the number of vertical scales of the initial image where the water level line is located.

[0121] By changing the scale space to extract features in different receptive fields, we can continuously find the most suitable scale for custom datasets. Since the target recognition framework often needs to scale the original image size, and the size of the target object in the dataset may also be different from that of the original dataset, multi-scale automatic re-learning of the size of the positioning box can effectively cope with changes in image scale. In this way, when we match feature points, the corresponding points are described at the most suitable scale, and the information described by the descriptor corresponding to the corresponding points is closer. By continuously refining the size of the output value after the scale change, and accumulating the millimeter-level scale values ​​according to their order, the schematic diagram of the multi-scale change of the image is shown below. Fig.11 As shown, the program results are shown as Fig.12 shown.

[0122] like Fig.13 As shown, a water level measuring device of the South-to-North Water Diversion Middle Route Project based on AI image recognition in the present invention comprises a solar panel 1, a fixing device 2, a wiring port 3, a monitoring host 4, a water level gauge 5 with a scale, a simulated water pool 6 and a camera 7, wherein the sensor embedded in the water level gauge can use the sensor to measure the water level to obtain a double precision of the water level gauge reading, the solar panel 1 is used to receive light to maintain the power supply of the sensor in the water level gauge, and the monitoring host 4 can collect the water level line measurement data of the water level gauge and transmit the measured water level line data to the cloud server in real time through the communication bus.

[0123] like Fig.14As shown, the video image acquisition device in the present invention mainly uses an ordinary gun-type color camera or a spherical camera to shoot the water level line of the water level gauge, and uses a wireless device to transmit the captured video image to the video image receiving device in real time. The video image receiving device transmits the video data to the cloud server. The background of the cloud server is equipped with a storage device, a processing device and a display device to identify the water level line of the received video image and display the identification result in real time. The storage device can be a computer-readable storage medium, a U disk, a mobile hard disk and other storage media, which can store computer programs and water level gauge readings. The processing device can be a digital signal processor, a central processing unit, a general processor and other devices. The data display device can include a computer, a mobile phone, a tablet, a television, an LCD display and other devices that can perform digital reception and display. The present invention also provides a computer program, and the processor runs the computer program to implement the above-mentioned water level gauge water level line recognition process, including frame extraction of video images; positioning and cropping of the extracted image to be recognized; cropping and recognition of the characters on the left side of the water level gauge; judging the water level line position according to the actual length of the water level gauge image to be recognized and the pixel size of its corresponding image, and calculating the final reading of the water level gauge according to the size of the last digit recognized and the water level line position reading. The water level gauge recognition device can include computers, mobile phones, tablets, cloud servers and other devices that can carry programs for processing and display, and integrated circuit board hardware devices installed on the water level gauge to directly read and display the results in real time.

[0124] like Fig.15As shown, the present invention uses an OV7725 camera as an image acquisition module. The single-chip microcomputer U1 in the monitoring host uses an STM32F1032ET6 chip to provide control and data transmission for the present invention; the single-chip microcomputer U1 is respectively connected to the 4G communication module G1 and the FIFO memory module F1 connected to the image acquisition module A1, wherein the 4G communication module G1 sends the collected image to the computer in time for real-time processing. The PC14 and PC15 serial ports of the single-chip microcomputer U1 are connected to an external low-speed clock module, which uses a crystal oscillator X2 with a frequency of 32.768KHZ. The crystal oscillator X2 is connected in parallel with 22pF capacitors C7 and C8 connected in series. The VDD and VSS interfaces of the single-chip microcomputer U1 are connected to the VCC and ground wires respectively. The single-chip microcomputer U1 is powered by the VBAT interface, which is connected to 1N4148 diodes D1, D2 and 10KpF capacitor C6 respectively. A 3.3V battery B1 is used, and its two ends are connected between capacitor C6 and diode D1. The single-chip microcomputer U1 is connected to the set module through the RESET interface. The module includes a 10K ohm resistor R3, a switch KEY1 and a 10KpF capacitor C5. Among them, the switch KEY1 and the capacitor C5 are connected in parallel, and the two ends of the resistor R3 are connected to VCC and the capacitor C5 respectively; the Vref-, OSC-IN and OSC-OUT interfaces of the microcontroller U1 are connected to the external high-speed clock module; the high-speed clock module adopts an 8MHZ crystal oscillator X1, the crystal oscillator X1 is connected in parallel with the 1M ohm resistor R2, and the two ends of the resistor R2 are connected to 22pF capacitors C3 and C4 respectively; the Vref+, VDDA and VSSA interfaces of the microcontroller U1 are connected to the analog voltage module, and Vref+ and VDDA are connected to the VSSA interface with a 10uF capacitor C1 and a 10KpF capacitor C2 in parallel, and the two ends of the 10 ohm resistor R1 are connected to VCC and the Vref+, VDDA interfaces respectively. On the acquisition module, the camera VCC interface is connected to the positive pole of the VCC power supply; the camera PWDN interface is connected to GND and connected to the camera VREFN and VREFH interfaces respectively through two 0.1uF capacitors C9 and C10; the camera RESET is connected to VCC2 through a 10K ohm resistor and connected to the camera GND interface through a 0.1uF capacitor C11. The above circuit completes the data acquisition and transmission functions.

Claims

1. A water level measurement method for the South-to-North Water Diversion Middle Route Project based on AI image recognition, characterized in that: The steps include: Step 1: According to the red scale feature of the water level gauge, a suitable color space mode is selected to locate the specific position of the water level gauge in the captured image and the portion of the water level gauge above the water level line is cropped; wherein cropping the water level gauge includes the following specific steps: Step 1-1, first take the RGB mode of the water level gauge image, determine whether its Euclidean distance with the red image is less than 2.28 pixels or greater than 2.80 pixels, and use its corresponding color space to locate the red scale area of ​​the water level gauge; The weighted red component Euclidean distance △C between the water gauge image and the standard red image in RGB space is calculated by the formula: （1） （2） （3） （4） （5） （6） Among them, in formula (1), formula (2), formula (3), and formula (4), , , are the component values ​​of the standard red image, are the values ​​of each component of the water level gauge image; in formula (5), △C is the weighted Euclidean distance between the two; formula (6) is a method for judging whether to use HSV color space or RGB color space according to the red component △C, or a method for weighted fusion of the two; Step 1-1-1, if the weighted Euclidean distance △C of the red component is greater than 2.80 pixels, then the RGB color space image is used to locate the water level gauge, and the flow chart of determining the position of the water level gauge using the RGB color space image; Step 1-1-2, if the weighted Euclidean distance △C of the red component is less than 2.28 pixels, then the HSV color space image is used to locate the water level gauge, and the flow chart of determining the position of the water level gauge using the HSV color space image; Step 1-1-3, if the weighted Euclidean distance △C of the red component is greater than 2.28 pixels and less than 2.80 pixels, its RGB color space image or HSV color space image can be used to locate the water level gauge; Step 2, from the cropped water gauge image, the scale of the water gauge on the left is cropped in sequence, including the numbers and the character "E" on the left; Step 3, using template recognition method to identify the size of each character and number; Step 4, determine the size and position of the minimum number, and determine whether the waterline is in the middle of the character; Step 5: Use a multi-scale method to estimate the pixel value per millimeter, and calculate the exact water level position based on the size of the last digit and the vertical pixel value of the water level gauge image.

2. According to claim 1, a water level measurement method for the South-to-North Water Diversion Middle Route Project based on AI image recognition is characterized in that: Determining the position of the water gauge using the RGB color space image in step 1-1-1 includes the following steps: Step 1-1-1-1, firstly, separate the red area component from other components according to the captured image and select a suitable threshold, and set the components that do not meet the threshold to white; Define its component graph as: P=(RG>level)&&(RB>level), then C=YP (7) Set the pixel of component image C to 255; where Y is the original water level gauge image, P is the red area component, C is the other color water level gauge image component, R is the red channel component of the water level gauge image, G is the green channel component of the water level gauge image, B is the blue channel component of the water level gauge image, level is the threshold, and the threshold level is finally selected as 30 pixels according to the environment and water quality characteristics of the South-to-North Water Diversion Pool; Step 1-1-1-2, binarize the image obtained in step 1-1-1-1, and use the maximum inter-class variance method to divide the image into foreground and background according to the grayscale characteristics of the image; Let T be the segmentation threshold of foreground and background, and the proportion of foreground points in the image is , the average grayscale is ; The ratio of background points to image is , the average grayscale is , the total average grayscale of the image is , the variance of the foreground and background images is: （8） （9） （10） Combining the above two equations: （11） When the variance g is the largest, the difference between the foreground and the background is the largest. The grayscale T at this time is the best threshold of 0.7255 pixels. This threshold is used to binarize the water level gauge image, and morphological opening and closing operations are performed to remove small impurities in the image and smooth the boundaries to obtain an image containing only black and white pixels, of which the black part is the water level gauge. Step 1-1-1-3, determine and crop the boundary of the water level gauge based on the image obtained in step 1-1-1-2, loop from the top, bottom, left, and right positions to the middle, find the position where the pixel is 0, which is its boundary, record the boundary position and crop it.

3. According to claim 1, a water level measurement method for the South-to-North Water Diversion Middle Route Project based on AI image recognition is characterized in that: The step 1-1-2 uses the HSV color space image to determine the water level gauge position, and includes the following steps: Step 1-1-2-1, first convert the captured image color space from RGB format to HSV format, and then limit the red pixels according to its color component table; Convert RGB to HSV: （12） = （13） （14） Where max is the largest value among R, G, and B, and min is the smallest value among R, G, and B; According to the HSV component model, the range of red is: [0,43,46]~[10,255,255]∪[156,43,46]~[180,255,255]; saturation S is a proportional value ranging from 0 to 1, which is expressed as the ratio between the purity of the selected color and the maximum purity of the color; when S=0, there is only grayscale; V represents the brightness of the color, ranging from 0 to 1; Select the following parameters as the limits of the water gauge part: a=[0.833 0.4 1 0.3 1] （15） P=H>(a(1)&&(S>=a(2)&&S<=a(3))&&(V>=a(4)&&V<=a(5)) (16) Where P is the red area component, H is the image hue, S is the image saturation, and V is the image brightness; Step 1-1-2-2, counting the number of red pixel values ​​of the image obtained in step 1-1-2-1, and counting and recording the row and column positions with the largest number of red pixels in the row and column directions respectively; Step 1-1-2-3, according to the position obtained in step 1-1-2-2, scan from the top, bottom, left, and right positions to the middle. In the top and bottom directions, if it does not exceed the boundary and the number of red pixels is greater than 0, scan to the middle direction; in the left and right directions, scan from the edge of the image to the middle to find the position where the number of red pixels is not 0 and record it; Step 1-1-2-4, trim the water level gauge according to the water level gauge boundary position obtained in step 1-1-2-3.

4. According to the method of water level measurement of the South-to-North Water Diversion Middle Route Project based on AI image recognition according to claim 1, it is characterized in that: The water level gauge scale cutting includes the following specific steps: Step 2-1, determining and recording the position of each pixel on the left half of the scale of the water level gauge according to the water level gauge image cropped in step 1; Step 2-2, according to the position obtained in step 2-1, cut out the characters, numbers and character "E" on the left scale from top to bottom.

5. According to claim 1, a method for measuring water level of the South-to-North Water Diversion Middle Route Project based on AI image recognition is characterized in that: Scale recognition includes the following specific steps: Step 3-1, establish a standard character template library, normalize the character size and convert it into grayscale; Step 3-2, establish a code table for automatic character recognition, using the character class array char; as follows: liccode=char(['0123456789''A''B''C''D''G''E''H']), where "A" represents the number "10"; "B" represents the number "11"; "C" represents the number "12"; "D" represents the number "13"; "H" represents the number "14"; "E" represents the symbol "E" on the left side of the scale; Step 3-3, adjusting the format of the cropped character image to be consistent with the template image; Step 3-4, use the corr2 function to calculate the correlation coefficient between each segmented character and each character in the character template library, and save them in an array; Step 3-5, use the find function to find the location of the maximum correlation coefficient, so as to find the corresponding character in the automatic recognition character code table in order, and output it to achieve character recognition; Step 3-6, completion of the blocked characters, determine the boundary coordinates of the characters by fitting a quadratic curve using pixel coordinates, and fit the blocked numbers and character "E" according to the size of the front and back digits and the character arrangement in the recognized character array.

6. The water level measurement method of the South-to-North Water Diversion Middle Route Project based on AI image recognition according to claim 1 is characterized in that: The minimum scale number identification includes the following steps: Step 4-1: Determine whether the last character is "E"; Step 4-1-1: Determine the value of the last character based on the template recognition result. If the last character is "E", the second to last character is the smallest number above the waterline. If the last character is not "E", the last character is the smallest number above the waterline. Step 4-2: Output the value c of the last numeric character and return the pixel position y where the last character ends.

7. The water level measurement method of the South-to-North Water Diversion Middle Route Project based on AI image recognition according to claim 1 is characterized in that: The specific scale recognition includes the following steps: Step 5-1: The length of each grid of the character "E" is 1 cm. Calculate the pixel value corresponding to the water level scale, that is, the pixel value per centimeter. m=h / s (17) Where m is the pixel value of each centimeter of the water level gauge scale image, h is the total pixel value of the vertical length of the water level gauge, and s is the actual length of the water level gauge; Step 5-2: In view of the possibility that the water level line is in the middle of a character, the length of the scale value above the water level line below the recognizable character is estimated based on the last character pixel value position returned in step 4-2; n=(hy) / m / 10 (18) Where h is the total length pixel value of the water level gauge in the vertical direction, and y is the pixel position of the last character; Step 5-3: Calculate the exact position of the water level according to the size of the last digit and the content and end position of the last character; If the last character is "E", the water level gauge scale is calculated as follows: result = c - 0.5 - n (19) If the last character is not "E", the water level gauge scale is calculated as follows: result=cn (20) Where result is the final result of the water level line position of the water level gauge, c is the size of the last number, that is, the smallest number above the water level line. Since the scale of the water level gauge starts from the character "0" and is arranged alternately with the character "E", if the last character is "E" instead of a number, 0.5dm should be subtracted; Step 5-4: Use a multi-scale method to improve the accuracy of water level gauge water level scale recognition, and adaptively calculate the optimal positioning frame size value for different data sets; Step 5-3-1: Preprocess the scale image captured by the camera, and use the data enhancement strategy to expand the data set. Segment a character "E" and connect it to a number and perform lossless spatial pixel expansion. The number segmentation part is defined in order, and the character "E" segmentation part is only defined as a sub-category. The segmented image is used as the model input, and its corresponding features are extracted. Select an image different from the training to test the model, compare the test result with the true value, and adjust the model parameters according to the error; Step 5-3-2: The water level line is located at a scale position where the upper and lower lengths of 1cm are intercepted and sent to the model for recognition. The result is accurate to the centimeter level. In order to further improve the recognition accuracy, the water level line segment image is further divided into ten equal parts after two times of lossless spatial pixel expansion, and then input into the network model for recognition. The image segment where the water level line is located is automatically identified, and the accuracy of the water level line is calculated according to the clipping segment value and the pixel expansion ratio. If the accuracy does not reach the millimeter level, the local image segment where the water level line is located is selected to further expand the spatial pixels. In this way, iterative feature extraction and pixel expansion superposition will gradually concretize the water level line, and finally the specific position of the water level line can be calculated according to its pixel expansion multiples and its clipping sub-segmentation times. Because excessive pixel expansion will cause image distortion, excessive proportions and other problems, when the calculated water level line accuracy reaches the millimeter level or the pixel expansion iteration exceeds 4 times, that is, the pixel expansion reaches 16 times, the recognition is considered to be over; the following is the calculation formula for the water level line position after two pixel expansions: （21） （22） （23） + （24） （25） Where p is the vertical pixel size of the initial image, is the vertical pixel size of each image segment after the first ten-equal division, for The pixel size of each image segment in the vertical direction is calculated by expanding the pixels of the identified image segment and dividing it into ten equal parts for the second time. for The pixel size of each image segment in the vertical direction after pixel expansion of the identified image segment is divided into ten equal parts for the nth time. is the image segment number of the first recognition, k is the multiple of pixel expansion, is the image segment number of the second recognition, is the image segment number of the third recognition, d is the pixel position of the water level line in the vertical direction of the initial image, h is the length of the initial image in the vertical direction, and s is the number of scales in the vertical direction of the initial image where the water level line is located.

8. A water level measurement device for the South-to-North Water Diversion Middle Route Project based on AI image recognition, characterized in that: It includes a solar panel, a monitoring host, a water level gauge, a simulated water pool and a camera. A solar panel is installed on the top of the water level gauge, and the solar panel is fixed on the water level gauge through a fixing device. A monitoring host is installed above the water level gauge, and the solar panel is connected to the monitoring host through a wiring port. The bottom of the water level gauge is placed in the simulated water pool, and the camera is aimed at the immersion of the water level gauge in the simulated water pool; the camera is connected to a video image receiving device through a wireless device, and the video image receiving device uploads the video data to a cloud server for processing, and displays the processing results through a data display device; the monitoring host includes a single-chip microcomputer U1; The single-chip computer U1 is respectively connected to the 4G communication module G1 and the FIFO memory module F1 connected to the image acquisition module A1. The PC14 and PC15 serial ports of the single-chip computer U1 are connected to the external low-speed clock module. The low-speed clock module adopts a crystal oscillator X2 with a frequency of 32.768KHZ. The crystal oscillator X2 is connected in parallel with capacitors C7 and C8 connected in series. The VDD and VSS interfaces of the single-chip computer U1 are respectively connected to VCC and the ground wire. The single-chip computer U1 is powered by the VBAT interface, which is respectively connected to the 1N4148 diode D1, the diode D2 and the capacitor C6. A 3.3V battery B1 is used, and its two ends are connected between the capacitor C6 and the diode D1. The single-chip computer U1 is connected to the set module through the RESET interface. The module includes a resistor R3, a switch KEY1 and a capacitor C5, wherein the switch KEY1 and the capacitor C5 are connected in parallel, and the two ends of the resistor R3 are respectively connected to VCC and the capacitor C5. The V The ref-, OSC-IN and OSC-OUT interfaces are connected to an external high-speed clock module, which adopts an 8MHZ crystal oscillator X1. The crystal oscillator X1 is connected in parallel with a resistor R2, and the two ends of the resistor R2 are connected to capacitors C3 and C4 respectively; the Vref+, VDDA and VSSA interfaces of the single-chip computer U1 are connected to the analog voltage module, and Vref+ and VDDA are connected to the VSSA interface in parallel with capacitors C1 and C2, and the two ends of the resistor R1 are connected to VCC and the Vref+ and VDDA interfaces of the single-chip computer U1 respectively; on the image acquisition module A1, the camera VCC interface is connected to the positive pole of the VCC power supply; the camera PWDN interface is connected to GND, and is connected to the camera VREFN and VREFH interfaces respectively through capacitors C9 and C10; the camera RESET is connected to VCC through resistor R4, and is connected to the camera GND interface through capacitor C11.

Images