A water level recognition method and system integrating camera calibration

Abstract

The present invention discloses a water level recognition method and system that integrates camera calibration, which involves data processing technology and solves the problems of low water level recognition accuracy and weak anti-interference ability of existing methods in complex environments. The present invention includes realizing the conversion between the pixel coordinate system and the world coordinate system by establishing a high-precision mapping from 2D pixel plane coordinates to 3D spatial coordinates, and then identifying the water boundary in the video image at the pixel level, marking the pixel coordinates and elevation of the known benchmark elevation pixel points, and then determining the pixel coordinates of the water surface line based on the straight line where the water boundary and the X-axis of the world coordinate system are located. Finally, the final water level elevation is obtained by calculating the 3D spatial coordinates corresponding to the pixel coordinates of the water surface line and the 3D spatial coordinates and elevation corresponding to the pixel coordinates of the benchmark elevation pixel points. The present invention can obtain water level data without installing a water gauge at the water level measurement site; effectively improves the anti-interference ability of water level recognition; and realizes non-contact and accurate acquisition of water level data.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a water level recognition method and system integrating camera calibration. Background Art

[0002] Image processing-based water level recognition methods are affected by a variety of factors, including complex on-site lighting, surface glare, and dirty water gauges. While they perform well under good imaging conditions, they can increase errors or even fail when the scene changes or interference factors increase, resulting in outlier-like results. Furthermore, due to geographical constraints such as on-site river / lake / reservoir conditions, it may not be possible to install a water gauge to assist in water level recognition, making water level gauge-based recognition methods unsuitable. Summary of the Invention

[0003] In order to solve the above problems existing in the prior art, the present invention provides a water level recognition method and system integrating camera calibration to solve the problem that the existing method cannot guarantee the effect of water level recognition in complex environments.

[0004] A water level recognition method that integrates camera calibration includes: establishing a high-precision mapping from 2D pixel plane coordinates to 3D spatial coordinates to achieve pixel-to-world coordinate conversion; then identifying water boundaries in video images at the pixel level; marking the pixel coordinates and elevations of pixels with known reference elevations; then determining the pixel coordinates of the water surface line based on the line between the water boundary and the X-axis of the world coordinate system; and finally, calculating the final water level elevation based on the 3D spatial coordinates corresponding to the pixel coordinates of the water surface line and the 3D spatial coordinates and elevations corresponding to the pixel coordinates of the reference elevation pixels. Specifically, the target to be recognized includes images or videos of river / lake / reservoir banks.

[0005] Preferably, the establishment of a high-precision mapping from 2D pixel plane coordinates to 3D space coordinates includes: assuming that there is a point P in the three-dimensional space with coordinates in the world coordinate system as (X W ,Y W ,Z W ), the coordinates in the camera coordinate system are (X C ,Y C ,Z C ), the coordinates in the image coordinate system are (x p ,y p ), the coordinates in the pixel coordinate system are (u, v); based on the camera's internal parameters, distortion coefficients, and external parameters, a high-precision mapping from 2D pixel plane coordinates to 3D space coordinates is established. The mapping process is as follows:

[0006] Calculate the camera projection matrix ProMtx:

[0007]

[0008] Convert pixel coordinate system to camera coordinate system:

[0009]

[0010] Convert the camera coordinate system to the world coordinate system:

[0011]

[0012] Where K is the intrinsic parameter matrix, R and T are the rotation matrix and translation vector of the world coordinate system to the camera coordinate system, f is the focal length, 1 / d x , 1 / d y are the number of pixels contained in 1 mm in the x-axis direction and the y-axis direction respectively, and (u0, v0) are the coordinates of the principal point of the image.

[0013] Preferably, the determination of the pixel coordinates of the water surface line includes: solving the equation of the pixel line on the X-axis based on the pixel coordinates of the origin of the world coordinate system and the pixel coordinates of any point on the X-axis of the world coordinate system, and then solving the pixel coordinates of the water surface line based on the pixel line.

[0014] Preferably, the pixel coordinates of the water surface line are obtained by calculating the coordinates of the intersection of the pixel line and the edge of the water area boundary, and selecting the coordinates of the intersection closest to the origin of the world coordinate system as the pixel coordinates of the water surface line.

[0015] Preferably, obtaining the final water level elevation includes: calculating the 3D spatial coordinates corresponding to the pixel coordinates of the water surface line and the pixel coordinates of the reference elevation pixel point through high-precision mapping of 2D pixel plane coordinates to 3D spatial coordinates, using the obtained 3D spatial coordinates to calculate the straight-line distance between the pixel coordinates of the water surface line and the pixel coordinates of the reference elevation pixel point, and then calculating the vertical height of the water surface line and the reference elevation point according to the slope gradient, and finally subtracting the vertical height from the reference elevation to obtain the water level elevation.

[0016] Preferably, the water area boundary includes pixel coordinates of water area boundary points obtained by identifying the water area recognition model.

[0017] Preferably, the reference elevation pixel point is located on the X-axis of the world coordinate system, and the reference elevation pixel point is located at the top edge of the slope.

[0018] Preferably, the process of obtaining the internal parameters and distortion coefficients is as follows: relevant personnel hold a calibration plate and change different postures on the slope, the camera captures an image containing the calibration plate, and then uses Zhang's calibration method to calibrate the internal parameters and distortion coefficients of the camera.

[0019] Preferably, the external parameter acquisition process is: placing a calibration plate at an appropriate location on the slope, establishing a world coordinate system on the calibration plate and calibrating the external parameters. The appropriate location includes bridge piers, vertical flat planes and slopes.

[0020] A water level recognition system integrating camera calibration includes a parameter calibration module, a key point calculation module and a water level elevation calculation module, wherein:

[0021] The parameter calibration module is used to calibrate the internal parameters, distortion coefficients, and external parameters of the camera, thereby establishing a world coordinate system on site and establishing a high-precision mapping from 2D pixel plane coordinates to 3D space coordinates;

[0022] The key point calculation module is used to calculate the pixel coordinates of the water surface line according to the world coordinate system and mark the pixel coordinates and height of the reference elevation pixel point;

[0023] The water level elevation calculation module is used to calculate the pixel coordinates of the water surface line and the 3D spatial coordinates corresponding to the pixel coordinates of the benchmark elevation pixel point based on the high-precision mapping of the 2D pixel plane coordinates to the 3D spatial coordinates, and calculate the water level elevation based on the obtained 3D spatial coordinates, the elevation of the benchmark elevation pixel point and the slope gradient.

[0024] The beneficial effects of the present invention include: 1. Water level data can be obtained without installing a water gauge at the water level measurement site; 2. The anti-interference capability of water level identification is effectively improved; 3. Non-contact, convenient, and accurate acquisition of water level data is achieved, with a water level identification error of less than 1 cm. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Flowchart of a water level recognition method integrating camera calibration.

[0026] Figure 2 Schematic diagram of the high-precision checkerboard calibration plate.

[0027] Figure 3 Schematic diagram of the placement of the calibration plate in a slope scene with a slope.

[0028] Figure 4 Schematic diagram of the placement of the calibration plate in a vertical slope scenario. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0030] The following is combined with Figure 1 The specific embodiments of the present invention are described in detail;

[0031] In the present invention, the slope refers to the slope of the river / lake / reservoir bank in the target to be identified.

[0032] A water level recognition method integrating camera calibration includes the following steps:

[0033] S1. Calibrate the camera's internal parameters and distortion coefficients using Zhang's calibration method on the shore of the target to be identified;

[0034] The high-precision checkerboard calibration plate made of aluminum alloy is made of Figure 2 The camera's internal parameters and distortion coefficients are calibrated on the bank of a river / lake / reservoir using Zhang's calibration method. The internal parameters include focal length and image principal point coordinates, and the distortion coefficients include radial distortion coefficient and tangential distortion coefficient.

[0035] S2. Calibrate the external parameters of the camera at locations such as bridge piers, vertical and flat slopes, and inclined slopes at the water level measurement site. The external parameters include a rotation matrix and a translation vector.

[0036] S21, such as Figure 3 As shown, a calibration plate is placed at an appropriate location on the water level measurement site, such as a bridge pier, a vertical flat surface, or a slope, and a world coordinate system is established on the calibration plate.

[0037] S22, solve the external parameters of the camera based on the PnP problem. It should be noted that for the convenience of calculation, the X coordinate system of the world coordinate system is placed when the calibration plate is placed. W The axis must be perpendicular to the waterline;

[0038] S3, establishing a high-precision mapping from 2D pixel plane coordinates to 3D space coordinates based on the camera's internal parameters, external parameters, and distortion coefficients;

[0039] Assume that the coordinates of point P in the three-dimensional space in the world coordinate system are (X W ,YW ,Z W ), the coordinates in the camera coordinate system are (X C ,Y C ,Z C ), the coordinates in the image coordinate system are (x p ,y p ), the pixel coordinates are (u, v). Based on the camera's internal parameters and distortion coefficients, a high-precision mapping from 2D pixel plane coordinates to 3D space coordinates (the inverse imaging process) is established. The mathematical process is as follows:

[0040] Calculate the camera projection matrix ProMtx:

[0041]

[0042] Convert pixel coordinate system to camera coordinate system:

[0043]

[0044] Convert the camera coordinate system to the world coordinate system:

[0045]

[0046] Where K is the intrinsic parameter matrix, R and T are the rotation matrix and translation vector of the world coordinate system to the camera coordinate system, f is the focal length, 1 / d x , 1 / d y are the number of pixels contained in 1 mm in the x-axis direction and the y-axis direction respectively, and (u0, v0) are the coordinates of the principal point of the image.

[0047] S4, using the trained semantic segmentation model, i.e., the water area recognition model, to identify the water area in the video, e.g. Figure 3 、 4 As shown, Figure 3 The confidence level of semantic segmentation in water area is 0.98. Figure 4 The confidence level of the semantic segmentation of the water area is 0.96. The pixel coordinates of the water level are determined, and the pixel coordinates corresponding to the benchmark elevation are pre-marked. The water area recognition model is a trained yolov5 model.

[0048] S41, identify the water area in the video through the trained water segmentation model, and obtain the pixel coordinates of the water boundary points (u1, v1), (u2, v2), (u3, v3), ..., (u n ,v n );

[0049] like Figure 3As shown in , the background of a common water level recognition scenario is a slope with different slopes. The slope α range is [10°, 90°]. Specific slope measurement methods include automatic measurement such as total station and slope measuring instrument, as well as manual measurement. Figure 4 As shown, when α is 90°, the background is a vertical bridge pier or a vertical plane.

[0050] S42. Mark X in advance in the target to be identified W The maximum range near the straight line where the axis is located is the base height corresponding to the element point B (u B ,v B ) pixel coordinates and elevation H meters, such as Figure 3 shown.

[0051] S42, in order to determine the position of the water surface line pixel coordinates, according to the origin pixel coordinates of the world coordinate system and X W The pixel coordinates of any point on the axis, solve for X W The pixel line where the axis is located l X The equation expression (rectangular coordinate system with pixel coordinate system as the origin):

[0052] l x =ku+b

[0053] Where k is the slope and b is the intercept.

[0054] S44, calculate pixel line l X The coordinates of the two intersection points on the edge of the polygon corresponding to the water area are selected, and the intersection point I (u I ,v I ) as the pixel coordinates of the water surface line;

[0055] S5. Calculate the final water level elevation based on the obtained data.

[0056] S51, calculate the point I (u I ,v I ), point B(u B ,v B ) corresponding 3D space coordinates:

[0057] S52, water surface line pixel I (u I ,v I ) along X W Direction distance pixel B(u B ,v B ) is the actual length of Meters, water level elevation WL is:

[0058]

[0059] In another embodiment, a water level recognition system integrating camera calibration is provided, comprising a parameter calibration module, a key point calculation module, and a water level elevation calculation module, wherein:

[0060] The parameter calibration module is used to calibrate the internal parameters, distortion coefficients, and external parameters of the camera, thereby establishing a world coordinate system on site and establishing a high-precision mapping from 2D pixel plane coordinates to 3D space coordinates;

[0061] The key point calculation module is used to calculate the pixel coordinates of the water surface line according to the world coordinate system and mark the pixel coordinates and height of the reference elevation pixel point;

[0062] The water level elevation calculation module is used to calculate the pixel coordinates of the water surface line and the 3D spatial coordinates corresponding to the pixel coordinates of the benchmark elevation pixel point based on the high-precision mapping of the 2D pixel plane coordinates to the 3D spatial coordinates, and calculate the water level elevation based on the obtained 3D spatial coordinates, the elevation of the benchmark elevation pixel point and the slope gradient.

[0063] The water level recognition system of the present embodiment, which integrates camera calibration, can be deployed on the Internet. After the user accesses the system through the interface and inputs the height of the target to be identified and the reference elevation, the height of the water level corresponding to the target to be identified can be accurately output. The system is simple to operate, has high recognition accuracy, and strong anti-interference ability, and is very practical.

[0064] The above-described embodiments merely represent specific implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of protection of the present application. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the technical concept of the present application, and all such variations and improvements fall within the scope of protection of the present application.

Claims

1. A water level recognition method integrating camera calibration, characterized in that: include: The conversion between the pixel coordinate system and the world coordinate system is achieved by establishing a high-precision mapping from 2D pixel plane coordinates to 3D spatial coordinates. The water boundary in the video image is then identified at the pixel level, and the pixel coordinates and elevation of the known benchmark elevation pixel points are marked. The pixel coordinates of the water surface line are then determined based on the straight line between the water boundary and the X-axis of the world coordinate system. Finally, the final water level elevation is calculated based on the 3D spatial coordinates corresponding to the pixel coordinates of the water surface line and the 3D spatial coordinates and elevation corresponding to the pixel coordinates of the benchmark elevation pixel points. Based on the camera's internal parameters, distortion coefficients, and external parameters, a high-precision mapping from 2D pixel plane coordinates to 3D space coordinates is established; The internal parameters and distortion coefficients are obtained by: a relevant person holds a calibration plate and changes different postures on the slope, and the camera captures an image containing the calibration plate, and then uses Zhang's calibration method to calibrate the internal parameters and distortion coefficients of the camera; The external parameter acquisition process is to place a calibration plate at an appropriate location on the slope, establish a world coordinate system on the calibration plate, and calibrate the external parameters. The appropriate location includes a bridge pier, a vertical flat surface, and a slope. Determining the pixel coordinates of the water surface line includes: solving an equation of a pixel line on the X axis based on the pixel coordinates of the origin of the world coordinate system and the pixel coordinates of any point on the X axis of the world coordinate system, and then solving the pixel coordinates of the water surface line based on the pixel line; The pixel coordinates of the water surface line are obtained by calculating the coordinates of the intersection of the pixel line and the water boundary, and selecting the coordinates of the intersection closest to the origin of the world coordinate system as the pixel coordinates of the water surface line; The reference elevation pixel point is located on the X-axis of the world coordinate system.

2. The water level recognition method based on camera calibration according to claim 1, characterized in that: The method of establishing a high-precision mapping from 2D pixel plane coordinates to 3D space coordinates includes: assuming that there is a point P in the three-dimensional space with coordinates (X W ,Y W ,Z W ), the coordinates in the camera coordinate system are (X C ,Y C ,Z C ), the coordinates in the image coordinate system are (x p ,y p ), the coordinates in the pixel coordinate system are (u, v); based on the camera's internal parameters, distortion coefficients, and external parameters, a high-precision mapping from 2D pixel plane coordinates to 3D space coordinates is established. The mapping process is as follows: Calculate the camera projection matrix ProMtx: Convert pixel coordinate system to camera coordinate system: Convert the camera coordinate system to the world coordinate system: Where K is the intrinsic parameter matrix, R and T are the rotation matrix and translation vector of the world coordinate system to the camera coordinate system, f is the focal length, 1 / d x , 1 / d y are the number of pixels contained in 1 mm in the x-axis direction and the y-axis direction respectively, and (u0, v0) are the coordinates of the principal point of the image.

3. The water level recognition method based on camera calibration according to claim 1, characterized in that: The method of obtaining the final water level elevation includes: calculating the 3D spatial coordinates corresponding to the pixel coordinates of the water surface line and the pixel coordinates of the benchmark elevation pixel point through high-precision mapping of 2D pixel plane coordinates to 3D spatial coordinates, using the obtained 3D spatial coordinates to calculate the straight-line distance between the pixel coordinates of the water surface line and the pixel coordinates of the benchmark elevation pixel point, then calculating the vertical height of the water surface line and the benchmark elevation point according to the slope gradient, and finally subtracting the vertical height from the benchmark elevation to obtain the water level elevation.

4. The water level recognition method based on camera calibration according to claim 1, characterized in that: The water area boundary includes pixel coordinates of water area boundary points obtained by identifying the water area recognition model.

5. The water level recognition method of fusion camera calibration according to claim 1, characterized in that: The reference elevation pixel point is located at the top edge of the slope.

6. A water level recognition system integrating camera calibration, characterized in that: A water level recognition method using a fusion camera calibration method according to any one of claims 1 to 5 is applied, comprising a parameter calibration module, a key point calculation module, and a water level elevation calculation module, wherein: The parameter calibration module is used to calibrate the internal parameters, distortion coefficients, and external parameters of the camera, thereby establishing a world coordinate system on site and establishing a high-precision mapping from 2D pixel plane coordinates to 3D space coordinates; The key point calculation module is used to calculate the pixel coordinates of the water surface line according to the world coordinate system and mark the pixel coordinates and height of the reference elevation pixel point; The water level elevation calculation module is used to calculate the pixel coordinates of the water surface line and the 3D spatial coordinates corresponding to the pixel coordinates of the benchmark elevation pixel point based on the high-precision mapping of the 2D pixel plane coordinates to the 3D spatial coordinates, and calculate the water level elevation based on the obtained 3D spatial coordinates, the elevation of the benchmark elevation pixel point and the slope gradient.

Images