A method for making a fast 3D water body model for water body monitoring

Abstract

The present invention provides a method for making a fast 3D model of water area for water body monitoring, comprising the following steps: S1, selecting a number of reference points in the target water area; S2, calibrating the external parameters of the camera using the reference points; S3, determining the position coordinates of each reference point according to the calibration results of the camera external parameters; S4, performing dense interpolation based on the position coordinates of each reference point to establish a 3D model of the water area. The present invention does not need to use a drone to perform oblique photography and point cloud collection on the target water area at various angles, and can directly use a camera and RTK equipment to collect data on the target water area, with a small amount of data and low processing cost; and because the 3D model of the water area is established based on the reference points selected from the target water area, the modeling accuracy will not be affected by environmental factors such as ground vegetation and stones, and the 3D modeling of the target water area can be quickly achieved.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional reconstruction of water bodies, and in particular to a method for quickly producing a 3D water body model for water body monitoring. Background Art

[0002] Video surveillance of water areas is of great significance for flood warning, disaster assessment, and disaster relief command, but current surveillance technology usually uses cameras to shoot videos, which can only be processed on two-dimensional images and cannot accurately predict water level changes and flooding. In order to accurately estimate the flooding situation after the water level changes, it is very necessary to make a 3D model of the monitoring area. The camera combined with the 3D model of the monitoring area can expand many practical applications, such as automatic water level monitoring and identification, water body modeling and image effect rendering, flooding estimation, stability judgment of wading structures during floods, dangerous area judgment, etc.

[0003] However, the production of 3D models is very difficult. The current mainstream method is to use drones for oblique photography and point cloud collection to obtain 3D point clouds and then model based on 3D point clouds. However, this method has several disadvantages: 1) The range and time of drone use are limited; 2) UAV scanning generates a large amount of data, which requires a lot of cost to process the corresponding data; 3) Ground grass, trees, and rocks seriously interfere with drone detection of the ground, which will cause large modeling errors. Summary of the invention

[0004] The present application provides a method for quickly producing a 3D water area model for water body monitoring, which solves the problems of traditional water area 3D modeling methods, such as limited range and time of drone use, high cost required to process corresponding data, and serious interference of drone detection of the ground by vegetation and rocks.

[0005] The present application provides a method for quickly producing a 3D water body model for water body monitoring, comprising the following steps:

[0006] S1, select several reference points in the target waters;

[0007] S2, calibrating the external parameters of the camera using the reference point;

[0008] S3, determining the position coordinates of each reference point according to the calibration results of the camera external parameters;

[0009] S4, dense interpolation is performed based on the position coordinates of each reference point to establish a 3D model of the water area.

[0010] In the technical solution provided by the present application, the camera is calibrated with external parameters by selecting reference points in the target water area, thereby determining the position coordinates of the reference points, and finally densely interpolating the reference points to establish a 3D model of the water area; there is no need to use drones to perform oblique photography and point cloud collection of the target water area at various angles, and the camera and RTK equipment can be used directly to collect data of the target water area, with a small amount of data and low processing cost; and because the 3D model of the water area is established based on the reference points selected in the target water area, the modeling accuracy will not be affected by environmental factors such as ground grass, trees, and stones, and the 3D modeling of the target water area can be quickly achieved.

[0011] Specifically, step S1 includes the following steps:

[0012] S11, select a sampling point at a certain distance around the target water area;

[0013] S12, using RTK equipment to obtain the GPS coordinates of each sampling point;

[0014] S13, selecting a number of sampling points from all sampling points as reference points;

[0015] S14, verifying the selected reference points and eliminating reference points that do not meet the requirements.

[0016] Furthermore, in step S13, the selected reference points must meet the following conditions:

[0017] a) All reference points are not located on the same straight line;

[0018] b) Prioritize sampling points located in key monitoring areas as reference points;

[0019] c) The number of reference points is not less than n, and the area covered by the reference points is not less than S.

[0020] Furthermore, in step S14, the method for verifying the selected reference points is as follows: clustering the selected reference points using a clustering algorithm to obtain an optimal clustering model; if n reference points are selected, randomly select k reference points from the n reference points to obtain a clustering model, and use the clustering model to verify whether the remaining nk reference points meet the conditions and calculate the errors, and after multiple iterations, find the optimal clustering model corresponding to the minimum error; substitute all n reference points into the optimal clustering model, and calculate whether the error exceeds a preset range. If the error is within the preset range, the n reference points meet the requirements; if the error is not within the preset range, remove the reference points with larger deviations, and then reselect the reference points for clustering.

[0021] Specifically, in step S2, the external parameters of the camera include the rotation matrix R and the translation matrix T of the camera relative to the earth coordinate system. The translation matrix T is obtained through the GPS coordinates of the camera. The rotation matrix R includes the camera internal rotation matrix R i and the rotation matrix R of the camera relative to the earth coordinate system when it is installed o , R i It is calculated by reading the pan angle and tilt angle of the camera; Ro is obtained by calibration, and the calibration method is:

[0022] The imaging formula from the geodetic coordinate system to the image coordinate system is:

[0023]

[0024] get:

[0025]

[0026] Among them, u and v are image coordinates, Indicates non-strict equality; K is the camera internal parameter; X, Y, Z are the geodetic coordinates, Xc, Yc, Zc are the camera coordinates;

[0027] The above formula is transformed into:

[0028]

[0029] make,

[0030]

[0031] Use the camera to shoot reference points in different directions, record the GPS coordinates of each reference point and the corresponding ptz parameters and image coordinates of the camera, and get:

[0032] [V1 V2 … V n ]=R o *[W1 W2 … W n ]

[0033] Through nonlinear optimization i ||V i -R o *W i || 2 Calculate the quaternion and use it to generate the rotation matrix R o As the external calibration parameters of the camera.

[0034] Specifically, in step S3, according to the calibration results of the camera extrinsic parameters, an XYZ rectangular coordinate system is established with the ground projection of the camera point as the coordinate origin, and the GPS coordinates of each reference point are converted into xyz coordinates.

[0035] Specifically, in step S4, the method for dense interpolation based on the position coordinates of each reference point is as follows: an appropriate number of interpolation points are selected according to the area of ​​the target water area, and a number of reference points are interpolated using a cubic spline function to form a dense point cloud; in the process of forming the dense point cloud, the point cloud is constrained using the bundle adjustment method to eliminate discrete noise points.

[0036] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings in this application are used to illustrate preferred embodiments, so that those skilled in the art can clearly understand various other advantages and benefits, and should not be considered as limitations of this application. In addition, the same reference numerals are used throughout the drawings to represent the same or similar components.

[0038] Figure 1 It is a flowchart of a method for quickly making a 3D water area model for water body monitoring in one embodiment of the present application. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0041] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two (including two), unless otherwise clearly and specifically defined.

[0042] In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0043] In the description of the embodiments of the present application, the technical terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the embodiments of the present application.

[0044] In the description of the embodiments of the present application, unless otherwise clearly specified and limited, technical terms such as "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0045] See also Figure 1 The present application embodiment provides a method for quickly making a 3D water area model for water body monitoring, comprising the following steps:

[0046] S1, select several reference points in the target waters;

[0047] S2, calibrating the external parameters of the camera using the reference point;

[0048] S3, determining the position coordinates of each reference point according to the calibration results of the camera external parameters;

[0049] S4, dense interpolation is performed based on the position coordinates of each reference point to establish a 3D model of the water area.

[0050] In the technical solution provided by the present application, the camera is calibrated with external parameters by selecting reference points in the target water area, thereby determining the position coordinates of the reference points, and finally densely interpolating the reference points to establish a 3D model of the water area; there is no need to use drones to perform oblique photography and point cloud collection of the target water area at various angles, and the camera and RTK equipment can be used directly to collect data of the target water area, with a small amount of data and low processing cost; and because the 3D model of the water area is established based on the reference points selected in the target water area, the modeling accuracy will not be affected by environmental factors such as ground grass, trees, and stones, and the 3D modeling of the target water area can be quickly achieved.

[0051] Specifically, step S1 includes the following steps:

[0052] S11, select a sampling point at a certain distance around the target water area;

[0053] S12, using RTK equipment to obtain the GPS coordinates of each sampling point;

[0054] S13, selecting a number of sampling points from all sampling points as reference points;

[0055] S14, verifying the selected reference points and eliminating reference points that do not meet the requirements.

[0056] Furthermore, in step S11, the sampling points include the bottom of the river channel, the top of the dam, the water gauge position, the stepped waterline position, etc.

[0057] Furthermore, in step S13, the selected reference points must meet the following conditions:

[0058] a) All reference points are not located on the same straight line;

[0059] b) Prioritize sampling points located in key monitoring areas as reference points;

[0060] c) The number of reference points shall not be less than 10 and the area covered by the reference points shall not be less than 500m 2 (The number of reference points is adaptively adjusted according to the coverage area).

[0061] Select some points with obvious features (about 10) from all the sampling points as reference points for calibration of the camera coordinate system and the geodetic coordinate system; use the camera to shoot the RTK device of the reference points and record the PTZ parameters of the camera for the external parameter calibration of the camera.

[0062] Further, in step S14, the method for verifying the selected reference points is: clustering the selected reference points using a clustering algorithm to obtain an optimal clustering model; if 10 reference points are selected, 8 reference points are randomly selected from the 10 reference points for clustering to obtain a clustering model, and the clustering model is used to verify whether the remaining 2 reference points meet the conditions and calculate the errors, and the optimal clustering model corresponding to the minimum error is found after multiple iterations; all 10 reference points are substituted into the optimal clustering model, and the error is calculated whether it exceeds a preset range (such as 1%). If the error is within the preset range, the 10 reference points meet the requirements; if the error is not within the preset range, the reference points with large deviations are eliminated, and the reference points are reselected for clustering; after multiple verifications, the reference points obtained are all within the allowable error range, and the camera is calibrated with external parameters using the screened reference points.

[0063] When the camera is installed, there are uncertainties in both position and orientation. To achieve monitoring, the parameters of the camera relative to the earth coordinate system must be known first, including six degrees of freedom: three-dimensional translation and three-dimensional rotation. Three-dimensional translation can be provided by the GPS coordinates of the camera position measured by the device. The method for determining three-dimensional rotation is to use the camera to shoot the RTK device in different directions, record the corresponding ptz parameters, GPS coordinates, and image coordinates, and optimize the calculation of the corresponding rotation matrix.

[0064] Specifically, in step S2, the external parameters of the camera include the rotation matrix R and the translation matrix T of the camera relative to the earth coordinate system. The translation matrix T is obtained through the GPS coordinates of the camera. The rotation matrix R includes the camera internal rotation matrix R i and the rotation matrix R of the camera relative to the earth coordinate system when it is installed o , R i Calculated by reading the pan angle and tilt angle of the camera; R o Obtained through calibration.

[0065] R o It is a rotation in three-dimensional space with three degrees of freedom, so at least three reference points are required for calibration. Because the RTK equipment used may have measurement errors, it is necessary to take more reference points to reduce the errors. In this embodiment, at least 10 reference points are selected for the external parameter calibration of the camera. First, the GPS coordinates of the camera are measured with the RTK equipment and converted into the geodetic coordinate system; then the RTK equipment of the reference point is photographed with the camera, the GPS coordinates of the reference point are recorded, and converted into the geodetic coordinate system, and the image coordinates and the tpz parameters of the camera are recorded at the same time; the above steps are repeated for the reference points in different directions, and the relevant parameters of at least 3 reference points are recorded; finally, the rotation matrix R is calculated using the above relevant parameters. o ; The calculation method is as follows:

[0066] The imaging formula from the geodetic coordinate system to the image coordinate system is:

[0067]

[0068] get:

[0069]

[0070] Among them, u and v are image coordinates, Indicates non-strict equality; K is the camera internal parameter; X, Y, Z are the geodetic coordinates, Xc, Yc, Zc are the camera coordinates;

[0071] The above formula is transformed into:

[0072]

[0073] The α here cannot be solved, so it is solved by normalization method;

[0074] make,

[0075]

[0076] Use the camera to shoot reference points in different directions, record the GPS coordinates of each reference point and the corresponding ptz parameters and image coordinates of the camera, and get:

[0077] [V1 V2 … V n ]=R o *[W1 W2 … W n ]

[0078] Due to R o It is a rotation matrix, which cannot be directly calculated using matrix pseudo-inverse. Quaternions can be used to generate this rotation matrix through nonlinear optimization ∑ i ||V i -R o *W i || 2 Calculate the quaternion and use it to generate the rotation matrix R o As the external calibration parameters of the camera.

[0079] Specifically, in step S3, according to the calibration results of the camera's external parameters, an XYZ rectangular coordinate system is established with the ground projection of the camera's location as the coordinate origin, the east direction as the positive direction of the X-axis, the north direction as the positive direction of the Y-axis, and the vertical upward direction as the positive direction of the Z-axis, and the GPS coordinates of each reference point are converted into xyz coordinates.

[0080] Since RTK sampling points are very sparse and cannot be used directly for 3D modeling, the sampling points need to be interpolated first to form a dense point cloud.

[0081] Specifically, in step S4, the method for performing dense interpolation based on the position coordinates of each reference point is: selecting a suitable number of interpolation points according to the area of ​​the target water area, and using a cubic spline function to interpolate a number of reference points to form a dense point cloud.

[0082] In a specific implementation, multiple cubic polynomial segments are used for interpolation, and every 0.5 meter is used as an interpolation particle point. In this embodiment, it is assumed that the coverage range of the actual scene is 200 meters × 100 meters (the interpolation interval can be adaptively adjusted according to the length and width of the actual scene). After interpolation through the cubic spline function, a dense point set of 400*200 can be formed.

[0083] In the process of forming a dense point cloud, the bundle adjustment method is used to constrain the point cloud to remove discrete noise points. By repeatedly iterating to remove discrete noise points, the dense points with errors greater than 5% are constrained to 5%, and finally a set of relatively stable dense point clouds is obtained. The purpose of bundle adjustment is to constrain the RTK points in the entire scene in a three-dimensional space. Some noise points with burrs can be filtered out to play the role of smoothing interpolation points. Finally, the above dense point set is generated into a 3D model using dedicated modeling software such as Halcon for subsequent practical applications.

[0084] Finally, the camera's field of view is virtualized through U3D software, and a three-dimensional model of the water level line is drawn. The model is projected onto the geodetic coordinate model established by the RTK points, thereby presenting the effect of a three-dimensional model of the water level or water body in the ball camera screen. It can be used to display the current water level and flow, warning water level alarms, and prediction of flooding during floods.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some or all of the technical features therein can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and they should all be included in the scope of the claims and specification of the present application. In particular, as long as there is no contradiction or conflict, the various technical features mentioned in the embodiments can be combined in any way. The present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions that fall within the scope of the claims.

Claims

1. A method for making a rapid 3D water body model for water body monitoring, characterized in that: The following steps are involved: S1, select several reference points in the target waters; S2, calibrating the external parameters of the camera using the reference point; S3, determining the position coordinates of each reference point according to the calibration results of the camera external parameters; S4, dense interpolation is performed based on the position coordinates of each reference point to establish a 3D model of the water area; Step S1 specifically includes the following steps: S11, select a sampling point at a certain distance around the target water area; S12, using RTK equipment to obtain the GPS coordinates of each sampling point; S13, selecting a number of sampling points from all sampling points as reference points; S14, verifying the selected reference points and eliminating reference points that do not meet the requirements; In step S14, the method for verifying the selected reference points is as follows: clustering the selected reference points using a clustering algorithm to obtain an optimal clustering model; if n reference points are selected, randomly select k reference points from the n reference points to cluster and obtain a clustering model, and use the clustering model to verify whether the remaining nk reference points meet the conditions and calculate the errors, and after multiple iterations, find the optimal clustering model corresponding to the minimum error; substitute all n reference points into the optimal clustering model, and calculate whether the error exceeds a preset range. If the error is within the preset range, the n reference points meet the requirements; if the error is not within the preset range, remove the reference points with large deviations, and then reselect the reference points for clustering.

2. A method for making a fast water area 3D model for water body monitoring according to claim 1, characterized in that: In step S13, the selected reference points must meet the following conditions: a) All reference points are not located on the same straight line; b) Prioritize sampling points located in key monitoring areas as reference points; c) The number of reference points is not less than n, and the area covered by the reference points is not less than S.

3. The method for making a fast 3D water area model for water body monitoring according to claim 1, characterized in that: In step S2, the external parameters of the camera include the rotation matrix R and the translation matrix T of the camera relative to the earth coordinate system. The translation matrix T is obtained through the GPS coordinates of the camera. The rotation matrix R includes the camera internal rotation matrix R i and the rotation matrix R of the camera relative to the earth coordinate system when it is installed o , R i Calculated by reading the pan angle and tilt angle of the camera; R o It is obtained by calibration, and the calibration method is: The imaging formula from the geodetic coordinate system to the image coordinate system is: get: Among them, u and v are image coordinates, Indicates non-strict equality; K is the camera internal parameter; X, Y, Z are the geodetic coordinates, Xc, Yc, Zc are the camera coordinates; The above formula is transformed into: make, Use the camera to shoot reference points in different directions, record the GPS coordinates of each reference point and the corresponding ptz parameters and image coordinates of the camera, and get: [V1 V2 ... V n ]=R o *[W1 W2 ... W n ] Through nonlinear optimization i ||V i -R o *W i || 2 Calculate the quaternion and use it to generate the rotation matrix R o As the external calibration parameters of the camera.

4. The method for making a fast 3D water area model for water body monitoring according to claim 1, characterized in that: In step S3, according to the calibration results of the camera external parameters, an XYZ rectangular coordinate system is established with the ground projection of the camera point as the coordinate origin, and the GPS coordinates of each reference point are converted into xyz coordinates.

5. The method for making a fast 3D water area model for water body monitoring according to claim 1, characterized in that: In step S4, the method for dense interpolation based on the position coordinates of each reference point is as follows: an appropriate number of interpolation points are selected according to the area of ​​the target water area, and a number of reference points are interpolated using a cubic spline function to form a dense point cloud; in the process of forming the dense point cloud, the point cloud is constrained using the bundle adjustment method to eliminate discrete noise points.

Images