Water-air collaborative multi-modal underwater terrain modeling method for shallow lake in plain

Abstract

The application discloses a water-air coordinated multi-modal underwater terrain modeling method for shallow lakes in plains, aiming to realize high-precision modeling of lakes based on multi-modal data. The method comprises the following steps: obtaining orthophoto and depth and elevation data of sounding points in a target lake area, generating a training image block set and a training structured parameter set, training and verifying a multi-modal water depth-elevation collaborative inversion model after aligning samples; generating a predicted image block set and a predicted structured parameter set corresponding to image points in the orthophoto of the target lake area, inputting the trained multi-modal water depth-elevation collaborative inversion model, and outputting the water depth and underwater elevation of the target lake area to form an underwater terrain digital elevation model. The method fuses multi-source data, improves the prediction accuracy of water depth and underwater elevation, and provides support for hydrographic surveying and mapping, environmental monitoring and the like; only a small amount of underwater equipment is needed to collect sounding point data, without the need for full coverage underwater detection of the entire lake, thereby greatly reducing equipment investment and operation time.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of lake underwater topography modeling, and particularly relates to a water-air collaborative multi-modal underwater topography modeling method for plain shallow lakes. BACKGROUND

[0002] Lake underwater topography modeling and water storage estimation are important foundations for hydrographic surveying and mapping, environmental monitoring, and ecological research. High-precision underwater topography models are the basis for accurate water storage estimation. Conducting underwater topography measurement is a prerequisite for establishing high-precision underwater topography models. Traditional underwater topography measurement mainly relies on ship-borne single-beam or multi-beam sounding equipment. Multi-beam sounding equipment can achieve full-coverage precision underwater topography measurement, but the data post-processing is tedious and the equipment and maintenance costs are relatively high. Single-beam sounding method has simple data processing and low cost, but sparse measurement lines can lead to data gaps in complex shoreline and terrain complex areas.

[0003] Lake water storage estimation methods based on ship-borne sounding data mainly include the following two types. One is to use ship-borne sounding data to generate underwater digital elevation models (DEM) directly through spatial interpolation algorithms (such as Kriging interpolation or inverse distance weighted interpolation). The underwater DEM is seamlessly spliced with the land DEM to construct a water-land integrated three-dimensional terrain model. Based on this, combined with lake water level monitoring data or lake area change information, the water storage of the lake can be dynamically calculated. This method is not only suitable for accurate estimation of lake water volume, but also can provide scientific basis for flood disaster warning and water resource management.

[0004] The second is to use ship-borne sounding data to carry out water depth remote sensing inversion. Through ship-borne single-beam sonar equipment, local lake water depth data are collected, and satellite remote sensing image inversion technology is used to obtain lake surface and surrounding environment information to generate high-resolution digital bathymetric model (DBM). The water storage of the lake is calculated by calculating the water depth of each grid. Both of the above methods have difficulties: relying only on measured data spatial interpolation to reconstruct complex terrain area underwater topography has large errors; using measured control points and optical images to measure water depth to establish a digital bathymetric model lacks elevation information and cannot be effectively combined with surrounding land terrain, limiting its application range. SUMMARY

[0005] The technical purpose of the present application aims at the current underwater terrain modeling technology based on water-air cooperative operation, which relies on a single data source, thereby possibly causing the result distortion due to error or the accuracy limitation due to signal attenuation, and provides a water-air cooperative multi-modal underwater terrain modeling method for plain shallow lake, realizes the multi-modal water depth-elevation cooperative inversion model, and realizes the joint prediction of water depth and underwater elevation.

[0006] To achieve the above technical purpose, the embodiment of the present application provides a water-air cooperative multi-modal underwater terrain modeling method for plain shallow lake, comprising:

[0007] An orthographic image of a target lake area is obtained by using an aerial device, and water depth and underwater elevation data of underwater sounding points are collected by using an underwater device;

[0008] A training image block set is generated according to the sounding points, and a prediction image block set is generated according to image points in the orthographic image;

[0009] A lake boundary line and a lake center point of the target lake area are generated according to the orthographic image;

[0010] A first relative geometric feature of the sounding points relative to the lake boundary line and the lake center point of the target lake area, and a first spatial feature of the sounding points are determined, and a training structured parameter set is generated, which comprises the first relative geometric feature and the first spatial feature;

[0011] The collected water depth and underwater elevation data are sample-aligned with the training image block set and the training structured parameter set, a training set and a verification set are determined, a multi-modal water depth-elevation cooperative inversion model is constructed, the training set and the verification set are used to train and verify the multi-modal water depth-elevation cooperative inversion model, and error and verification accuracy are calculated;

[0012] A second relative geometric feature of the image points relative to the lake boundary line and the lake center point of the target lake area, and a second spatial feature of the image points are determined, a prediction structured parameter set is generated, which comprises the second relative geometric feature and the second spatial feature, the prediction image block set and the prediction structured parameter set are input into the multi-modal water depth-elevation cooperative inversion model, the water depth and the underwater elevation of the target lake area are obtained, and a final lake underwater terrain digital elevation model is formed.

[0013] Compared with the prior art, the water-air collaborative multi-modal underwater terrain modeling method for shallow lakes in plains provided by the embodiment of the application has the following beneficial technical effects: the aerial equipment obtains high-resolution orthographic images, and the underwater equipment collects accurate water depth and underwater elevation data. The combination of the two provides comprehensive and rich data, which provides strong support for accurately constructing underwater terrain. The image block set is constructed, the relationship between texture and water terrain features is established, the relative geometry and spatial features are constructed, and the relationship between points and lake surface shape is established, which reflects the terrain features from multiple dimensions; multi-modal fusion of images and structured parameters is carried out, the disadvantages of a single data source are avoided, the elevation error in complex areas is reduced, the synchronization prediction accuracy of water depth and underwater elevation is improved, and the terrain construction accuracy is improved. The application only needs to collect sounding point data by a small amount of underwater equipment, does not need to perform full coverage underwater detection on the entire lake, greatly reduces the equipment investment and operation time, and the efficiency of the aerial equipment in obtaining orthographic images is much higher than that of underwater detection, which can quickly cover a large area of the lake region. BRIEF DESCRIPTION OF DRAWINGS

[0014] The drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present application in any way. In addition, the shapes and scale sizes of the components in the drawings are only illustrative and are used to help understand the present application and are not specific limitations on the shapes and scale sizes of the components of the present application. Those skilled in the art can select various possible shapes and scale sizes to implement the present application according to specific circumstances under the teaching of the present application. In the drawings:

[0015] Figure 1 The water-air collaborative multi-modal underwater terrain modeling method flowchart provided by the embodiment of the application;

[0016] Figure 2 The lake center point, boundary line, sounding point and checkpoint schematic diagram provided by the embodiment of the application;

[0017] Figure 3 The Figure 2 The enlarged schematic diagram of the middle gray box part;

[0018] Figure 4 The relative geometric feature schematic diagram provided by the embodiment of the application;

[0019] Figure 5 The water depth, underwater elevation and water surface elevation schematic diagram provided by the embodiment of the application in a calm water surface;

[0020] Figure 6 The network structure diagram of the multi-modal water depth-elevation collaborative inversion module provided by the embodiment of the application;

[0021] Figure 7Figures (a) and (b) are schematic diagrams of the elevation points of the sounding lines and land, (c) is a schematic diagram of the TIN corresponding to the elevation points of the sounding lines and land, and (d) is a schematic diagram of the TIN generated based on the land and the dense elevation points obtained by remote sensing inversion.

[0022] Figure 8 Figures (a) and (b) are schematic diagrams of the elevation points of the sounding lines and land, (c) is a schematic diagram of the TIN corresponding to the elevation points of the sounding lines and land, and (d) is a schematic diagram of the TIN generated based on the land and the dense elevation points obtained by remote sensing inversion. DETAILED DESCRIPTION

[0023] In order for those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor should fall within the scope of protection of the present application.

[0024] In the current research, the construction of the underwater terrain model through the collaborative inversion of water depth and elevation has not been fully focused on. First, the advantages of water-air collaborative operation (such as unmanned aerial vehicle-unmanned ship collaborative operation) provide data advantages for the collaborative inversion of water depth and elevation. One is that the unmanned ship carrying a single-beam depth sounder can achieve rapid and low-cost underwater terrain measurement, obtaining accurate water depth and underwater elevation information, while the aerial photography image of the unmanned aerial vehicle can quickly obtain spectral information of a large area of water, reflecting the water depth change in the shallow water area (without elevation information), which can assist in the construction of the underwater terrain. Two is that the aerial photography image can not only reflect the water depth change through the spectrum, but also reflect the geometric characteristics of the lake's shoreline change, area and shape, which to some extent makes up for the sparseness of single-beam data. Through the combination of unmanned aerial vehicle global coverage aerial photography and unmanned ship key measurement, the field measurement workload is reduced, the cost is reduced, and the high-precision global underwater terrain construction is realized.

[0025] Secondly, the development of multi-modal technology and deep learning methods provides technical advantages for the collaborative inversion of water depth and underwater elevation. At present, the research on water depth inversion mainly focuses on the application of BP neural network, and the potential of deep learning methods in the field of collaborative inversion of water depth and elevation is not fully explored. Although the BP network performs well in some scenarios, its structure is relatively simple and it is difficult to cope with complex nonlinear relationships and multi-modal data fusion problems.

[0026] The present application aims to solve the technical problems existing in the current underwater terrain modeling technology based on water-air cooperative operation, and provides a water-air cooperative multi-modal underwater terrain modeling method for plain shallow lakes, realizes multi-modal water depth-elevation cooperative inversion modeling based on the multi-modal water depth-elevation cooperative inversion model, obtains the final lake underwater terrain digital elevation model, and realizes joint prediction of water depth and underwater elevation.

[0027] The present application is further described below in combination with the accompanying drawings and specific embodiments.

[0028] The present application provides a water-air cooperative multi-modal underwater terrain modeling method for plain shallow lakes, which is used for quickly constructing a lake underwater terrain digital elevation model of a target lake area, and includes the following steps.

[0029] An aerial device is used to obtain orthographic images of the target lake area, and an underwater device is used to collect water depth and underwater elevation data of underwater sounding points;

[0030] A training image block set is generated according to the sounding points, and a prediction image block set is generated according to image points in the orthographic images;

[0031] A lake boundary line and a lake center point of the target lake area are generated according to the orthographic images, a first relative geometric feature of the sounding points relative to the lake boundary line and the lake center point is determined, and a first spatial feature of the sounding points is determined; a training structured parameter set is generated, and the training structured parameter set includes the first relative geometric feature and the first spatial feature;

[0032] The collected water depth and underwater elevation data are sample-aligned with the training image block set and the training structured parameter set, a training set and a verification set are determined, a multi-modal water depth-elevation cooperative inversion model is constructed, the training set and the verification set are used to train and verify the multi-modal water depth-elevation cooperative inversion model, and an error and a verification precision are calculated;

[0033] A second relative geometric feature of the image points relative to the lake boundary line and the lake center point of the target lake area is determined, and a second spatial feature of the image points is determined; a prediction structured parameter set is generated, and the prediction structured parameter set includes the second relative geometric feature and the second spatial feature; the prediction image block set and the prediction structured parameter set are input into the multi-modal water depth-elevation cooperative inversion model, water depth and underwater elevation of the target lake area are obtained, and a final lake underwater terrain digital elevation model is formed.

[0034] In some embodiments, as shown in Figure 1 The method includes the following steps:

[0035] Step 1: An aerial device is used to obtain orthographic images of the target lake area, and an underwater device is used to collect water depth and underwater elevation data of underwater sounding points;

[0036] Step 2 generates a training image block set and a prediction image block set, including:

[0037] Step 2-1: determine the image block size;

[0038] Step 2-2: batch crop image blocks;

[0039] Step 2-3: generate a training image block set of the sounding points and a prediction image block set of the image points in the orthophoto.

[0040] Step 3 generates a training structured parameter set and a prediction structured parameter set, including:

[0041] Step 3-1: generate a lake boundary line of the target lake area and a lake center point according to the orthophoto;

[0042] Step 3-2: determine the first relative geometric feature of the sounding points relative to the lake boundary line and the lake center point of the target lake area, determine the first spatial feature of the sounding points; determine the second relative geometric feature of the image points in the orthophoto relative to the lake boundary line and the lake center point, determine the second spatial feature of the image points;

[0043] Step 3-3: generate a training structured parameter set, the training structured parameter set including the first relative geometric feature and the first spatial feature; generate a prediction structured parameter set, including the second relative geometric feature and the second spatial feature;

[0044] Step 4: build a multi-modal water depth-elevation collaborative inversion model to collaboratively predict water depth-water depth;

[0045] Step 4-1: sample alignment of the collected water depth and underwater elevation data with the training structured parameter set and the training image block set, and divide into a training set and a validation set; generate a prediction set according to the prediction image block set and the prediction structured parameter set;

[0046] Step 4-2: build a multi-modal water depth-elevation collaborative inversion model;

[0047] Step 4-3: precision verification using the training set and the validation set; when predicting the underwater topographic digital elevation model of the target lake area, use the prediction set to obtain the water depth and underwater elevation of the target lake area, and form the final lake underwater topographic digital elevation model.

[0048] In the embodiment, the aerial device can be a drone, a helicopter, etc.; the underwater device is a device that can operate on the water surface or underwater, such as an unmanned ship, an underwater robot, etc.

[0049] In some embodiments, the underwater device is used to carry a single-beam depth sounder to collect water depth and underwater elevation data of underwater sounding points, specifically:

[0050] According to the shape and area size of the target lake area, the sounding lines and inspection lines are laid out in advance, wherein the sounding lines are perpendicular to the long side direction of the lake shoreline of the target lake area, the inspection lines are distributed orthogonally to the sounding lines, and the spacing between the measuring points of the inspection lines is not greater than 1 / 2 of the spacing between the measuring points of the sounding lines (as shown in Figure 2 and Figure 3 The underwater equipment sails along the sounding lines and the inspection lines, and synchronously collects the water depth and underwater elevation data of the underwater measuring points of the target lake area through the single-beam depth finder.

[0051] As an example, the water depth is measured by using the unmanned ship carrying the single-beam depth finder, the water depth data is preprocessed by positioning data correction, denoising and smoothing, transducer draft correction, water level correction, etc., to obtain the water surface elevation, the water depth point position coordinates and the water depth information. The transducer is a device used for measuring water depth on the unmanned ship, which determines the distance from the water surface to the water bottom by transmitting and receiving acoustic signals. The GNSS antenna is installed on the unmanned ship and is used to receive signals of the global navigation satellite system (such as GPS, Beidou, etc.). The position information of the unmanned ship determined by the GNSS antenna, the relative position between the geometric depth finder transducer and the GNSS antenna on the unmanned ship, the instantaneous position of the transducer and its elevation relative to the geodetic surface (water surface elevation) are calculated, and the water surface elevation of the transducer is obtained. After the water depth value measured by the depth finder transducer is obtained, the water surface elevation is subtracted from the water depth value, and the elevation of the underwater measuring point (i.e. the corresponding position of the water bottom) can be obtained, that is, the underwater elevation. Thus, a series of measuring points containing water depth and underwater elevation can be obtained, each of which contains four parameters of east coordinate, north coordinate, water depth and underwater elevation.

[0052] The table format only records the attribute data of the coordinates, water depth and underwater elevation of the measuring points, and cannot be directly spatially associated with the aerial image (spatial data). After being converted into a vector point format, the measuring points have clear spatial position attributes, can be superimposed and matched with the aerial image in ARCGIS, and are used for subsequent spatial feature extraction. A unique FID (Feature ID) code is assigned: each measuring point corresponds to a unique FID, which can be used as the identity of the measuring point, avoids confusion of the point position in subsequent data processing (such as feature extraction and sample division), and ensures data traceability.

[0053] As an example, in ARCGIS software, the sounding points stored in table format (.xls format) are converted to vector points storage (.shp format) using the spread point tool. Each sounding point contains a unique FID code after spreading. Using the "Rasterize" tool in ARCGIS, the vector format sounding points are converted to raster label images. The value of the raster where the sounding point is located is the FID of the sounding point. The resolution and coordinate system of the label image are consistent with the aerial image. If the spacing between the sounding points is less than the image resolution, multiple sounding points will fall into the same raster, i.e. the same raster contains multiple FID values. In this case, the largest or smallest FID value is selected as the unique value of the raster.

[0054] In some embodiments, high-resolution images of the lake are obtained using a drone equipped with RGB and near-infrared cameras. The drone adopts a tilt photogrammetry mode, and the flight height is set to 50-120 meters according to the resolution requirement, the heading overlap rate is 80%, and the lateral overlap rate is 70%, ensuring the image stitching quality. Using the processing software built-in the drone, pre-processing such as distortion correction and POS solution is performed to generate high-resolution orthophotos.

[0055] In some embodiments, in ARCGIS software, a lake surface vector is drawn based on high-resolution orthophotos, which is used for subsequent relative geometric feature extraction. In ARCGIS software, the "Raster Crop" tool is used to crop the image using the lake surface vector, and only the lake part of the image is retained; the "Raster to Point" tool is used to convert each raster of the high-resolution image to a vector point (hereinafter referred to as an image point), which is used for relative geometric feature and spatial feature extraction, and each image point has a unique FID code.

[0056] In some embodiments, step 3-1 includes: drawing a lake surface vector based on the orthophoto; converting the drawn lake surface vector to a line vector to generate a lake boundary line; and calculating a lake center point based on the lake surface vector.

[0057] As an example, in ARCGIS software, the "Surface to Line" tool is used to convert the lake surface vector to a line vector to generate a lake boundary line. In ARCGIS software, the "Feature to Point" tool is selected to calculate the lake center point based on the lake surface vector, and "Use the position contained in the input feature as the output point position" is selected to ensure that the lake center point is located inside the lake surface vector.

[0058] There is a complex interaction between the morphological characteristics (such as shape, bottom shape, size, and shoreline tortuosity) of the lake and the water depth. Long and narrow lakes tend to form a central deep water area with gradually shallow edges, while disc-shaped lakes have a gentle water depth variation. Due to the difference between point measurement and surface morphology, point-like sounding points / image points are difficult to directly correlate with lake morphology, such as shape index.

[0059] In the embodiments, the first relative geometric features include a shortest distance of the sounding point to a lake boundary line of the target lake region, an Euclidean distance of the sounding point to the lake boundary line of the target lake region, and / or an angle of the sounding point to a center point of the target lake region in a counterclockwise rotation; and the second relative geometric features include a shortest distance of the image point to the lake boundary line of the target lake region, an Euclidean distance of the image point to the lake boundary line of the target lake region, and / or an angle of the image point to the center point of the target lake region in a counterclockwise rotation. It can be understood that the first relative geometric features, the first spatial features selected in the training stage and the second relative geometric features, the second spatial features selected in the prediction stage are the same.

[0060] As shown in Figure 4 , the embodiments of the present application realize the establishment of the relationship between the point (sounding point or image point) and the surface (lake shape) by constructing the relative geometric features, calculating at least one type of relative geometric features of the shortest distance of the arbitrary point (sounding point or image point) to the lake boundary line of the target lake region, the distance of the arbitrary point (sounding point or image point) to the center point of the target lake region, and the angle of the arbitrary point (sounding point or image point) to the center point of the target lake region in a counterclockwise rotation, which is used for analyzing and evaluating the water depth features and underwater topographic features of different regions.

[0061] The relative geometric features themselves contain clear topographic trends, such as the rules of “the closer to the shore, the smaller the water depth usually is”, “the water depth changes gently in the lake center region”, and “the underwater topography of the lake is usually a gentle disc shape” in the shallow lake of a plain region in a province, which can be directly transmitted to the model, reducing the difficulty of the model in mining the implicit rules from the original data.

[0062] The shortest distance of the sounding point and the image point to the lake boundary is helpful to understand the boundary between the shallow water area and the deep water area of the lake. As an example, for the shortest distance of the arbitrary sounding point or image point to the lake boundary line , the calculation formula is as follows:

[0063] ;

[0064] wherein represents the distance of the i th point (sounding point or image point) to the lake boundary line, which can be one of the relative geometric features; the coordinates of the i th point (arbitrary sounding point or image point) are (x i , y i ); represents the lake shoreline polygon boundary, P j represents the j th point on the lake shoreline boundary, and the coordinates are (x j , y j ).

[0065] In some implementations, the "nearest neighbor analysis" tool in ArcGIS software is used to automatically search and calculate the shortest distance between sounding points, image points and lake boundaries, without having to manually traverse all boundary points.

[0066] In some embodiments, the relative geometric features include the Euclidean distance from any sounding point or image point to the center of the lake. As an example, the Euclidean distance from any sounding point or image point to the center of the lake can be expressed as:

[0067] .

[0068] in This represents the distance from the i-th point (depth sounding point or image point) to the center point of the lake, where the coordinates of the i-th point are (x, y). i ,y i The coordinates of the center point of the lake are (x0, y0).

[0069] In some embodiments, the relative geometric features include an angle of counterclockwise rotation from any point to the center of the lake.

[0070] As an example, with due east as the reference axis (0°), the counterclockwise rotation angle from the i-th sounding point or image point (i.e., any sounding point or image point) to the center of the lake can be expressed as:

[0071] ;

[0072] Where θ i Let x be the angle of counterclockwise rotation from the i-th point to the center of the lake, and let the coordinates of the i-th point be (x, y). i ,y i The coordinates of the center point of the lake are (x0, y0), and arctan2() is a function used to calculate the arctangent value.

[0073] The embodiments clearly define the distances and angles from the sounding points and image points to the center point of the lake, which can reflect the relative positions of the sounding points and image points within the lake.

[0074] In some implementations, the "Nearest Neighbor Analysis" tool in ArcGIS software is used to calculate the shortest distance and azimuth from sounding points and image points to the center point of the lake, which are then used as one of the relative geometric features. The "Nearest Neighbor Analysis" tool is set to use the lake center point layer as the "Target Feature" and the sounding point or image point layer as the "Source Feature" to achieve automated batch calculations, with output results accurate to the 0.1-meter level.

[0075] In some embodiments, step 3-2 specifically includes the X-axis coordinates and Y-axis coordinates of the sounding point or image point as spatial feature indicators (i.e., the first spatial feature of the sounding point and the second spatial feature of the image point).

[0076] In some embodiments, in ARCGIS software, the X-axis coordinate and Y-axis coordinate of the sounding points and the image points are calculated as spatial feature indicators using the "Computational Geometry" tool. When extracting spatial feature indicators, the X-axis and Y-axis coordinates need to use a unified projection coordinate system to avoid numerical deviation caused by the spherical coordinate system of the geographic coordinate system (latitude and longitude). Through the "Coordinate" option of the ARCGIS "Computational Geometry" tool, the planar coordinates of the point features can be directly extracted to the attribute table field, with the precision reserved to six decimal places, ensuring the sub-meter spatial positioning accuracy. The coordinate data serves as an absolute position feature, which is complementary to the relative geometric features described above. The relative geometric features and spatial feature parameters are easy to obtain without the need for additional high-precision equipment or complex algorithms.

[0077] In some embodiments, the training structured parameter set and the prediction structured parameter set further include information of at least one of four preset bands included in the orthophoto, and the four preset bands include a red band, a green band, a blue band, and a near-infrared band. In embodiments, the information of the band can include spectral information and radiation information (such as radiance, reflectivity, etc.).

[0078] It is assumed that the small range of lake surface is a plane, and under the same water surface elevation, the water depth is related to the underwater elevation, that is, the decrease of the underwater elevation generally corresponds to the increase of the water depth, such as Figure 5 The present application realizes the double output prediction of water depth and underwater elevation by sharing the bottom feature extraction layer, so the training set and the validation set contain two target variables of water depth and underwater elevation.

[0079] In some embodiments, step 2-1 specifically includes setting the image block size. According to the geographical features of the lake and the image resolution, the image block size is dynamically set, small-scale ground object areas (for example, the area of the lake is less than 10 square kilometers) are adapted to 32x32 size, and large-scale terrain areas (for example, the area of the lake is greater than or equal to 10 square kilometers) are adapted to 64x64 size. The size of the training image block and the prediction image block of the same lake needs to be consistent, and different sizes of image blocks should construct different multi-modal water depth-elevation collaborative inversion models.

[0080] In some embodiments, step 2-3 specifically includes generating a training image block set of sounding points or a prediction image block set of image points. For the training image block set of sounding points, according to the label image, the image block is extracted by expanding around a single sounding point as the geometric center, and is named and sorted by the sounding point FID. For the prediction image block set of image points, the image block is extracted by expanding around each image point as the geometric center, and is named and sorted by the image point FID.

[0081] For the sounding points and the image points close to the edge, the boundary overflow problem is handled by filling 0 to ensure that the size of each image block of the same lake is consistent.

[0082] In some embodiments, step 4-1 specifically includes processing training data, validation data, and prediction data. Due to the fact that the spacing between sounding points is not completely consistent with the resolution of aerial images or geographic registration, multiple sounding points may fall within the same grid, thereby losing some sounding points (multiple sounding points correspond to only one image block), resulting in inconsistent number of image block samples and structured parameter samples. The FID value is used for sample screening and alignment to ensure that the FID values corresponding to the image block and the structured parameter sample are consistent and the arrangement order is consistent.

[0083] After obtaining the training image block set and the training structured parameter set (first relative geometric feature and first spatial feature) of all sounding points, 70% of the sounding points are randomly selected as the training set, and the remaining 30% are randomly selected as the validation set. The prediction image block set and the prediction structured parameter set (second relative geometric feature and second spatial feature) of all image points are obtained as the prediction set. After dividing the training set and the validation set with the same random number, the FID values and the arrangement order of the image block set and the structured parameter set samples of the training set and the validation set still need to be consistent.

[0084] The training set and the validation set divided by the sounding points are used to construct and verify the multi-modal water depth-elevation collaborative inversion model. The image points are input into the trained multi-modal water depth-elevation collaborative inversion model as the prediction set, and the water depth and underwater elevation of the image points are output, and the underwater topographic digital elevation model is further constructed. The training set, the validation set, and the prediction set all include the image block set and the structured parameter set.

[0085] As an example, the data structure is as follows: the training set includes p image blocks and structured parameters, the validation set includes q image blocks and structured parameters, as shown in Figure 8 , wherein the training structured parameter set of the training set and the validation set includes the first spectral feature, the first relative geometric feature, and the first spatial feature, and the training set and the validation set also include two target variables of water depth and underwater elevation, and the sum of p and q is equal to the total number of sounding points; the prediction structured parameter set of the prediction set includes k prediction data, including the second spectral feature (the same as the wave band included in the first spectral feature), the second relative geometric feature, and the second spatial feature, and no target variable; the image blocks of the training set, the validation set, and the prediction set are image blocks with the same size and the same number of wave bands. As shown in Figure 8 , the wave bands include red wave band, green wave band, and blue wave band, and the relative geometric feature includes the shortest distance of the sounding point or the image point (coordinates (x i ,y i ) to the lake boundary line of the target lake area and the angle of the counterclockwise rotation to the target lake area lake center point .

[0086] In some embodiments, the step 4-2 multi-modal water depth-elevation collaborative inversion model is a fusion of water body images and structured parameters, which can be predicted by constructing an additive fusion neural network.

[0087] As an example, the multi-modal water depth-elevation collaborative inversion model includes a CPO module, an image branch, a structured parameter branch, a branch fusion module, and a water depth-elevation collaborative inversion module, as shown in Figure 6 .

[0088] In some embodiments, the CPO module uses the Crested Porcupine Optimizer (CPO) to find the optimal parameters of the number of convolution layer filters of the image branch, the number of hidden units of the structured parameter branch, the batch size, the maximum number of training times, etc. The parameter value range of the number of convolution layer filters of the image branch, the number of hidden units of the structured parameter branch, the batch size, the maximum number of training times, etc. can be customized, and the CPO finds the optimal combination of parameters within the customized value range. After determining the optimal parameters, the image branch, the structured parameter branch, the branch fusion module, and the water depth-elevation collaborative inversion module are retrained, reducing the adjustment parameter time and providing optimized parameter support for the operation of the image branch and the structured parameter branch.

[0089] In some embodiments, the CPO module has the number of convolution layer filters of the image branch in the range of 16-100, the number of hidden units of the structured parameter branch in the range of 16-200, the batch size in the range of 16-500, and the maximum number of training times in the range of 1-1000. The wider the parameter value range, the longer the parameter optimization time.

[0090] In some embodiments, the image branch extracts spatial features related to water depth and elevation from water body image blocks, including:

[0091] 1. Image input layer, defining the size of the input image block, and reducing the interference of light and water quality differences on feature extraction through zscore standardization method;

[0092] 2. Two-dimensional convolution layer, using the number of filters optimized by CPO, extracting local features of the image through a 3x3 sliding window, which can capture key spatial features such as shorelines, shallow water area substrate textures, lake island or shoal outlines in lake images, and using Padding method to ensure that edge features are not lost;

[0093] 3. Normalization layer, standardizing the feature map output by the convolution layer to avoid gradient vanishing / explosion caused by feature value fluctuations and improve the adaptability of the model to complex water environments;

[0094] 4. Max pooling layer, further compress the feature map size by taking the maximum value within a 3x3 window, while preserving the strongest response features in local regions such as coastlines;

[0095] 5. Fully connected layer, compress the high-dimensional spatial features after pooling into a 128-dimensional dense vector;

[0096] 6. Flattening layer, convert the multi-dimensional feature vector output by the fully connected layer into a one-dimensional vector to adapt to the subsequent fusion structure with the structured parameter branch.

[0097] In some embodiments, the structured parameter branch is to extract spatial correlation features related to water depth, elevation, and dynamic rules between sounding points (sounding points distributed along the same line have continuity) from data such as relative geometric features and absolute spatial coordinates, including:

[0098] 1. Structured parameter input layer, receiving a set of structured parameters;

[0099] 2. Patch embedding layer, patch size 1, embedding dimension 128, preserving parameter distance, angle, and other semantics;

[0100] 3. Embedding concatenation layer, converting structured parameters such as angles and distances into interactive high-dimensional features and embedding them to form comprehensive features containing multi-dimensional spatial information; converting discrete spatial parameters (distance, angle, coordinate) into interactive high-dimensional features while preserving their physical semantics;

[0101] 4. Position embedding layer, sequence length 128, embedding dimension 128, encoding spatial position information and capturing the correlation between distance and water depth, angle and terrain, and other parameters;

[0102] 5. Addition layer, fusing the embedding concatenation layer and the position embedding layer to associate sounding point parameter values with spatial distribution;

[0103] 6. Self-attention layer, number of heads 32, hidden dimension 32x3, capturing spatial dependency between structured parameters;

[0104] 7. Random dropout layer, optional dropout ratio of 10%, inhibiting model overfitting to abnormal spatial parameters;

[0105] 8. Time index layer, extracting the latest time step of spatial structured features;

[0106] 9. Flattening layer, converting the indexed high-dimensional spatial features into a one-dimensional vector to adapt to the BiLSTM layer input;

[0107] 10. First-level BiLSTM layer, using CPO optimized hidden unit number, for processing the correlation between spatial distribution of sounding points and water depth-elevation changes;

[0108] 11. Random Dropout layer, e.g. 10% dropout rate, to suppress BiLSTM's memory of temporal noise;

[0109] 12. Second BiLSTM layer, with CPO optimized number of hidden units, to deepen the extraction of complex features such as the distribution pattern between sounding points, structured parameters, and the correlation between water depth and underwater elevation;

[0110] 13. Self-attention layer, to fine-tune the key correlations between parameters;

[0111] 14. Fully connected layer, to compress the extracted complex features into a 128-dimensional dense vector, consistent with the output dimension of the image branch;

[0112] 15. Flatten layer, to output a one-dimensional 128-dimensional feature vector as the core feature of the structured parameter branch, and to fuse it with the features of the image branch.

[0113] In some embodiments, the branch fusion module fuses the 128-dimensional features of the image branch and the 128-dimensional features of the structured parameter branch through addition, achieving the synergy of "image features + structured parameter features".

[0114] In some embodiments, the water depth-elevation co-inversion module is used to map the reinforced features through a fully connected layer to the two target variables of water depth and elevation, to finally generate the prediction results of water depth and underwater elevation, including:

[0115] 1. Multi-head attention layer, with 4 heads of attention mechanism, output dimension of 2, aligned to the two target variables of water depth and underwater elevation;

[0116] 2. Water surface elevation consistency constraint layer, designed based on the loss function of the logical constraint of water surface elevation = water depth + underwater elevation, to synchronously generate the prediction results of water depth and underwater elevation.

[0117] The loss function of the consistency constraint is composed of a basic loss term and a constraint loss term. The basic loss is the sum of the mean square errors (MSE) of water depth and underwater elevation, and the constraint loss is the deviation of the predicted values of water depth (WD) and underwater elevation (UE) from the true value of water surface elevation (WSE), which is equal to the sum of the true values of water depth and underwater elevation, as shown in Figure 7 .

[0118] The basic loss and the constraint loss are balanced by the parameter λ. λ takes a value of 0-1, and the specific parameter is adjusted according to the prediction results of the model. When λ = 0, it degenerates into an MSE loss function. The loss function of the consistency constraint is expressed as follows:

[0119] ,

[0120] ,

[0121] ,

[0122] wherein, is the base loss; is the constraint loss; is the loss of consistency constraint; λ is a hyperparameter that adjusts the base loss and the constraint loss; is the water depth predicted value; is the water depth true value; is the underwater elevation predicted value; UE is the underwater elevation true value; WSE is the water surface elevation true value; total number of samples, subscript i is used to identify the sample number.

[0123] The loss function of the consistency constraint adds the logical constraint of "water depth + underwater elevation = water surface elevation" in the regression task, and improves the logical self-consistency of the collaborative prediction results of water depth and underwater elevation.

[0124] The multi-modal water depth-elevation collaborative inversion model in the application fuses multi-source data to avoid the shortcomings of a single data source. The high-resolution orthographic image obtained by the aerial equipment (such as a drone) contains rich texture and spectral information, which can reflect the water depth changes in the shallow water area; the accurate water depth and underwater elevation data collected by the underwater equipment (such as a single-beam echo sounder carried by an unmanned ship) provide accurate measurement values. The combination of the two can reflect the terrain features from multiple dimensions, reduce the elevation error in complex areas, and improve the synchronous prediction accuracy of water depth and underwater elevation.

[0125] The CPO module of the inversion model can find the optimal parameters of the batch size, the number of convolution layer filters, the maximum number of training times, and the number of hidden units of the network, and provide optimized parameter support for the operation of the image branch and the structured parameter branch. The image branch and the structured parameter branch use specific structures and algorithms to effectively extract relevant features from the water body image block and the structured parameter respectively, and improve the adaptability of the model to complex water environment. For example, the structured parameter branch uses a double-layer BiLSTM structure to mine long-term dependence features, so that the model can better process data under different conditions and adapt to diversified application scenarios.

[0126] In some embodiments, in step 4-2: the water depth and underwater elevation inversion accuracy is calculated by the deep learning model using the verification set, and the calculation index of the inversion accuracy is the coefficient of determination (R2) and the root mean square error (RMSE). The closer R2 is to 1 (such as 1>R2>0.9), the better the model prediction effect. The closer RMSE is to 0 (0<RMSE<0.5 meters), the better the prediction effect. RMSE is suitable for evaluating the overall prediction accuracy of the model fitting. After R2 and RMSE reach the expected target, the global water depth and underwater elevation of the lake are inverted by the deep learning model using the prediction set.

[0127] In some embodiments, based on 4-band aerial images, a set of 64*64 size image blocks (training image block set and prediction image block set) and a set of structured parameters (training structured parameter set and prediction structured parameter set, both including 9 features: 4 spectral features, 3 relative geometric features and 2 spatial features) are constructed, the number of convolution layer filters of the image branch optimized by CPO is 64, the number of hidden units of the structured parameter branch is 128, the batch size is 162, and the maximum number of training is 433. When λ is 0.01, the water depth R2 of the multi-modal water depth-elevation collaborative inversion model is 0.92, the underwater elevation R2 is 0.91; the water depth R2 of the model based only on the image branch is 0.83, the underwater elevation R2 is 0.85; the water depth R2 of the model based only on the structured branch is 0.82, the underwater elevation R2 is 0.81; the water depth R2 of the model based on 4-band spectral features and CNN-LSTM deep learning model is 0.64, the underwater elevation R2 is 0.64.

[0128] The multi-modal water depth-elevation collaborative inversion model is suitable for the construction of water-land integrated elevation model of shallow lake in plain area. Due to the influence of earth curvature, the multi-modal water depth-elevation collaborative inversion model is not suitable for the construction of underwater topography of large area lake.

[0129] As an example, the underwater topographic vector points of the lake can be generated in ARCGIS through the "point display" and "merge" tools, as shown in Figure 7 Through the "create TIN" tool, the digital elevation model of the lake underwater topography is constructed. TIN (Triangulated Irregular Network) is a vector data model for representing three-dimensional terrain surface.

[0130] In the embodiment, the elevation points of the sounding line and the remote sensing inversion elevation points are combined with the land DEM to construct the underwater topography. Figure 7 In the embodiment, (a) is the elevation point of the sounding line and the land, and only a small amount of linear elevation points are present in the lake, the sampling point density is low, the distribution is sparse, and the coverage of the terrain is insufficient. Figure 7In the middle (c), the TIN is constructed based on the corresponding sounding lines and the elevation points of the land, the terrain boundary is obviously distorted, the contour transition is relatively rough, part of the terrain details is lost, and the fitting accuracy is low. Figure 7 In the middle (b), the dense elevation points obtained by remote sensing inversion are shown, which basically cover the entire lake area, and provide more rich terrain feature support for TIN fitting. Based on the TIN generated by the dense elevation points obtained by remote sensing inversion and the land, the contour level is clear, the transition is smooth, the regional boundary can be accurately fitted, the terrain details are accurately expressed, and the fitting effect is excellent, as shown in the middle (d). Figure 7 Figure 7 It can be seen that the present application only needs a small amount of underwater equipment to collect sounding point data, without the need for full coverage underwater detection of the entire lake, greatly reducing equipment investment and operation time; the present application only needs a small amount of measured data plus a multi-modal model inversion mode to expand the information of the limited measured points to the entire lake area, solving the limitations of the traditional method.

[0131] As an example, in ARCGIS, the "3D Analysis Tools"-"Functional Surface"-"Surface Volume" tool can be used to calculate the water storage corresponding to each water level or water surface based on the underwater terrain digital elevation model of the lake.

[0132] The modeling method provided by the embodiment of the present application is used to form the underwater terrain digital elevation model of the final target lake area, which can be used to calculate the water storage of the lake, and provides a scientific basis for water resource management. Combined with real-time water level monitoring data or lake area change information, the water storage of the lake can be dynamically calculated, meeting the needs of hydrographic surveying and mapping, environmental monitoring and ecological research, and the application range is effectively expanded compared with the traditional method.

[0133] The above describes in detail the lake underwater terrain construction method provided by the present application, and the principles and implementation modes of the present application are described by using specific examples. The above embodiment is only used to help understand the concept of the present application, and should not be understood as limiting the protection scope of the present application; the steps in the above embodiment should not be understood as limiting the execution order, unless otherwise specified.

[0134] The above embodiments only express several implementation modes of the present application, and the description is relatively specific and detailed, but it should not be understood as limiting the patent scope of the present application. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the present application patent should be subject to the appended claims.​

Claims

1. A water-air collaborative multimodal underwater terrain modeling method for shallow lakes in plains, characterized in that, include: Use aerial equipment to acquire orthophotos of the target lake area; Underwater equipment is used to collect water depth and underwater elevation data at underwater sounding points; A training image patch set is generated based on the depth sounding points, and a prediction image patch set is generated based on the image points in the orthophoto. Generate the lake boundary line and lake center point of the target lake area based on the orthophoto; determine the first relative geometric features of the sounding point relative to the lake boundary line and lake center point, and the first spatial features of the sounding point; Generate a training structured parameter set, the training structured parameter set including the first relative geometric feature and the first spatial feature; The collected water depth and underwater elevation data are aligned with the training image patch set and the training structured parameter set to determine the training set and validation set. A multimodal water depth-elevation co-inversion model is constructed. The multimodal water depth-elevation co-inversion model is trained and validated using the training set and validation set. The error and validation accuracy are calculated. Determine the second relative geometric features of the image point relative to the lake boundary line and the center point of the lake in the target lake region, as well as the second spatial features of the image point; Generate a predicted structured parameter set, the predicted structured parameter set including the second relative geometric feature and the second spatial feature; The predicted image patch set and the predicted structured parameter set are input into the multimodal water depth-elevation co-inversion model to obtain the water depth and underwater elevation of the target lake area, forming the final underwater topographic digital elevation model of the lake. Wherein, the first relative geometric feature includes the shortest distance of the sounding point relative to the lake boundary line of the target lake area, the second relative geometric feature includes the shortest distance of the image point relative to the lake boundary line of the target lake area, the first relative geometric feature includes the Euclidean distance of the sounding point relative to the lake boundary line of the target lake area, the second relative geometric feature includes the Euclidean distance of the image point relative to the lake boundary line of the target lake area, and / or the first relative geometric feature includes the counterclockwise rotation angle of the sounding point to the center point of the lake in the target lake area, and the second relative geometric feature includes the counterclockwise rotation angle of the image point to the center point of the lake in the target lake area; The first spatial feature of a sounding point is its X-axis and Y-axis coordinates, while the second spatial feature of an image point is its X-axis and Y-axis coordinates.

2. The water-air collaborative multimodal underwater terrain modeling method according to claim 1, characterized in that, The method for generating the lake boundary line and lake center point of the target lake region based on the orthophoto image includes: Draw the lake surface vector based on the orthophoto; The drawn lake surface vector is converted into a line vector, and finally the lake boundary line is generated; The center point of the lake is calculated based on the lake surface vector.

3. The water-air collaborative multimodal underwater terrain modeling method according to claim 1, characterized in that, The formula for calculating the shortest distance from any point to the boundary line of the target lake area is as follows: ； in This represents the distance from the i-th point to the lake boundary line, and the coordinates of the i-th point are (x, y). i ,y i ), ∂Ω represents the polygonal boundary of the lake shoreline of the target lake area, P j This represents the j-th point on the shoreline boundary of the target lake area, with coordinates (x, y). j ,y j ).

4. The water-air collaborative multimodal underwater terrain modeling method according to claim 1, characterized in that, The formula for calculating the counterclockwise rotation angle from any point to the center point of the target lake area is as follows: ； Where θ i Let x be the angle of counterclockwise rotation from the i-th point to the center of the lake, and let the coordinates of the i-th point be (x, y). i ,y i The coordinates of the center point of the lake are (x0, y0), and arctan2() is a function used to calculate the arctangent value.

5. The water-air collaborative multimodal underwater terrain modeling method according to claim 1, characterized in that, The training structured parameter set and the prediction structured parameter set also include information on at least one of the four preset bands contained in the orthophoto, which include the red band, green band, blue band and near-infrared band.

6. The water-air collaborative multimodal underwater terrain modeling method according to claim 1, characterized in that, The multimodal depth-elevation co-inversion model includes a CPO module, an image branch, a structured parameter branch, a branch fusion module, and a depth-elevation co-inversion module. The CPO module is used to find the batch size, number of convolutional layer filters, maximum number of training iterations, and number of hidden units of the network, providing optimization parameter support for the operation of the image branch and the structured parameter branch; The image branch input image block set is subjected to local feature extraction, and the data processed by the convolutional layer, activation layer, pooling layer and flattening layer is passed to the branch fusion module; The structured parameter branch input structured parameter set adopts a two-layer BiLSTM structure to mine long-term dependency features in the data and input them into the branch fusion module; The branch fusion module uses an additive approach to fuse the output features of the image branch and the structured parameter branch to obtain enhanced features, and then inputs the enhanced features into the depth-elevation collaborative inversion module. The water depth-elevation collaborative inversion module is used to map the enhanced features to multiple target variables through a multi-head attention layer and a fully connected layer, and finally generate prediction results for water depth and underwater elevation.

7. The water-air collaborative multimodal underwater terrain modeling method according to claim 1, characterized in that, The aerial equipment is a drone. When using the drone to acquire the orthophoto, the drone adopts an oblique photogrammetry mode, with the flight altitude set to 50-120 meters, the forward overlap rate to be 80%, and the lateral overlap rate to be 70%.

8. The water-air collaborative multimodal underwater terrain modeling method according to claim 1, characterized in that, The process of collecting water depth and underwater elevation data at underwater sounding points using underwater equipment includes: the underwater equipment is equipped with a single-beam echo sounder for data acquisition, specifically: Based on the shape and size of the target lake area, depth sounding lines and inspection lines are pre-laid out. The depth sounding lines are perpendicular to the long side of the shoreline of the target lake area, and the inspection lines are orthogonal to the depth sounding lines. The spacing between the measurement points of the inspection lines is no greater than 1 / 2 of the spacing between the measurement points of the depth sounding lines. The underwater equipment navigates along the sounding line and inspection line, and simultaneously collects water depth and underwater elevation data of the underwater sounding points using the single-beam echo sounder.

9. The water-air collaborative multimodal underwater terrain modeling method according to claim 1, characterized in that, During the training of the multimodal water depth-elevation co-inversion model, when the accuracy of the multimodal water depth-elevation co-inversion model is verified using the validation set, the expected target is that R² is greater than or equal to the first threshold and less than 1, and RMSE is less than or equal to the second threshold and greater than 0, where R² is the coefficient of determination and RMSE is the root mean square error.

Images