A Method and System for Intelligent Reconstruction of Underwater Topography in Dike Breach Based on Multi-UAV Monitoring

Abstract

This invention belongs to the field of hydraulic surveying technology. It proposes a method and system for intelligent reconstruction of underwater topography at dike breaches based on multi-UAV monitoring. The main scheme is as follows: collect existing underwater topography data and surface velocity data, establish a database, train a deep learning model, and deploy the trained deep learning model through a dynamic link library; acquire visible light images of the dike breach location using a first UAV, and acquire underwater topography elevation data along the breach using a second UAV; process the visible light images to obtain surface velocity data at the dike breach location and river channel cross-sectional width data at different locations near the dike breach; through coordinate transformation, match the coordinates of local underwater measuring points in the underwater topography elevation data along the breach with the coordinates of the surface velocity data and the river channel cross-sectional width data, and input the results into the deployed deep learning model; and reconstruct the underwater topography at the dike breach using the deployed deep learning model.

Description

Technical Field

[0001] This invention relates to the field of water conservancy technology, and in particular to a method and system for intelligent reconstruction of underwater topography of dike breaches based on multi-UAV monitoring. Background Technology

[0002] Accurate and rapid reconstruction of underwater topography is crucial in emergency response to levee breaches; however, existing technologies have significant limitations in such complex environments. Traditional manual measurement methods are difficult to implement when levee breaches occur, and the harsh environmental conditions threaten the lives of surveyors, thus limiting their practical application.

[0003] Currently, underwater topographic surveying is commonly conducted using unmanned surface vessels (USVs) or drones equipped with sonar or laser equipment. This typically relies on amphibious drones or USVs traveling along predetermined routes to cover the reconstructed area and collect data from different cross-sections. Then, algorithms such as inverse distance weighted interpolation or kriging interpolation are used to reconstruct the underwater 3D topography. However, this method has significant drawbacks: firstly, the measurement time is long, and the accuracy of the underwater topography reconstruction depends on the number of measurement points; secondly, during levee breaches, the high water velocity and harsh environment make it difficult for USVs or drones to operate near the water surface, and they cannot collect enough underwater topographic points, resulting in low reconstruction accuracy or even failure to complete the reconstruction.

[0004] Furthermore, aerial inspection by drones faces significant challenges at night or in low visibility conditions. Even with infrared camera technology, the image resolution is low, limiting the effective information that can be acquired, which affects measurement accuracy and reconstruction results.

[0005] See patent application CN119354152A, which discloses a method and device for rapidly constructing underwater topography of dike breaches using a drone. However, this patent application mainly utilizes a drone to measure a large amount of underwater topographic data, then uses flow velocity data, infrared images, and visible light images to correct the topographic data, and finally reconstructs the underwater topography at the breach. In reality, measuring underwater topographic data when a dike breach occurs is extremely difficult. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for intelligent reconstruction of underwater topography of dike breaches based on multi-UAV monitoring. It can accurately complete the high-precision reconstruction of underwater topography of dike breaches by using less underwater topography data, combined with surface flow velocity data and river channel width, and by training a deep learning model.

[0007] The technical solution adopted by this invention to solve its technical problem is as follows:

[0008] On the one hand, the present invention provides a method for intelligent reconstruction of underwater topography of dike breaches based on multi-UAV monitoring, including the following steps:

[0009] Collect existing underwater topographic data and surface flow velocity data, and establish a database;

[0010] A deep learning model is trained based on the established database;

[0011] Deploy the trained deep learning model using dynamic link libraries;

[0012] The first drone acquires visible light images of the location of the dike breach, and the second drone acquires underwater topographic elevation data along the way while floating along the water flow line with the propulsion of the natural water flow.

[0013] The visible light image is processed to obtain surface flow velocity data at the location of the dike breach and river channel cross-sectional width data at different locations near the dike breach.

[0014] By transforming the coordinates, the coordinates of local underwater measuring points in the underwater topographic elevation data along the route are matched with the coordinates of surface velocity data and river channel width data. The matched surface velocity data, river channel width at different locations near the breach of the dike, and underwater topographic elevation data along the route are then input into the deployed deep learning model.

[0015] The underwater topography at the breach in the dike is reconstructed using a well-deployed deep learning model.

[0016] As a further optimization, before acquiring visible light images of the breach location of the dike through the first UAV and acquiring underwater topographic elevation data along the way by the second UAV floating along the water flow line with the propulsion of the natural water flow, it also includes: a first UAV equipped with an image acquisition device and a second amphibious UAV equipped with a sonar device.

[0017] The first UAV equipped with an image acquisition device refers to:

[0018] The first drone will be flown to a position where it can fully capture images of the location of the dike breach and the upstream and downstream flow areas through the image acquisition device.

[0019] The aforementioned amphibious second unmanned aerial vehicle equipped with sonar refers to:

[0020] Based on the upstream-to-downstream process, the intervals between multiple amphibious second UAVs equipped with sonar devices, the number of local underwater sampling points and the sampling frequency of a single second UAV equipped with sonar devices are obtained.

[0021] The second unmanned aircraft will fly upstream of the breach in the dike and land on the water at the intervals mentioned above.

[0022] As a further optimization, acquiring visible light images of the dike breach location via the first UAV means:

[0023] The first UAV uses an image acquisition device to capture multiple frames of visible light images at a set shooting speed, forming a multi-frame image sequence;

[0024] The acquisition of underwater topographic elevation data along the course of a second UAV while it floats along the streamline of the water flow using the propulsion of the natural water current refers to:

[0025] After the second UAV lands on the water surface at the intervals, it acquires the upstream water flow velocity and determines whether it reaches the first threshold. When it does, it shuts off the flight and navigation power system of the second UAV and acquires underwater terrain elevation data along the way while floating along the water flow line with the propulsion of the natural water flow.

[0026] When the second drone drifts downstream, it obtains the downstream water flow velocity and determines whether it is less than a second threshold. If it is less than a threshold, it activates the flight and navigation power system of the second drone and flies back to its starting position.

[0027] As a further optimization, the visible light image is processed to obtain surface flow velocity data at the location of the dike breach and river channel cross-sectional width data at different locations near the dike breach. This refers to:

[0028] The visible light image is processed using the pyramidal layered optical flow method to obtain surface flow velocity data at the location of the dike breach. Image recognition and segmentation are then performed on the visible light image to obtain cross-sectional width data of the river channel at different locations near the dike breach.

[0029] The width of the river channel at different locations near the breach of the dike refers to the width of the river channel at different locations between the breach of the dike and its upstream and downstream sections.

[0030] As a further optimization, before processing the visible light image using the pyramid-based hierarchical optical flow method, the method further includes:

[0031] Obtain the visibility of the location of the dike breach and the upstream and downstream flow locations, and determine whether it is below the set threshold. If it is below the threshold, add a third drone and use the third drone to drop luminous tracers before the landing position of the second drone upstream.

[0032] Adjust the brightness and contrast of multi-frame image sequences to enhance the details of water surface texture and the brightness and darkness of luminescent tracers in the images.

[0033] As a further optimization, the process of processing the visible light image based on the pyramid-layered optical flow method to obtain surface velocity data at the location of the dike breach refers to:

[0034] The multi-frame image sequence is downsampled layer by layer to generate a multi-scale image stack;

[0035] Initialize the optical flow field, calculate the coarse displacement between the top layer images, and upsample the upper layer optical flow field to the next layer resolution as the initial value. Combine the current layer image data to iteratively optimize the displacement.

[0036] The final optical flow field is smoothed and filtered to remove outliers;

[0037] By combining geographic correction parameters, pixel displacement is converted into actual flow velocity and used as the surface flow velocity data at the location of the dike breach.

[0038] As a further optimization, the deep learning model is a U-Net model with multimodal input fusion attention;

[0039] The multimodal input fusion attention U-Net model includes a dual convolutional layer, an attention mechanism module, and a feature concatenation module, wherein:

[0040] After obtaining the three inputs of the U-Net model with multimodal input fusion attention, the model input three-channel matrix has a size of H×W×3. The initial feature C1 with a size of H×W×64 is extracted through a double convolutional layer. Then it is processed in parallel by two branches. One branch C1 is input to the attention mechanism module to obtain a feature layer A1 of the same size. The other branch C1 is max pooled to reduce the dimensionality to a feature layer M2 with a size of H / 2×W / 2×64. After double convolution, it obtains a feature layer C2 of the same size. It is further passed through the attention module to obtain the deep abstract feature A2. Then it is restored to the original resolution through bilinear upsampling to obtain U2 with a size of H×W×64.

[0041] By using a skip connection, A1 and U2 are concatenated through the feature concatenation module to obtain feature word C3 of size H×W×128. C3 is then double-convolved to obtain feature layer C4 of size H×W×64. Finally, a 1*1 convolution kernel is used to obtain the output matrix of underwater elevation data of size H×W×1, which is used as the real underwater terrain elevation data.

[0042] As a further optimization, the U-Net model with multimodal input fusion attention is trained by adding the root mean square error of the underwater terrain elevation data measured by the second UAV to its loss function. The loss function is expressed as follows:

[0043] ,

[0044] in, The root mean square error between the predicted and measured values ​​of all underwater topographic elevation data. The root mean square error of the data corresponding to the underwater terrain location measured by the second UAV is given. As a weighting factor, it is greater than or equal to 1;

[0045] During training, the optimization function of the U-Net model with multimodal input fusion attention is obtained through the Adam optimizer.

[0046] As a further optimization, both root mean square errors are calculated using the following formula:

[0047] ,

[0048] in, To predict underwater topographic elevation data, This is actual underwater topographic elevation data. This represents the number of underwater topographic elevation data points.

[0049] On the other hand, the present invention also provides an intelligent underwater topography reconstruction system for dike breaches based on multi-UAV monitoring, applied to the aforementioned intelligent underwater topography reconstruction method for dike breaches based on multi-UAV monitoring, comprising:

[0050] The first client is equipped with a deep learning module, which is used to collect existing underwater topographic data and surface flow velocity data, establish a database, train a deep learning model based on the established database, and deploy the trained deep learning model through a dynamic link library.

[0051] The first data acquisition module is used to acquire visible light images of the location of the dike breach via the first UAV;

[0052] The second data acquisition module is used to acquire underwater topographic elevation data along the way when the second UAV floats along the water flow line with the propulsion of the natural water flow.

[0053] The second client is equipped with a data processing module, which is used to process the visible light image, obtain surface flow velocity data at the location of the dike breach and river cross-sectional width data at different locations near the location of the dike breach, and match the coordinates of local underwater measuring points in the underwater topographic elevation data along the route with the coordinates of the surface flow velocity data and the river cross-sectional width data through coordinate transformation.

[0054] The first data transmission module is located between the first client and the second client. It is used to input the matched surface velocity data, the width of the river channel cross section at different locations near the breach of the dike, and the underwater topographic elevation data along the route from the second client to the deep learning model deployed in the first client.

[0055] The second data transmission module is located between the first client and the third client. It is used to reconstruct the underwater topography at the breach of the dike in the first client by using a deployed deep learning model and then transmit the data to the third client through the second data transmission module.

[0056] The third client is equipped with a display module, which is used to display the underwater topography at the reconstructed dike breach.

[0057] The beneficial effects of this invention are as follows: On the one hand, this invention only needs to use surface flow velocity data and river channel cross-sectional width, combined with a small amount of underwater topographic data, to ensure the accuracy of underwater topographic reconstruction. On the other hand, this invention fuses the above three types of data through multi-source data and uses a deep learning model for training, which ultimately ensures the accuracy of underwater topographic reconstruction and prediction. Especially under complex environmental conditions, the use of deep learning technology makes up for the shortcomings of traditional mathematical models in describing the complex relationship between surface flow velocity and underwater topography, thereby improving the accuracy of underwater topographic prediction. Attached Figure Description

[0058] Figure 1 This is a flowchart of the intelligent underwater topography reconstruction method for dike breaches based on multi-UAV monitoring in Embodiment 1 of the invention.

[0059] Figure 2 This is a flowchart illustrating the data acquisition process using a first drone and a second drone in Embodiment 1 of the present invention.

[0060] Figure 3 This is a flowchart of the visibility determination steps before processing the visible light image based on the pyramid layered optical flow method in Embodiment 1 of the present invention.

[0061] Figure 4 This is a schematic diagram of the training framework of the U-Net model with multimodal input fusion attention in Embodiment 1 of the invention;

[0062] Figure 5 This is a schematic diagram of the composition structure of the underwater terrain intelligent reconstruction system for dike breaches based on multi-UAV monitoring in Embodiment 2 of the present invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0064] Example 1

[0065] This embodiment provides a method for intelligent reconstruction of underwater topography in dike breaches based on multi-UAV monitoring. See the flowchart below. Figure 1 The method includes the following steps:

[0066] S1. Collect existing underwater topographic data and surface current velocity data, and establish a database;

[0067] S2. Train a deep learning model based on the established database;

[0068] S3. Deploy the trained deep learning model using a dynamic link library;

[0069] S4. Obtain visible light images of the breach location of the dike using the first UAV, and obtain underwater topographic elevation data along the way using the second UAV while floating along the water flow line with the propulsion of the natural water flow.

[0070] S5. Process the visible light image to obtain surface flow velocity data at the location of the dike breach and river channel cross-sectional width data at different locations near the location of the dike breach.

[0071] S6. Through coordinate transformation, the coordinates of local underwater measuring points in the underwater topographic elevation data along the route are matched with the coordinates of surface velocity data and river channel width data. The matched surface velocity data, river channel width at different locations near the breach of the dike, and underwater topographic elevation data along the route are then input into the deployed deep learning model.

[0072] S7. Reconstruct the underwater topography at the breach in the dike using a deployed deep learning model.

[0073] In this embodiment, high-precision and rapid reconstruction can be achieved by working collaboratively with multiple UAVs when a dike breaches. Compared to traditional underwater topographic surveying, this embodiment relies on the measurement of only a small amount of underwater topographic data. It uses UAV oblique photography combined with optical flow methods to measure surface flow velocity and the shape of the dike above water, reconstructing the underwater topography near the breach. Since the UAVs operate in the air when measuring surface flow velocity, their measurement process is not affected by the high-speed water flow at the time of the breach.

[0074] In practical applications, although surface flow velocity and underwater topography are strongly correlated, a simple mathematical model cannot establish a mapping relationship between them, and underwater topography is not the sole determinant of surface flow velocity. Therefore, this embodiment further utilizes deep learning methods to establish a mapping model between surface flow velocity and underwater topography. This model not only overcomes the shortcomings of simple mathematical models in accurately describing the complex relationship between surface flow velocity and underwater topography, but also improves the accuracy and precision of underwater topography prediction and reconstruction by combining levee topography data measured by UAVs and a limited number of underwater topography measurement points.

[0075] Furthermore, the amphibious unmanned aerial vehicle (UAV) operation scheme used in this embodiment differs from the traditional scheme. The traditional measurement scheme involves unmanned vessels and amphibious UAVs relying on their own power to navigate or fly along a planned route when the water flow speed is low. In this embodiment, when the amphibious UAV is measuring, after it comes to rest on the water surface, it shuts off its flight or navigation power source and drifts rapidly downstream with the water flow, measuring the corresponding underwater topography during its floating process.

[0076] It should be noted that in this embodiment, the measurement of multi-source data is mainly completed by the cooperation of the first UAV and the second UAV. Unlike the traditional UAV measurement method, in this embodiment, the first UAV and the second UAV need to be configured separately. Therefore, in this embodiment, before the first UAV acquires the visible light image of the breach location of the dike and the second UAV acquires the underwater topographic elevation data along the way when floating along the water flow line with the propulsion of the natural water flow, it is also necessary to include: configuring the first UAV equipped with an image acquisition device and the second UAV equipped with a sonar device.

[0077] After configuring the first and second drones, data collection can be performed using them, corresponding to steps S4-S6 in this embodiment. See the flowchart below. Figure 2 It includes the following steps:

[0078] a. Obtain visible light images of the location of the dike breach using the first drone;

[0079] b. Acquire underwater topographic elevation data along the course of the water flow by using a second UAV to float along the water flow line with the help of the natural water flow.

[0080] c. Process the visible light image based on the pyramidal layered optical flow method to obtain surface flow velocity data at the location of the dike breach;

[0081] d. Perform image recognition and segmentation processing on the visible light image to obtain the cross-sectional width data of the river channel at different locations near the breach of the dike;

[0082] e. By transforming coordinates, the coordinates of local underwater measuring points in the underwater topographic elevation data along the river are matched with the coordinates of surface velocity data and river channel width data.

[0083] f. Input the matched surface velocity data, the width of the river channel cross section at different locations near the breach of the dike, and the underwater topographic elevation data along the route into the deployed deep learning model.

[0084] The first UAV equipped with an image acquisition device refers to flying the first UAV to a position where the image acquisition device can fully capture images of the location of the dike breach and the upstream and downstream flow locations.

[0085] The aforementioned amphibious second UAV equipped with sonar devices refers to: obtaining the intervals of multiple amphibious second UAVs equipped with sonar devices, the number of local underwater sampling points and the sampling frequency of a single second UAV equipped with sonar devices based on the upstream-to-downstream process; flying the second UAV to the upstream of the dike breach and landing on the water surface at the aforementioned intervals.

[0086] In practical applications, the interval setting can be adjusted according to the actual situation of the dike breach. In this embodiment, it can be set to 20m. The water flow velocity threshold also needs to be set according to the actual situation of the dike breach. For the first threshold upstream, this embodiment sets it to 5m / s. That is, when the water flow velocity is greater than 5m / s, the amphibious drone (second drone) cannot provide enough power to navigate along the given route. At this time, the flight and navigation power system of the amphibious drone is turned off, and it floats along the water flow line with the propulsion of the natural water flow. In this way, the drone can float in the direction of the water flow, thereby collecting underwater topographic data along the way. For the second threshold downstream, this embodiment sets it to 0-0.5m / s. That is, when the amphibious drone floats to the area where the downstream water flow velocity is lower than 0-0.5m / s, the amphibious drone returns to the predetermined starting position.

[0087] Here are some options for amphibious drones: equip them with a lightweight single-beam or multi-beam echo sounder to measure underwater topography; equip them with a real-time dynamic positioning global navigation satellite system to record the drone's drift trajectory in real time; or equip them with an inertial navigation module to compensate for changes in the drone's attitude (roll and pitch) and ensure that the acoustic beam is incident vertically.

[0088] Therefore, in this embodiment, acquiring the visible light image of the dike breach location via the first UAV means:

[0089] The first UAV uses an image acquisition device to capture multiple frames of visible light images at a set shooting speed, forming a multi-frame image sequence;

[0090] The acquisition of underwater topographic elevation data along the course of a second UAV while it floats along the streamline of the water flow using the propulsion of the natural water current refers to:

[0091] After the second UAV lands on the water surface at the intervals, it acquires the upstream water flow velocity and determines whether it reaches the first threshold. When it does, it shuts off the flight and navigation power system of the second UAV and acquires underwater terrain elevation data along the way while floating along the water flow line with the propulsion of the natural water flow.

[0092] When the second drone drifts downstream, it obtains the downstream water flow velocity and determines whether it is less than a second threshold. If it is less than a threshold, it activates the flight and navigation power system of the second drone and flies back to its starting position.

[0093] In this embodiment, traditional methods for measuring water surface velocity face numerous challenges under nighttime or low-visibility conditions, especially in emergencies such as dike breaches, where effective monitoring cannot rely on natural light sources. To address this issue, this embodiment utilizes a third drone to scatter luminescent objects upstream of the breach, serving as a reference for optical flow velocity measurement, thereby achieving high-precision water surface velocity measurement in low-visibility environments. Specifically, this embodiment employs the following setup: for nighttime or low-visibility conditions with visibility less than 100 meters, one or more third drones are deployed 30-50 meters upstream of the breach, simultaneously scattering luminescent objects. By increasing ambient light, the accuracy of optical flow velocity measurement is improved. When the breach width is less than 50 meters, two additional third drones are deployed; for every 200 meters increase in breach width, one more third drone is added. When visibility is greater than 100 meters, the initial number of third drones is maintained.

[0094] Therefore, in this embodiment, see Figure 3 Before processing the visible light image using the pyramid-based hierarchical optical flow method, the following steps may also be included:

[0095] The system determines whether the visibility at the location of the dike breach and the upstream and downstream flow points is below a set threshold. If it is below the threshold, a third drone is added, and the third drone is used to drop luminous tracers before the second drone lands upstream.

[0096] Adjust the brightness and contrast of multi-frame image sequences to enhance the details of water surface texture and the brightness and darkness of luminescent tracers in the images.

[0097] In practical application, the number of underwater terrain data samples for a single second UAV can be estimated by the following process: if the amphibious UAV drifts from upstream to downstream for a distance of about 40m, the average water flow velocity is 8m / s, and the water depth detector sampling frequency is 10Hz, then the number of sampling points is 50.

[0098] After the configuration of the first, second, and third drones is completed, each drone can be put into operation according to its configuration.

[0099] In this embodiment, the acquisition conditions for the visible light images obtained by the first UAV are not uniform, which makes the various image features in the visible light images not obvious. Therefore, it is necessary to process the visible light images to adjust the brightness and contrast of the images, thereby enhancing the details of the water surface texture and the bright and dark features of natural or artificial tracers in the images. At the same time, in order to solve the tracking of large displacement motion and video frame loss problems, these images can be processed based on the pyramid layered optical flow method to further extract the surface flow velocity of the water flow.

[0100] Therefore, in this embodiment, processing the visible light image to obtain surface flow velocity data at the location of the dike breach and river cross-sectional width data at different locations near the location of the dike breach refers to: processing the visible light image based on the pyramid layered optical flow method to obtain surface flow velocity data at the location of the dike breach, and performing image recognition and segmentation processing on the visible light image to obtain river cross-sectional width data at different locations near the location of the dike breach.

[0101] Specifically, in this embodiment, to solve the problems of tracking large displacement motions and video frame dropping, the step of processing the visible light image based on the pyramid hierarchical optical flow method to obtain surface flow velocity data at the location of the dike breach refers to:

[0102] In the low-resolution layer, a pixel represents a larger area in the original image. A large physical displacement is manifested as a smaller pixel displacement in the low-resolution image. Therefore, multi-frame image sequences are downsampled layer by layer (e.g., 4-layer pyramid, downsampling factor 0.5) to generate a multi-scale image stack.

[0103] In order to break down the impossible large-scale search problem into a series of controllable local optimization problems, and at the same time to approximate physical reality by utilizing the multi-scale characteristics of images, the optical flow field is initialized, the coarse displacement between the top layer images is calculated, and the upper layer optical flow field is upsampled to the next layer resolution as the initial value. The displacement is then iteratively optimized by combining the current layer image data.

[0104] In order to remove outlier data, the final optical flow field is smoothed and filtered to eliminate outliers.

[0105] In order to convert the displacement on the image into actual displacement during coordinate transformation, the pixel displacement is converted into actual flow velocity by combining geographic correction parameters (such as satellite attitude and viewpoint) and used as the surface flow velocity data of the obtained dike breach location.

[0106] Since it is necessary to combine surface flow velocity and underwater topography data along the course for subsequent deep learning, it is also necessary to obtain the river channel cross-sectional width near the dike breach. Therefore, in this embodiment, the river channel cross-sectional width at different locations near the dike breach refers to the river channel cross-sectional width at different locations between the dike breach location and its upstream and downstream processes.

[0107] In practical applications, to guide closure operations, it is necessary to measure the underwater topography at different locations 10-20m upstream and downstream of the breach cross-section along the flow direction. To constrain the subsequent deep learning model and improve its prediction accuracy, the width of the river channel cross-section at different locations 10-20m upstream and downstream of the breach cross-section along the flow direction is calculated to determine the width of different levee cross-sections, and then to estimate the average velocity of the water flow below the water surface.

[0108] It should be noted that the deep learning model can be different deep learning algorithms, such as Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), U-Net, etc. Furthermore, attention mechanisms can be incorporated to further enhance the algorithm's performance, reducing computational redundancy and improving processing efficiency by focusing on details in important regions. In this embodiment, the deep learning model is selected and constructed using a U-Net model with an attention mechanism fused to it. Therefore, in this embodiment, the deep learning model is an end-to-end multimodal input U-Net model with attention fused.

[0109] See Figure 4 In this embodiment, the input channels of the U-Net model with multimodal input fusion attention include surface velocity field (H×W grid, normalized to 0-1), levee cross-section width (width distribution along the river channel, converted to H×W grid, scalar values ​​at the same position), and sparse measuring point mask (H×W matrix with known measuring point positions for measuring underwater elevation values, and the rest being 0). The output is dense underwater elevation values ​​(H×W grid), where H is the number of data points in the velocity field image numerical direction, and W is the number of points in the horizontal direction.

[0110] The network structure of the U-Net model, which integrates multimodal input and attention, can include a dual convolutional layer, an attention mechanism module, and a feature concatenation module. Specifically, the first convolutional layer extracts local basic features, and the first convolutional layer combines higher-order features on the primary features. The attention mechanism module can suppress interference information and enhance the detection of small targets. The feature concatenation module can reconstruct accurate water-land boundaries for irregular flooded areas.

[0111] After obtaining the three inputs of the U-Net model with multimodal input fusion attention, the model input three-channel matrix has a size of H×W×3. The initial feature C1 with a size of H×W×64 is extracted through a double convolutional layer. Then it is processed in parallel by two branches. One branch C1 is input to the attention mechanism module to obtain a feature layer A1 of the same size. The other branch C1 is max pooled to reduce the dimensionality to a feature layer M2 with a size of H / 2×W / 2×64. After double convolution, it obtains a feature layer C2 of the same size. It is further passed through the attention module to obtain the deep abstract feature A2. Then it is restored to the original resolution through bilinear upsampling to obtain U2 with a size of H×W×64.

[0112] By using a skip connection, A1 and U2 are concatenated through the feature concatenation module to obtain feature word C3 of size H×W×128. C3 is then double-convolved to obtain feature layer C4 of size H×W×64. Finally, a 1*1 convolution kernel is used to obtain the output matrix of underwater elevation data of size H×W×1, which is used as the real underwater terrain elevation data.

[0113] It should be noted that, during the training process of the multimodal input fusion attention U-Net model, in order to improve prediction accuracy by measuring some underwater terrain data points, the root mean square error corresponding to the underwater terrain elevation data along the route measured by the second UAV is additionally added to its loss function. The loss function is expressed as follows:

[0114] ,

[0115] in, The root mean square error between the predicted and measured values ​​of all underwater topographic elevation data. The root mean square error of the data corresponding to the underwater terrain location measured by the second UAV is given. The weighting factor is greater than or equal to 1; the specific number is 10 points spaced 1 meter apart.

[0116] During training, the optimization function of the U-Net model with multimodal input fusion attention is obtained through the Adam optimizer.

[0117] Here, both root mean square errors can be calculated using the following formula:

[0118] ,

[0119] in, To predict underwater topographic elevation data, This is actual underwater topographic elevation data. This represents the number of underwater topographic elevation data points.

[0120] In practical applications, the deep learning model trained in this embodiment can be deployed through dynamic link libraries or other means.

[0121] In summary, this embodiment, through the collaborative work of multiple UAVs, combined with oblique photography, optical flow, deep learning, and an innovative amphibious UAV operation scheme, achieved high-precision and rapid reconstruction of the underwater topography near the breach when a dike breaches. It overcomes the limitations of existing technologies in complex environments, realizes topographic measurement under high water flow conditions, and provides strong technical support for emergency response to dike breaches.

[0122] Example 2

[0123] Based on Example 1, this example provides an intelligent underwater topography reconstruction system for dike breaches based on multi-UAV monitoring. A schematic diagram of the system's structure can be found here. Figure 5 The system in this embodiment may include:

[0124] The first client is equipped with a deep learning module, which is used to collect existing underwater topographic data and surface flow velocity data, establish a database, train a deep learning model based on the established database, and deploy the trained deep learning model through a dynamic link library.

[0125] The first data acquisition module is used to acquire visible light images of the location of the dike breach via the first UAV;

[0126] The second data acquisition module is used to acquire underwater topographic elevation data along the way when the second UAV floats along the water flow line with the propulsion of the natural water flow.

[0127] The second client is equipped with a data processing module, which is used to process the visible light image, obtain surface flow velocity data at the location of the dike breach and river cross-sectional width data at different locations near the location of the dike breach, and match the coordinates of local underwater measuring points in the underwater topographic elevation data along the route with the coordinates of the surface flow velocity data and the river cross-sectional width data through coordinate transformation.

[0128] The first data transmission module is located between the first client and the second client. It is used to input the matched surface velocity data, the width of the river channel cross section at different locations near the breach of the dike, and the underwater topographic elevation data along the route from the second client to the deep learning model deployed in the first client.

[0129] The second data transmission module is located between the first client and the third client. It is used to reconstruct the underwater topography at the breach of the dike in the first client by using a deployed deep learning model and then transmit the data to the third client through the second data transmission module.

[0130] The third client is equipped with a display module, which is used to display the underwater topography at the reconstructed dike breach.

[0131] In this embodiment, the deployed deep learning model can be called by other clients (such as a third client) through various communication interfaces set by the first client, thereby obtaining the underwater terrain at the dike breach reconstructed in the first client and displaying it in the relevant software (such as 3D software) of other clients.

[0132] As can be seen from the description of this embodiment, the application scenario and implementation principle of this embodiment are the same as those of Embodiment 1, so they will not be repeated here.

[0133] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent reconstruction of underwater topography in dike breaches based on multi-UAV monitoring, characterized in that, Includes the following steps: Collect existing underwater topographic data and surface flow velocity data, and establish a database; A deep learning model is trained based on the established database; Deploy the trained deep learning model using dynamic link libraries; The first drone acquires visible light images of the location of the dike breach, and the second drone acquires underwater topographic elevation data along the way while floating along the water flow line with the propulsion of the natural water flow. The acquisition of visible light images of the location of the dike breach via the first drone refers to: The first UAV uses an image acquisition device to capture multiple frames of visible light images at a set shooting speed, forming a multi-frame image sequence; The acquisition of underwater topographic elevation data along the course of a second UAV while it floats along the streamline of the water flow using the propulsion of the natural water current refers to: After the second UAV lands on the water at a preset interval, it acquires the upstream water flow velocity and determines whether it has reached the first threshold. When it does, it shuts off the flight and navigation power system of the second UAV and acquires underwater terrain elevation data along the way by floating along the water flow line with the propulsion of the natural water flow. When the second drone drifts downstream, it obtains the downstream water flow velocity and determines whether it is less than the second threshold. If it is less than the threshold, it activates the flight and navigation power system of the second drone and flies back to its starting position. The visible light image is processed to obtain surface flow velocity data at the location of the dike breach and river channel cross-sectional width data at different locations near the dike breach. Processing the visible light image to obtain surface flow velocity data at the location of the dike breach and river channel cross-sectional width data at different locations near the dike breach refers to: The visible light image is processed using the pyramidal layered optical flow method to obtain surface flow velocity data at the location of the dike breach. Image recognition and segmentation are then performed on the visible light image to obtain cross-sectional width data of the river channel at different locations near the dike breach. The width of the river channel at different locations near the breach of the dike refers to the width of the river channel at different locations between the breach of the dike and its upstream and downstream sections. By transforming the coordinates, the coordinates of local underwater measuring points in the underwater topographic elevation data along the route are matched with the coordinates of surface velocity data and river channel width data. The matched surface velocity data, river channel width at different locations near the breach of the dike, and underwater topographic elevation data along the route are then input into the deployed deep learning model. The underwater topography at the breach in the dike is reconstructed using a well-deployed deep learning model.

2. The method for intelligent reconstruction of underwater topography of dike breaches based on multi-UAV monitoring as described in claim 1, characterized in that, Before acquiring visible light images of the breach location of the dike through the first UAV and acquiring underwater topographic elevation data along the way by the second UAV floating along the water flow line with the propulsion of the natural water flow, the method further includes: a first UAV equipped with an image acquisition device and a second amphibious UAV equipped with a sonar device. The first UAV equipped with an image acquisition device refers to: The first drone will be flown to a position where it can fully capture images of the location of the dike breach and the upstream and downstream flow areas through the image acquisition device. The aforementioned amphibious second unmanned aerial vehicle equipped with sonar refers to: Based on the upstream-to-downstream process, the intervals between multiple amphibious second UAVs equipped with sonar devices, the number of local underwater sampling points and the sampling frequency of a single second UAV equipped with sonar devices are obtained. The second unmanned aircraft will fly upstream of the breach in the dike and land on the water at the intervals mentioned above.

3. The method for intelligent reconstruction of underwater topography of dike breaches based on multi-UAV monitoring as described in claim 1, characterized in that, Before processing the visible light image using the pyramid-based hierarchical optical flow method, the method further includes: Obtain the visibility of the location of the dike breach and the upstream and downstream flow locations, and determine whether it is below the set threshold. If it is below the threshold, add a third drone and use the third drone to drop luminous tracers before the landing position of the second drone upstream. Adjust the brightness and contrast of multi-frame image sequences to enhance the details of water surface texture and the brightness and darkness of luminescent tracers in the images.

4. The method for intelligent reconstruction of underwater topography of dike breaches based on multi-UAV monitoring as described in claim 1, characterized in that, The process of processing the visible light image based on the pyramid-layered optical flow method to obtain surface velocity data at the location of the dike breach refers to: The multi-frame image sequence is downsampled layer by layer to generate a multi-scale image stack; Initialize the optical flow field, calculate the coarse displacement between the top layer images, and upsample the upper layer optical flow field to the next layer resolution as the initial value. Combine the current layer image data to iteratively optimize the displacement. The final optical flow field is smoothed and filtered to remove outliers; By combining geographic correction parameters, pixel displacement is converted into actual flow velocity and used as the surface flow velocity data at the location of the dike breach.

5. The method for intelligent reconstruction of underwater topography of dike breaches based on multi-UAV monitoring according to claim 1, characterized in that, The deep learning model is a U-Net model that integrates multimodal input and attention. The multimodal input fusion attention U-Net model includes a dual convolutional layer, an attention mechanism module, and a feature concatenation module, wherein: After obtaining the three inputs of the U-Net model with multimodal input fusion attention, the model input three-channel matrix has a size of H×W×3. The initial feature C1 with a size of H×W×64 is extracted through a double convolutional layer. Then it is processed in parallel by two branches. One branch C1 is input to the attention mechanism module to obtain a feature layer A1 of the same size. The other branch C1 is max pooled to reduce the dimensionality to a feature layer M2 with a size of H / 2×W / 2×64. After double convolution, it obtains a feature layer C2 of the same size. It is further passed through the attention module to obtain the deep abstract feature A2. Then it is restored to the original resolution through bilinear upsampling to obtain U2 with a size of H×W×64. By using a skip connection, A1 and U2 are concatenated through the feature concatenation module to obtain feature word C3 of size H×W×128. C3 is then double-convolved to obtain feature layer C4 of size H×W×64. Finally, a 1*1 convolution kernel is used to obtain the output matrix of underwater elevation data of size H×W×1, which is used as the real underwater terrain elevation data.

6. The method for intelligent reconstruction of underwater topography of dike breaches based on multi-UAV monitoring as described in claim 5, characterized in that, During training, the U-Net model with multimodal input fusion attention additionally incorporates the root mean square error of the underwater terrain elevation data measured by the second UAV into its loss function. The loss function is expressed as follows: ， in, The root mean square error between the predicted and measured values ​​of all underwater topographic elevation data. The root mean square error of the data corresponding to the underwater terrain location measured by the second UAV is given. As a weighting factor, it is greater than or equal to 1; During training, the optimization function of the U-Net model with multimodal input fusion attention is obtained through the Adam optimizer.

7. The method for intelligent reconstruction of underwater topography of dike breaches based on multi-UAV monitoring as described in claim 6, characterized in that, Both root mean square errors are calculated using the following formula: ， in, To predict underwater topographic elevation data, This is actual underwater topographic elevation data. This represents the number of underwater topographic elevation data points.

8. A smart underwater terrain reconstruction system for dike breaches based on multi-UAV monitoring, applied to the smart underwater terrain reconstruction method for dike breaches based on multi-UAV monitoring as described in any one of claims 1-7, characterized in that, include: The first client is equipped with a deep learning module, which is used to collect existing underwater topographic data and surface flow velocity data, establish a database, train a deep learning model based on the established database, and deploy the trained deep learning model through a dynamic link library. The first data acquisition module is used to acquire visible light images of the location of the dike breach via the first UAV; The second data acquisition module is used to acquire underwater topographic elevation data along the way when the second UAV floats along the water flow line with the propulsion of the natural water flow. The second client is equipped with a data processing module, which is used to process the visible light image, obtain surface flow velocity data at the location of the dike breach and river cross-sectional width data at different locations near the location of the dike breach, and match the coordinates of local underwater measuring points in the underwater topographic elevation data along the route with the coordinates of the surface flow velocity data and the river cross-sectional width data through coordinate transformation. The first data transmission module is located between the first client and the second client. It is used to input the matched surface velocity data, the width of the river channel cross section at different locations near the breach of the dike, and the underwater topographic elevation data along the route from the second client to the deep learning model deployed in the first client. The second data transmission module is located between the first client and the third client. It is used to reconstruct the underwater topography at the breach of the dike in the first client by using a deployed deep learning model and then transmit the data to the third client through the second data transmission module. The third client is equipped with a display module, which is used to display the underwater topography at the reconstructed dike breach.

Images