Dike breach underwater terrain intelligent reconstruction method and system based on multi-unmanned aerial vehicle monitoring

Abstract

The invention belongs to the technical field of water conservancy measurement, and provides a dike breach underwater topography intelligent reconstruction method and system based on multi-unmanned aerial vehicle monitoring, and the main scheme is that the method comprises the steps: collecting existing underwater topography data and surface flow velocity data, building a database, training a deep learning model, and deploying the trained deep learning model through a dynamic link library; obtaining a visible light image of a dike breach position through the first unmanned aerial vehicle, and obtaining on-way underwater terrain elevation data through the second unmanned aerial vehicle; the visible light image is processed, and surface flow velocity data of the dike breach position and river section width data of different positions near the dike breach position are obtained; and through coordinate transformation, matching the coordinates of the local underwater measuring points in the on-way underwater terrain elevation data with the coordinates of the surface flow velocity data and the river channel section width data, inputting the matched coordinates into a deployed deep learning model, and reconstructing the underwater terrain at the dike breach through the deployed deep learning model.

Description

Technical Field

[0001] The present invention relates to the field of water conservancy technology, and in particular to a method and system for intelligently reconstructing underwater terrain of a dike breach based on multi-UAV monitoring. Background Art

[0002] Accurate and rapid reconstruction of underwater terrain is crucial for emergency response to dike breaches, but existing technologies have significant limitations in such complex environments. Traditional manual surveying methods are difficult to implement in dike breaches, and the harsh environmental conditions threaten the lives of surveyors, limiting their practical application.

[0003] Currently, underwater topography surveys are often performed using unmanned vessels or drones equipped with sonar or laser equipment. These typically rely on amphibious drones or unmanned vessels following a predetermined route, covering the reconstruction area to collect data from different sections. Three-dimensional underwater topography reconstruction is then performed using algorithms such as inverse distance weighted interpolation or kriging interpolation. However, this method has significant drawbacks: Firstly, the measurement process is time-consuming, and the accuracy of underwater topography reconstruction depends on the number of measurement points. Secondly, during a dike breach, the high water velocity and harsh environment make it difficult for unmanned vessels or drones to operate near the water surface, and they cannot collect sufficient underwater topographic points, resulting in low reconstruction accuracy or even incompletion.

[0004] Furthermore, drone aerial inspections at night or in low-visibility conditions present significant challenges. Even with infrared camera technology, the image resolution is low, and the effective information that can be obtained is limited, affecting measurement accuracy and reconstruction results.

[0005] Patent application CN119354152A discloses a method and device for rapidly reconstructing underwater topography at a dike breach using a drone. However, this patent application primarily uses drones to measure a large amount of underwater topography data, then uses flow velocity data, infrared images, and visible light images to correct the topography data and reconstruct the underwater topography at the breach. In reality, measuring underwater topography data when a dike breach occurs is extremely difficult. Summary of the Invention

[0006] The purpose of the present invention is to provide a method and system for intelligent reconstruction of the underwater terrain of a dike breach based on multi-UAV monitoring. It can use less underwater terrain data, combine it with surface flow velocity data and river section width, and train a deep learning model to accurately complete the high-precision reconstruction of the underwater terrain of the dike breach.

[0007] The present invention solves the technical problem and adopts the following technical solution: In one aspect, the present invention provides a method for intelligently reconstructing underwater terrain of a dike breach based on multi-UAV monitoring, comprising the following steps: Collect existing underwater topography data and surface velocity data and establish a database; Training a deep learning model based on the established database; Deploy the trained deep learning model through the dynamic link library; The first UAV is used to obtain visible light images of the location of the dike breach, and the second UAV is used to obtain underwater terrain elevation data along the flow line while floating along the flow line with the help of the natural water flow. Processing the visible light image to obtain surface flow velocity data at the dike breach location and river channel cross-sectional width data at different locations near the dike breach location; Through coordinate transformation, the coordinates of the local underwater measurement points in the underwater terrain elevation data along the route are matched with the coordinates of the surface flow velocity data and the river cross-sectional width data. The matched surface flow velocity data, the river cross-sectional width at different locations near the embankment breach, and the underwater terrain elevation data along the route are then input into the deployed deep learning model. The underwater terrain at the embankment breach is reconstructed using the deployed deep learning model.

[0008] As a further optimization, before obtaining a visible light image of the dike breach location by the first drone and obtaining underwater terrain elevation data along the flow line by the second drone while floating along the flow line with the help of the natural water flow, the method further includes: configuring the first drone equipped with an image acquisition device and the second amphibious drone equipped with a sonar device; The first drone equipped with an image acquisition device is: Flying the first UAV to a position where the image acquisition device can fully capture images of the levee breach location and upstream and downstream process locations; The second amphibious drone equipped with a sonar device is: Based on the upstream-to-downstream process, the intervals between multiple amphibious second UAVs equipped with sonar devices, and the number and frequency of local underwater sampling points of a single second UAV equipped with sonar devices are obtained; The second drone is flown upstream of the dike breach and landed on the water surface at the intervals.

[0009] As a further optimization, obtaining a visible light image of the dike breach location by the first drone means: The first UAV captures multiple frames of visible light images at a set shooting speed through an image acquisition device to form a multi-frame image sequence; The method of obtaining underwater terrain elevation data along the flow line by the second drone while the drone is floating along the flow line with the help of the natural water flow refers to: When the second UAV lands on the water surface at the interval, the upstream water velocity is obtained and determined to determine whether it reaches a first threshold. If so, the flight and navigation propulsion systems of the second UAV are shut down, and the underwater terrain elevation data along the flow line is obtained while the second UAV floats along the flow line with the help of the natural water flow. When the second UAV drifts downstream, the downstream water flow velocity is obtained and it is determined whether it is less than a second threshold. If it is less than a second threshold, the flight and navigation power system of the second UAV is turned on to fly to its starting position.

[0010] As a further optimization, processing the visible light image to obtain surface flow velocity data at the dike breach location and river channel cross-sectional width data at different locations near the dike breach location means: Processing the visible light image based on a pyramid hierarchical optical flow method to obtain surface flow velocity data at the levee breach location, and performing image recognition and segmentation processing on the visible light image to obtain river channel cross-sectional width data at different locations near the levee breach location; The cross-sectional width of the river channel at different positions near the dike breach refers to the cross-sectional width of the river channel at different positions between the dike breach and its upstream and downstream processes.

[0011] As a further optimization, before processing the visible light image based on the pyramid layered optical flow method, the method further includes: Obtain visibility at the levee breach location and upstream and downstream process locations, and determine whether it is below a set threshold. If so, deploy a third drone to drop a luminous tracer before the second drone's upstream landing location. Adjust the brightness and contrast of multi-frame image sequences to enhance the water surface texture details and the bright and dark characteristics of luminous tracers in the image.

[0012] As a further optimization, the processing of the visible light image based on the pyramid layered optical flow method to obtain the surface flow velocity data at the embankment breach location refers to: Downsample the multi-frame image sequence layer by layer to generate a multi-scale image stack; Initialize the optical flow field, calculate the rough displacement between the top-level images, and upsample the upper-level optical flow field to the next-level resolution as the initial value. Combined with the current-level image data, iteratively optimize the displacement. Smooth the final optical flow field and remove outliers; Combined with geographic correction parameters, the pixel displacement is converted into actual flow velocity and used as the surface flow velocity data at the levee breach location.

[0013] As a further optimization, the deep learning model is a U-Net model with multimodal input fusion attention; The multimodal input fusion attention U-Net model includes a dual convolutional layer, an attention mechanism module and a feature splicing module, wherein: After obtaining the three inputs of the U-Net model with multimodal input fusion attention, the model input three-channel matrix size is H×W×3, and the initial feature C1 is extracted through a double convolution layer, whose size is H×W×64. It is then processed in two parallel branches. One branch C1 is input into the attention mechanism module to obtain a feature layer A1 of the same size, and the other branch C1 is subjected to maximum pooling to reduce the dimension to a feature layer M2 of size H / 2×W / 2×64. After double convolution, it obtains a feature layer C2 of the same size. The deep abstract feature A2 is further obtained through the attention module, and then restored to the original resolution through bilinear upsampling to obtain U2 of size H×W×64; Through the jump connection, A1 and U2 are channel-spliced ​​through the feature splicing module to obtain the feature word C3 of size H×W×128. C3 is subjected to double convolution to obtain the feature layer C4 of size H×W×64. Finally, through a 1*1 convolution kernel, the output matrix of underwater elevation data of size H×W×1 is obtained, which is used as the real underwater terrain elevation data.

[0014] As a further optimization, during the training process of the multimodal input fusion attention U-Net model, the root mean square error corresponding to the underwater terrain elevation data along the route measured by the second UAV is additionally added to its loss function. The loss function is expressed as: , in, is the root mean square error between the predicted and measured values ​​of all underwater terrain elevation data, is the root mean square error of the data corresponding to the underwater terrain position measured by the second UAV, is the trade-off factor, greater than or equal to 1; During the training process of the multimodal input fusion attention U-Net model, its optimization function is obtained through the Adam optimizer.

[0015] As a further optimization, the two root mean square errors are calculated by the following formula: , in, To predict underwater terrain elevation data, is the actual underwater terrain elevation data, is the number of underwater terrain elevation data points.

[0016] On the other hand, the present invention also provides a system for intelligent reconstruction of underwater terrain of dike breaches based on multi-UAV monitoring, which is applied to the method for intelligent reconstruction of underwater terrain of dike breaches based on multi-UAV monitoring, comprising: The first client is equipped with a deep learning module, which is used to collect existing underwater topography data and surface flow velocity data, establish a database, train a deep learning model based on the established database, and deploy the trained deep learning model through a dynamic link library; a first data acquisition module, configured to acquire a visible light image of the dike breach location via a first drone; The second data acquisition module is used to obtain underwater terrain elevation data along the flow line when the second UAV floats along the flow line with the help of the natural water flow; The second client is equipped with a data processing module for processing the visible light image to obtain surface flow velocity data at the dike breach location and river channel cross-sectional width data at different locations near the dike breach location, and through coordinate transformation, matches the coordinates of the local underwater measurement points in the underwater terrain elevation data along the process with the coordinates of the surface flow velocity data and the river channel cross-sectional width data; A first data transmission module is provided between the first client and the second client, and is used to input the matched surface flow velocity data, the river channel cross-sectional width at different locations near the embankment breach, and the underwater terrain elevation data along the river channel from the second client into the deep learning model deployed in the first client through the first data transmission module; A second data transmission module is provided between the first client and the third client, and is used to reconstruct the underwater topography at the dike breach using the deployed deep learning model in the first client and transmit the reconstructed topography to the third client via the second data transmission module; The third client is equipped with a display module for displaying the reconstructed underwater terrain at the embankment breach.

[0017] The beneficial effects of the present invention are: on the one hand, the present invention only needs to use surface flow velocity data and river cross-sectional width, combined with a small amount of underwater terrain data, to ensure the accuracy of underwater terrain reconstruction; on the other hand, the present invention fuses the above three data through multi-source data and uses a deep learning model for training, ultimately ensuring the accuracy of underwater terrain reconstruction and prediction. Especially under complex environmental conditions, the use of deep learning technology makes up for the shortcomings of traditional mathematical models in describing the complex relationship between surface flow velocity and underwater terrain, thereby improving the accuracy of underwater terrain prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of the method for intelligent reconstruction of underwater terrain of a dike breach based on multi-UAV monitoring in embodiment 1 of the invention; Figure 2 This is a flow chart of data collection by the first UAV and the second UAV in the first embodiment of the present invention; Figure 3This is a flowchart of the visibility determination step before processing the visible light image based on the pyramid layered optical flow method in the first embodiment of the present invention; Figure 4 Schematic diagram of the training framework of the U-Net model with multimodal input fusion attention in Example 1 of the invention; Figure 5 Schematic diagram of the composition structure of the intelligent reconstruction system for underwater terrain of dike breaches based on multi-UAV monitoring in Example 2 of the present invention. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0020] Example 1

[0021] This embodiment provides a method for intelligent reconstruction of underwater terrain of dike breach based on multi-UAV monitoring. Figure 1 , wherein the method comprises the following steps: S1. Collect existing underwater topography data and surface velocity data and establish a database; S2. Training a deep learning model based on the established database; S3. Deploy the trained deep learning model through the dynamic link library; S4. Obtaining a visible light image of the dike breach location using the first UAV, and obtaining underwater terrain elevation data along the flow line using the second UAV while floating along the flow line with the help of the natural water flow; S5. Processing the visible light image to obtain surface flow velocity data at the dike breach location and river channel cross-sectional width data at different locations near the dike breach location; S6. Match the coordinates of the local underwater measurement points in the underwater terrain elevation data along the route with the coordinates of the surface flow velocity data and the river cross-sectional width data through coordinate transformation, and input the matched surface flow velocity data, the river cross-sectional width at different locations near the embankment breach, and the underwater terrain elevation data along the route into the deployed deep learning model; S7. Reconstruct the underwater terrain at the embankment breach using the deployed deep learning model.

[0022] In this embodiment, multiple drones working together enable high-precision and rapid reconstruction of dike breaches. Compared to traditional underwater topography surveys, this embodiment relies on only a limited amount of underwater topographic data. By combining drone oblique photography with optical flow to measure surface flow velocity and the surface dike topography, the underwater topography near the breach can be reconstructed. Because drones operate in mid-air when measuring surface flow velocity, the measurement process is unaffected by the high-speed water flow during a breach.

[0023] In practical applications, although surface velocity and underwater topography are strongly correlated, a simple mathematical model cannot be used to establish a mapping relationship between surface velocity and underwater topography. Furthermore, underwater topography is not the sole factor determining surface velocity. Therefore, this embodiment further utilizes deep learning methods to establish a mapping model between surface velocity and underwater topography. This model not only overcomes the shortcomings of simple mathematical models in accurately describing the complex relationship between surface velocity and underwater topography, but also optimizes the model by combining levee topography data measured by drones with a limited number of underwater topography measurement points, thereby improving the precision and accuracy of underwater topography prediction and reconstruction.

[0024] Furthermore, the amphibious drone (second drone) operating scheme employed in this embodiment differs from traditional methods. In traditional measurement schemes, unmanned boats and amphibious drones rely on their own power to sail or fly under low currents, measuring underwater topographic point cloud data along a planned route. In this embodiment, the amphibious drone, after landing on the water surface, shuts off its flight or navigation power source and rapidly drifts downstream with the current, measuring the corresponding underwater topography while floating.

[0025] It should be pointed out that in this embodiment, the measurement of multi-source data is mainly completed by the cooperation of the first UAV and the second UAV. Different from the traditional UAV measurement method, in this embodiment, the first UAV and the second UAV need to be configured separately. Therefore, in this embodiment, before obtaining the visible light image of the embankment breach location by the first UAV and obtaining the underwater terrain elevation data along the way by the second UAV while floating along the water flow streamline with the help of the driving force of the natural water flow, it is also necessary to include: configuring a first UAV equipped with an image acquisition device and a second amphibious UAV equipped with a sonar device.

[0026] After the first UAV and the second UAV are configured, the first UAV and the second UAV can be used to collect data, which corresponds to steps S4-S6 of this embodiment. Figure 2 , which includes the following steps: a. Obtaining a visible light image of the dike breach location using a first UAV; b. Using a second UAV to float along the water flow line with the help of the natural water flow, obtain underwater terrain elevation data along the way; c. Processing the visible light image based on a pyramid layered optical flow method to obtain surface flow velocity data at the embankment breach location; d. performing image recognition and segmentation processing on the visible light image to obtain river channel cross-sectional width data at different locations near the embankment breach; e. Through coordinate transformation, the coordinates of the local underwater measuring points in the underwater terrain elevation data along the course are matched with the coordinates of the surface flow velocity data and the river section width data; f. Input the matched surface flow velocity data, the river channel cross-sectional width at different locations near the levee breach, and the underwater terrain elevation data along the way into the deployed deep learning model.

[0027] The configuring of the first UAV equipped with an image acquisition device means: flying the first UAV to a position where the image acquisition device can fully capture images of the embankment breach location and upstream and downstream process locations; The configuration of the second amphibious drone equipped with a sonar device means: obtaining the intervals between multiple second amphibious drones equipped with sonar devices, the number of local underwater sampling points and the sampling frequency of a single second drone equipped with a sonar device based on an upstream to downstream process; flying the second drone to the upstream of the dike breach and landing on the water surface at the said intervals.

[0028] During actual application, the interval setting can be set according to the actual situation of the dike breach. In this embodiment, it can be set to 20m. As for the water flow velocity threshold, it also needs to be set according to the actual situation of the dike breach. For the first threshold upstream, this embodiment is set to 5m / s, that is, when the water flow velocity is greater than 5m / s, the amphibious drone (second drone) cannot provide sufficient power to navigate along the given route. At this time, the flight and navigation power system of the water-air amphibious drone is turned off, and with the help of the natural water flow, it floats along the water flow line. In this way, the drone can float in the direction of the water flow, thereby collecting underwater terrain data along the way. For the second threshold downstream, this embodiment is set to 0-0.5m / s, that is, when the amphibious drone floats to the area where the downstream water flow velocity is lower than 0-0.5m / s, the amphibious drone returns and flies back to the predetermined starting position.

[0029] Here, the optional options for water-air amphibious drones are: equipped with a lightweight single-beam or multi-beam depth sounder to measure underwater terrain; equipped with a real-time dynamic positioning global navigation satellite system to record the drone's drift trajectory in real time; equipped with an inertial navigation module to compensate for the drone's attitude changes (roll, pitch) to ensure vertical incidence of the sound beam.

[0030] Therefore, in this embodiment, the acquisition of a visible light image of the dike breach location by the first drone refers to: The first UAV captures multiple frames of visible light images at a set shooting speed through an image acquisition device to form a multi-frame image sequence; The method of obtaining underwater terrain elevation data along the flow line by the second drone while the drone is floating along the flow line with the help of the natural water flow refers to: When the second UAV lands on the water surface at the interval, the upstream water velocity is obtained and determined to determine whether it reaches a first threshold. If so, the flight and navigation propulsion systems of the second UAV are shut down, and the underwater terrain elevation data along the flow line is obtained while the second UAV floats along the flow line with the help of the natural water flow. When the second UAV drifts downstream, the downstream water flow velocity is obtained and it is determined whether it is less than a second threshold. If it is less than a second threshold, the flight and navigation power system of the second UAV is turned on to fly to its starting position.

[0031] This embodiment addresses the challenges of traditional surface velocity measurement methods at night or in low-visibility conditions, particularly in emergency situations such as dike breaches, where effective monitoring cannot rely on natural light. To address this issue, this embodiment utilizes a third drone to drop luminous objects upstream of the breach, serving as reference points for optical flow velocity measurement. This allows for highly accurate surface velocity measurement in low-visibility environments. Specifically, this embodiment employs the following configuration: At night or in low-visibility conditions with visibility less than 100 meters, one or more third drones are deployed 30-50 meters upstream of the breach, simultaneously dropping luminous objects. This increased ambient light improves the accuracy of optical flow velocity measurements. When the breach width is less than 50 meters, two additional third drones are deployed, with one additional drone added for every 200 meters increase in breach width. When visibility is greater than 100 meters, the initial number of third drones remains.

[0032] Therefore, in this embodiment, see Figure 3 Before processing the visible light image based on the pyramid layered optical flow method, the following steps may also be included: Obtaining whether the visibility of the embankment breach location and upstream and downstream process locations is lower than a set threshold, and determining whether it is lower than the set threshold. If it is lower than the threshold, adding a third drone and using the third drone to drop a luminous tracer before the upstream landing position of the second drone; Adjust the brightness and contrast of multi-frame image sequences to enhance the water surface texture details and the bright and dark characteristics of luminous tracers in the image.

[0033] In actual application of this embodiment, the number of underwater terrain data samples of a single second UAV can be estimated by the following process. If the distance of the amphibious UAV drifting from upstream to downstream is about 40m, the average water flow velocity is 8m / s, and the sampling frequency of the water depth detector is 10Hz, the number of sampling points is 50 points.

[0034] After the configuration of the first UAV, the second UAV, and the third UAV is prepared as described above, each UAV can be made to work according to the configuration.

[0035] In this embodiment, the visible light images acquired by the first drone have non-uniform acquisition conditions, which makes the various image features in the visible light images unclear. Therefore, it is necessary to process the visible light images and adjust the brightness and contrast of the images to enhance the water surface texture details and the bright and dark features of natural or artificial tracers in the images. At the same time, in order to solve the problems of tracking large displacement motion and video frame loss, these images can be processed based on the pyramid layered optical flow method to further extract the surface flow velocity of the water flow.

[0036] Therefore, in this embodiment, processing the visible light image to obtain surface flow velocity data at the embankment breach location and river channel cross-sectional width data at different locations near the embankment breach location means: processing the visible light image based on the pyramid layered optical flow method to obtain surface flow velocity data at the embankment breach location, and performing image recognition and segmentation processing on the visible light image to obtain river channel cross-sectional width data at different locations near the embankment breach location.

[0037] Specifically, in this embodiment, in order to solve the problems of tracking large displacement motion and video frame loss, the pyramid-based layered optical flow method is used to process the visible light image to obtain surface flow velocity data at the embankment breach location, which means: At low-resolution layers, a pixel represents a larger area in the original image. Large physical displacements appear as smaller pixel displacements in low-resolution images. Therefore, the multi-frame image sequence is downsampled layer by layer (e.g., a 4-layer pyramid with a downsampling factor of 0.5) to generate a multi-scale image stack. In order to decompose the impossible large-scale search problem into a series of controllable local optimization problems and at the same time use the multi-scale characteristics of images to approximate physical reality, the optical flow field is initialized, the rough displacement between the top-level images is calculated, and the upper-level optical flow field is upsampled to the next-level resolution as the initial value. The displacement is then iteratively optimized based on the current-level image data. In order to remove abnormal data, the final optical flow field is smoothed and filtered to remove abnormal values; In order to convert the displacement on the image into actual displacement during coordinate transformation, the pixel displacement is converted into actual flow velocity by combining geographic correction parameters (such as satellite attitude and viewing angle), and used as the surface flow velocity data at the embankment breach location.

[0038] Since subsequent deep learning is required in conjunction with surface flow velocity and underwater topography data along the way, it is also necessary to obtain the cross-sectional width of the river channel near the dike breach. Therefore, in this embodiment, the cross-sectional width of the river channel at different positions near the dike breach location refers to: the cross-sectional width of the river channel at different positions between the dike breach location and its upstream and downstream processes.

[0039] In practice, to guide plugging operations, it's necessary to measure the underwater topography at the breach and at various locations 10-20 meters upstream and downstream along the flow direction. To constrain the subsequent deep learning model and improve its prediction accuracy, the river channel cross-sectional width is calculated at various locations 10-20 meters upstream and downstream along the flow direction. This determines the width of different levee sections and, in turn, estimates the average underwater flow velocity.

[0040] It should be noted that the deep learning model can be a variety of deep learning algorithms, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and U-Nets. Furthermore, an attention mechanism can be incorporated to further enhance the algorithm's performance. By focusing on details in important areas, this reduces computational redundancy and improves processing efficiency. In this embodiment, a U-Net model is integrated with an attention mechanism to select and construct a deep learning model. Therefore, in this embodiment, the deep learning model is an end-to-end multimodal input-integrated attention U-Net model.

[0041] See also Figure 4 In this embodiment, the input channels of the U-Net model with multimodal input fusion attention include the surface velocity field (H×W grid, normalized to 0-1), the embankment section width (width distribution along the river channel, converted to H×W grid, scalar value at the same position), and the sparse measurement point mask (the known measurement point position is the measured underwater elevation value, and the rest is an H×W matrix of 0). The output is a dense underwater elevation value (H×W grid), where H is the number of data points in the numerical direction of the velocity field image, and W is the number of points in the horizontal direction.

[0042] As for the network structure of the U-Net model with multimodal input fusion attention, it can include dual convolutional layers, an attention mechanism module and a feature splicing module, among which: the first layer of convolution extracts local basic features, the first layer of convolution combines higher-order features on the primary features, the attention mechanism module can suppress interference information and enhance small target detection, and the feature splicing module can reconstruct the precise water-land boundary for irregular flooded areas after splicing.

[0043] After obtaining the three inputs of the U-Net model with multimodal input fusion attention, the model input three-channel matrix size is H×W×3, and the initial feature C1 is extracted through a double convolution layer, whose size is H×W×64. It is then processed in two parallel branches. One branch C1 is input into the attention mechanism module to obtain a feature layer A1 of the same size, and the other branch C1 is subjected to maximum pooling to reduce the dimension to a feature layer M2 of size H / 2×W / 2×64. After double convolution, it obtains a feature layer C2 of the same size. The deep abstract feature A2 is further obtained through the attention module, and then restored to the original resolution through bilinear upsampling to obtain U2 of size H×W×64; Through the jump connection, A1 and U2 are channel-spliced ​​through the feature splicing module to obtain the feature word C3 of size H×W×128. C3 is subjected to double convolution to obtain the feature layer C4 of size H×W×64. Finally, through a 1*1 convolution kernel, the output matrix of underwater elevation data of size H×W×1 is obtained, which is used as the real underwater terrain elevation data.

[0044] It should be noted that during the training process of the multimodal input fusion attention U-Net model, in order to improve the prediction accuracy by measuring some underwater terrain data points, the root mean square error corresponding to the underwater terrain elevation data along the route measured by the second UAV is additionally added to its loss function. The loss function is expressed as: , in, is the root mean square error between the predicted and measured values ​​of all underwater terrain elevation data, is the root mean square error of the data corresponding to the underwater terrain position measured by the second UAV, is a weighting factor, which is greater than or equal to 1; the specific number is 10 points at an interval of 1 meter.

[0045] During the training process of the multimodal input fusion attention U-Net model, its optimization function is obtained through the Adam optimizer.

[0046] Here, both root mean square errors can be calculated by the following formula: , in, To predict underwater terrain elevation data, is the actual underwater terrain elevation data, is the number of underwater terrain elevation data points.

[0047] In actual application, the deep learning model trained in this embodiment can be deployed through dynamic link libraries and other methods.

[0048] In summary, this embodiment achieves high-precision and rapid reconstruction of the underwater terrain near the dike breach through the collaborative work of multiple drones, combined with oblique photography, optical flow method, deep learning and an innovative amphibious drone operation scheme. It overcomes the limitations of existing technologies in complex environments, realizes terrain measurement under high water flow conditions, and provides strong technical support for dike breach emergency response.

[0049] Example 2 Based on the first embodiment, this embodiment provides an intelligent reconstruction system for underwater terrain of dike breaches based on multi-UAV monitoring. The system structure diagram is shown in FIG. Figure 5 , the system of this embodiment may include: The first client is equipped with a deep learning module, which is used to collect existing underwater topography data and surface flow velocity data, establish a database, train a deep learning model based on the established database, and deploy the trained deep learning model through a dynamic link library; a first data acquisition module, configured to acquire a visible light image of the dike breach location via a first drone; The second data acquisition module is used to obtain underwater terrain elevation data along the flow line when the second UAV floats along the flow line with the help of the natural water flow; The second client is equipped with a data processing module for processing the visible light image to obtain surface flow velocity data at the dike breach location and river channel cross-sectional width data at different locations near the dike breach location, and through coordinate transformation, matches the coordinates of the local underwater measurement points in the underwater terrain elevation data along the process with the coordinates of the surface flow velocity data and the river channel cross-sectional width data; A first data transmission module is provided between the first client and the second client, and is used to input the matched surface flow velocity data, the river channel cross-sectional width at different locations near the embankment breach, and the underwater terrain elevation data along the river channel from the second client into the deep learning model deployed in the first client through the first data transmission module; A second data transmission module is provided between the first client and the third client, and is used to reconstruct the underwater topography at the dike breach using the deployed deep learning model in the first client and transmit the reconstructed topography to the third client via the second data transmission module; The third client is equipped with a display module for displaying the reconstructed underwater terrain at the embankment breach.

[0050] In this embodiment, the deployed deep learning model can be called by other clients (such as a third client) through various communication interfaces set by the first client, thereby obtaining the underwater terrain at the embankment breach reconstructed in the first client and displaying it in the relevant software of other clients (such as 3D software).

[0051] It can be seen from the description of this embodiment that the application scenario and implementation principle of this embodiment are consistent with those of the first embodiment, and therefore will not be repeated here.

[0052] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. An intelligent reconstruction method of underwater terrain at embankment breaches based on multi-UAV monitoring, characterized by: The steps include: Collect existing underwater topography data and surface velocity data and establish a database; Training a deep learning model based on the established database; Deploy the trained deep learning model through the dynamic link library; The first UAV is used to obtain visible light images of the location of the dike breach, and the second UAV is used to obtain underwater terrain elevation data along the flow line while floating along the flow line with the help of the natural water flow. Processing the visible light image to obtain surface flow velocity data at the dike breach location and river channel cross-sectional width data at different locations near the dike breach location; Through coordinate transformation, the coordinates of the local underwater measurement points in the underwater terrain elevation data along the route are matched with the coordinates of the surface flow velocity data and the river cross-sectional width data. The matched surface flow velocity data, the river cross-sectional width at different locations near the embankment breach, and the underwater terrain elevation data along the route are then input into the deployed deep learning model. The underwater terrain at the embankment breach is reconstructed using the deployed deep learning model.

2. The method for intelligent reconstruction of underwater terrain of dike breach based on multi-UAV monitoring according to claim 1 is characterized in that: Before obtaining a visible light image of the dike breach location by the first UAV and obtaining underwater terrain elevation data along the flow line by the second UAV while floating along the flow line with the help of the natural water flow, the method further includes: configuring the first UAV equipped with an image acquisition device and the second amphibious UAV equipped with a sonar device; The first drone equipped with an image acquisition device is: Flying the first UAV to a position where the image acquisition device can fully capture images of the levee breach location and upstream and downstream process locations; The second amphibious drone equipped with a sonar device is: Based on the upstream-to-downstream process, the intervals between multiple amphibious second UAVs equipped with sonar devices, and the number and frequency of local underwater sampling points of a single second UAV equipped with sonar devices are obtained; The second drone is flown upstream of the dike breach and landed on the water surface at the intervals.

3. The method for intelligent reconstruction of underwater terrain of dike breach based on multi-UAV monitoring according to claim 1 is characterized in that: The step of acquiring a visible light image of the dike breach location by the first drone refers to: The first UAV captures multiple frames of visible light images at a set shooting speed through an image acquisition device to form a multi-frame image sequence; The method of obtaining underwater terrain elevation data along the flow line by the second drone while the drone is floating along the flow line with the help of the natural water flow refers to: When the second UAV lands on the water surface at the interval, the upstream water velocity is obtained and determined to determine whether it reaches a first threshold. If so, the flight and navigation propulsion systems of the second UAV are shut down, and the underwater terrain elevation data along the flow line is obtained while the second UAV floats along the flow line with the help of the natural water flow. When the second UAV drifts downstream, the downstream water flow velocity is obtained and it is determined whether it is less than a second threshold. If it is less than a second threshold, the flight and navigation power system of the second UAV is turned on to fly to its starting position.

4. The method for intelligent reconstruction of underwater terrain of dike breach based on multi-UAV monitoring according to claim 1 is characterized in that: Processing the visible light image to obtain surface flow velocity data at the dike breach location and river channel cross-sectional width data at different locations near the dike breach location refers to: Processing the visible light image based on a pyramid hierarchical optical flow method to obtain surface flow velocity data at the levee breach location, and performing image recognition and segmentation processing on the visible light image to obtain river channel cross-sectional width data at different locations near the levee breach location; The cross-sectional width of the river channel at different positions near the dike breach refers to the cross-sectional width of the river channel at different positions between the dike breach and its upstream and downstream processes.

5. The method for intelligent reconstruction of underwater terrain of dike breach based on multi-UAV monitoring according to claim 4 is characterized in that: Before processing the visible light image based on the pyramid layered optical flow method, the method further includes: Obtain visibility at the levee breach location and upstream and downstream process locations, and determine whether it is below a set threshold. If so, deploy a third drone to drop a luminous tracer before the second drone's upstream landing location. Adjust the brightness and contrast of multi-frame image sequences to enhance the water surface texture details and the bright and dark characteristics of luminous tracers in the image.

6. The method for intelligent reconstruction of underwater terrain of dike breach based on multi-UAV monitoring according to claim 4 is characterized in that: Processing the visible light image based on the pyramid layered optical flow method to obtain surface flow velocity data at the embankment breach location refers to: Downsample the multi-frame image sequence layer by layer to generate a multi-scale image stack; Initialize the optical flow field, calculate the rough displacement between the top-level images, and upsample the upper-level optical flow field to the next-level resolution as the initial value. Combined with the current-level image data, iteratively optimize the displacement. Smooth the final optical flow field and remove outliers; Combined with geographic correction parameters, the pixel displacement is converted into actual flow velocity and used as the surface flow velocity data at the levee breach location.

7. The method for intelligent reconstruction of underwater terrain of dike breach based on multi-UAV monitoring according to claim 1 is characterized in that: The deep learning model is a U-Net model that integrates multimodal input and attention; The multimodal input fusion attention U-Net model includes a dual convolutional layer, an attention mechanism module and a feature splicing module, wherein: After obtaining the three inputs of the U-Net model with multimodal input fusion attention, the model input three-channel matrix size is H×W×3, and the initial feature C1 is extracted through a double convolution layer, whose size is H×W×64. It is then processed in two parallel branches. One branch C1 is input into the attention mechanism module to obtain a feature layer A1 of the same size, and the other branch C1 is subjected to maximum pooling to reduce the dimension to a feature layer M2 of size H / 2×W / 2×64. After double convolution, it obtains a feature layer C2 of the same size. The deep abstract feature A2 is further obtained through the attention module, and then restored to the original resolution through bilinear upsampling to obtain U2 of size H×W×64; Through the jump connection, A1 and U2 are channel-spliced ​​through the feature splicing module to obtain the feature word C3 of size H×W×128. C3 is subjected to double convolution to obtain the feature layer C4 of size H×W×64. Finally, through a 1*1 convolution kernel, the output matrix of underwater elevation data of size H×W×1 is obtained, which is used as the real underwater terrain elevation data.

8. The method for intelligent reconstruction of underwater terrain of dike breach based on multi-UAV monitoring according to claim 7 is characterized in that: During the training process of the multimodal input fusion attention U-Net model, the root mean square error corresponding to the underwater terrain elevation data along the route measured by the second UAV is additionally added to its loss function. The loss function is expressed as: ， in, is the root mean square error between the predicted and measured values ​​of all underwater terrain elevation data, is the root mean square error of the data corresponding to the underwater terrain position measured by the second UAV, is the trade-off factor, greater than or equal to 1; During the training process of the multimodal input fusion attention U-Net model, its optimization function is obtained through the Adam optimizer.

9. The method for intelligent reconstruction of underwater terrain of dike breach based on multi-UAV monitoring according to claim 8 is characterized in that: The two root mean square errors are calculated by the following formula: ， in, To predict underwater terrain elevation data, is the actual underwater terrain elevation data, is the number of underwater terrain elevation data points.

10. A system for intelligent reconstruction of underwater terrain of dike breaches based on multi-UAV monitoring, applied to the method for intelligent reconstruction of underwater terrain of dike breaches based on multi-UAV monitoring as claimed in any one of claims 1 to 9, characterized in that: include: The first client is equipped with a deep learning module, which is used to collect existing underwater topography data and surface flow velocity data, establish a database, train a deep learning model based on the established database, and deploy the trained deep learning model through a dynamic link library; a first data acquisition module, configured to acquire a visible light image of the dike breach location via a first drone; The second data acquisition module is used to obtain underwater terrain elevation data along the flow line when the second UAV floats along the flow line with the help of the natural water flow; The second client is equipped with a data processing module for processing the visible light image to obtain surface flow velocity data at the dike breach location and river channel cross-sectional width data at different locations near the dike breach location, and through coordinate transformation, matches the coordinates of the local underwater measurement points in the underwater terrain elevation data along the process with the coordinates of the surface flow velocity data and the river channel cross-sectional width data; A first data transmission module is provided between the first client and the second client, and is used to input the matched surface flow velocity data, the river channel cross-sectional width at different locations near the embankment breach, and the underwater terrain elevation data along the river channel from the second client into the deep learning model deployed in the first client through the first data transmission module; A second data transmission module is provided between the first client and the third client, and is used to reconstruct the underwater topography at the dike breach using the deployed deep learning model in the first client and transmit the reconstructed topography to the third client via the second data transmission module; The third client is equipped with a display module for displaying the reconstructed underwater terrain at the embankment breach.

Images