A data processing method and system for emergency flow measurement UAVs

Abstract

This invention relates to the field of data processing technology and discloses a data processing method and system for emergency flow measurement UAVs. The method involves collecting multimodal data using the UAV's lidar, visible light camera, thermal imager, and positioning system. The collected multimodal data is preprocessed, and the preprocessed data is aligned in both time and space, with an attention mechanism introduced to obtain fused data. This fused data is then input into a U-Net model based on the Grey Wolf optimization algorithm for feature extraction. A particle image velocimetry algorithm is used to track the displacement of water surface feature points, and terrain errors are corrected using point cloud data. The corrected terrain errors and water surface feature point displacement data are combined to calculate the water surface velocity distribution and cross-sectional flow rate. Based on the velocity distribution and cross-sectional flow rate results, a three-dimensional dynamic water flow simulation scene is constructed. Combined with the UAV flight trajectory data, a spatiotemporally continuous hydrological monitoring map is generated, and a visualization report is produced. This invention improves data processing efficiency and reduces reliance on manual operation.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a data processing method and system for emergency flow measurement drones. Background Technology

[0002] Traditional river flow monitoring technologies, such as wading flow measurement, cableway flow measurement, and mobile ADCP, have many limitations when facing complex terrain, severe weather, and sudden disasters. For example, wading flow measurement poses challenges to personnel safety during flood season, in high-velocity areas, and in dangerous sections of the river. Cableway systems are costly to build, complex to maintain, and lack flexibility. Mobile ADCP requires highly skilled vessels and operators and is difficult to implement in shallow water or areas with many obstacles.

[0003] With the rapid development of UAV technology, the application of UAVs in hydrological monitoring is gradually increasing. UAVs have advantages such as maneuverability, ease of operation, minimal terrain limitations, and high safety, enabling them to quickly reach monitoring areas for data collection. However, UAVs generate a large amount of data during flow measurement. How to efficiently and accurately process this data to obtain reliable flow information has become a key research focus and challenge. Existing UAV flow measurement data processing methods suffer from low data processing efficiency, low accuracy, and poor adaptability to complex environments, failing to meet the needs for rapid and accurate data processing in emergency flow measurement scenarios. Summary of the Invention

[0004] The purpose of this invention is to solve the above problems by designing an emergency flow measurement UAV data processing method and system.

[0005] The first aspect of this invention provides a data processing method for an emergency flow measurement drone, the method comprising the following steps:

[0006] Multimodal data is collected using the drone's lidar, visible light camera, thermal imager, and positioning system. This multimodal data includes point cloud data, image data, and flight trajectory data.

[0007] The collected multimodal data is preprocessed, the preprocessed data is aligned in terms of spatiotemporal dimensions, and an attention mechanism is introduced to obtain fused data.

[0008] The fused data is input into the U-Net model based on the Grey Wolf optimization algorithm for feature extraction. The displacement of water surface feature points is tracked by the particle image velocimetry algorithm, and the terrain error is corrected by combining point cloud data.

[0009] The water surface velocity distribution and cross-sectional flow rate are calculated by combining the corrected topographic error and the displacement data of water surface feature points.

[0010] Based on the velocity distribution and cross-sectional flow results, a three-dimensional dynamic water flow simulation scenario is constructed. Combined with UAV flight trajectory data, a spatiotemporally continuous hydrological monitoring map is generated, and a visualization report is produced.

[0011] Optionally, in a first implementation of the first aspect of the present invention, the preprocessing of the acquired multimodal data includes:

[0012] Randomly select a point in the point cloud data of the multimodal data, count the number of neighboring points within a 0.5-meter radius around it, and calculate the mean and standard deviation of the distance between the selected point and its neighboring points.

[0013] If the deviation of the selected point from the mean exceeds 3 times the standard deviation, it is judged as a noise point and removed. Traverse all point cloud data to complete point cloud denoising.

[0014] The distribution of image pixel gray values ​​in multimodal data is statistically analyzed, the cumulative distribution function is calculated, and the original gray values ​​of the image are mapped to a new gray range to enhance image contrast.

[0015] The image is traversed using a 3×3 pixel window, and the center pixel value is replaced with the median value of the pixels within the window to remove salt-and-pepper noise.

[0016] Optionally, in a second implementation of the first aspect of the present invention, the preprocessing of the collected multimodal data, the spatiotemporal alignment of the preprocessed data, and the introduction of an attention mechanism to obtain fused data include:

[0017] Extract the timestamps from the preprocessed point cloud, image, and flight trajectory data, and map them to the same time series using linear interpolation, based on the timestamps of the UAV positioning system.

[0018] Using the three-dimensional coordinates of the lidar point cloud data as a reference, the image is transformed into the point cloud coordinate system through camera intrinsic and extrinsic parameters for spatial position matching;

[0019] Features are extracted from the aligned point cloud, image, and flight trajectory data respectively. The importance weight of each feature is calculated through an attention mechanism. The features are then weighted and superimposed according to their weights to obtain the fused data.

[0020] Optionally, in a third implementation of the first aspect of the present invention, the step of inputting the fused data into the U-Net model based on the Grey Wolf optimization algorithm includes:

[0021] Several sets of U-Net model parameters are randomly generated as individual gray wolves. The distance between each individual gray wolf and the optimal solution of the objective function is calculated, and the parameters are adjusted to move closer to the optimal solution.

[0022] By updating the individual gray wolf positions, retaining better parameter combinations, and iteratively optimizing until the parameters are stable, the optimized U-Net model is obtained.

[0023] The fused data is input into the optimized U-Net model, where the encoder extracts low-dimensional features, the decoder restores the feature details, and the output is the water surface features.

[0024] Optionally, in a fourth implementation of the first aspect of the present invention, the step of tracking the displacement of water surface feature points using a particle image velocimetry algorithm and correcting terrain errors using point cloud data includes:

[0025] An adaptive thresholding method is used to mark water surface features in images and record the initial position. In consecutive frames of images, the similarity between feature points and neighboring pixels is calculated using a normalized cross-correlation algorithm to determine the position of the same feature point in different frames. The position changes of feature points in time series images are tracked and displacement data is recorded.

[0026] Riverbank and riverbed points are extracted from the preprocessed point cloud data to construct an initial terrain model. The lidar point cloud data is compared with historical terrain data, and areas with a deviation of more than 0.3 meters are marked as terrain error areas. The coordinates of the error areas are corrected by using the nearest neighbor interpolation method with the surrounding error-free point cloud data to obtain the corrected terrain data.

[0027] Optionally, in a fifth implementation of the first aspect of the present invention, the step of calculating the water surface velocity distribution and cross-sectional flow rate by combining the corrected terrain error and the displacement data of water surface feature points includes:

[0028] Based on the data acquisition frequency of the UAV, the time interval corresponding to the displacement data of water surface feature points is determined. The flow velocity of a single feature point is obtained by dividing the displacement data of water surface feature points by the time interval.

[0029] The inverse distance weighted interpolation method is used to obtain the velocity distribution of the entire water surface area based on the velocity at a single point and its spatial location.

[0030] Based on the corrected topographic data, the cross section is divided into several equal-width sub-sections. The product of the water depth and width of each sub-section is calculated to obtain the sub-section area.

[0031] Multiply the average flow velocity within a sub-section by the sub-section area to obtain the sub-section flow rate. Add up the flow rates of all sub-sections to obtain the total flow rate of the section.

[0032] Optionally, in the sixth implementation of the first aspect of the present invention, the step of constructing a three-dimensional dynamic water flow simulation scenario based on the flow velocity distribution and cross-sectional flow results, generating a spatiotemporally continuous hydrological monitoring map by combining UAV flight trajectory data, and generating a visualization report includes:

[0033] Using UAV flight trajectory data as the spatial path, the Kriging interpolation method is used to interpolate discrete cross-sectional flow and velocity data into a continuous spatial distribution.

[0034] Spatial distribution data at different times are arranged in chronological order to form a spatiotemporally continuous monitoring map. The three-dimensional dynamic water flow simulation scene, spatiotemporal monitoring map, and cross-sectional flow data are then summarized to generate a visualization report.

[0035] A second aspect of the present invention provides an emergency flow measurement UAV data processing system, the system comprising:

[0036] The data acquisition module is used to acquire multimodal data through the UAV's lidar, visible light camera, thermal imager and positioning system. The multimodal data includes point cloud data, image data and flight trajectory data.

[0037] The alignment module is used to preprocess the collected multimodal data, align the preprocessed data in terms of spatiotemporal dimensions, and introduce an attention mechanism to obtain fused data.

[0038] The extraction module is used to input the fused data into the U-Net model based on the Grey Wolf optimization algorithm, perform feature extraction, track the displacement of water surface feature points through the particle image velocimetry algorithm, and correct terrain errors by combining point cloud data;

[0039] The calculation module is used to calculate the water surface velocity distribution and cross-sectional flow rate by combining the corrected terrain error and the displacement data of water surface feature points;

[0040] The generation module is used to construct a three-dimensional dynamic water flow simulation scene based on the flow velocity distribution and cross-sectional flow results, combine UAV flight trajectory data to generate a spatiotemporally continuous hydrological monitoring map, and generate a visualization report.

[0041] A third aspect of the present invention provides an emergency flow measurement drone data processing device, the emergency flow measurement drone data processing device including a memory and at least one processor, the memory storing instructions; the at least one processor calling the instructions in the memory to cause the emergency flow measurement drone data processing device to perform the various steps of the emergency flow measurement drone data processing method as described in any of the preceding claims.

[0042] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of the emergency flow measurement UAV data processing method as described in any of the preceding claims.

[0043] The technical solution provided by this invention involves collecting multimodal data using a drone's lidar, visible light camera, thermal imager, and positioning system. The collected multimodal data is preprocessed, and the preprocessed data is aligned in both spatiotemporal dimensions. An attention mechanism is then introduced to obtain fused data. This fused data is input into a U-Net model based on the Grey Wolf optimization algorithm for feature extraction. A particle image velocimetry algorithm is used to track the displacement of water surface feature points, and terrain errors are corrected using point cloud data. The corrected terrain errors and water surface feature point displacement data are combined to calculate the water surface velocity distribution and cross-sectional flow rate. Based on the velocity distribution and cross-sectional flow rate results, a three-dimensional dynamic water flow simulation scene is constructed, which is then combined with the drone's flight... The invention generates a spatiotemporally continuous hydrological monitoring map from trajectory data and produces a visual report. Compared with traditional wading flow measurement, this invention uses a drone equipped with multiple sensors to remotely collect data, eliminating the need for personnel to enter dangerous areas such as flood seasons and high-flow-velocity zones, thus avoiding the safety risks of personnel wading into water. It adopts a multimodal data synchronous acquisition mechanism to improve data processing efficiency and meet the timeliness requirements of emergency flow measurement. Data preprocessing improves the accuracy of flow monitoring and enhances data reliability. The automated data processing process reduces reliance on manual operation and reduces the result deviation caused by differences in operator experience. Furthermore, the visual report can be directly used for decision-making reference, reducing the labor costs of subsequent secondary data processing. Attached Figure Description

[0044] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0045] Figure 1 A flowchart of the emergency flow measurement UAV data processing method provided in an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of the structure of the emergency flow measurement UAV data processing system provided in an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of the structure of the emergency flow measurement UAV data processing equipment provided in an embodiment of the present invention. Detailed Implementation

[0048] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0049] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 A flowchart of the emergency flow measurement UAV data processing method provided in this embodiment of the invention is shown. The method specifically includes the following steps:

[0050] Step 101: Collect multimodal data using the UAV's lidar, visible light camera, thermal imager, and positioning system. The multimodal data includes point cloud data, image data, and flight trajectory data.

[0051] In this embodiment, the UAV and its onboard LiDAR, visible light camera, thermal imager, and positioning system (such as GPS / BeiDou combined positioning) are activated. The LiDAR scanning frequency, visible light camera shooting resolution, thermal imager temperature measurement range, and positioning system sampling frequency are set. LiDAR point cloud data acquisition: The LiDAR emits a laser beam towards the river channel and surrounding area, receives the reflected beam, and records the beam propagation time. Based on the propagation time, the distance between the measurement point and the UAV is calculated. Combined with the real-time position and attitude of the UAV provided by the positioning system (such as longitude, latitude, altitude, and heading angle), a three-dimensional point cloud data package is generated. The system continuously collects 3D coordinates of the measurement points until the area to be measured is covered; a visible light camera captures images of the river surface and shoreline at a set frame rate and records the timestamp of the capture time; a thermal imager simultaneously captures images of the water surface heat distribution, capturing the characteristics of water surface temperature differences, and also records the timestamp to ensure that the image data is correlated with time; the positioning system receives satellite signals in real time and outputs the UAV's 3D coordinates such as longitude, latitude, altitude, and flight attitude such as pitch angle, roll angle, and heading angle every second to form flight trajectory data, which is synchronously correlated with the timestamps of the lidar and image data to achieve data time stamping.

[0052] Step 102: Preprocess the collected multimodal data, align the preprocessed data in terms of spatiotemporal dimensions, and introduce an attention mechanism to obtain fused data;

[0053] In this embodiment, a point is randomly selected from the point cloud data in the multimodal data, and the number of neighboring points within a 0.5-meter radius around it is counted. The mean and standard deviation of the distance between the selected point and its neighboring points are calculated. If the deviation of the selected point from the mean exceeds 3 times the standard deviation, it is judged as a noise point and removed. All point cloud data are traversed to complete point cloud denoising. The distribution of image pixel gray values ​​in the multimodal data is statistically analyzed, and the cumulative distribution function is calculated to map the original gray values ​​of the image to a new gray range to enhance image contrast. The image is traversed with a 3×3 pixel window, and the median value of the pixels in the window is used to replace the center pixel value to remove salt-and-pepper noise.

[0054] In this embodiment, timestamp information is extracted from the preprocessed point cloud data, image data, and flight trajectory data. The timestamp of the point cloud data corresponds to the LiDAR scanning time, the timestamp of the image data corresponds to the camera shooting time, and the timestamp of the flight trajectory data corresponds to the sampling time of the positioning system. Since the sampling frequencies of different devices differ (e.g., the LiDAR scanning frequency is higher than the positioning system sampling frequency), these timestamps often cannot be fully synchronized. In this case, the timestamp of the UAV positioning system is used as the reference time axis. For non-reference timestamp data from other devices, the corresponding values ​​on the reference time axis are calculated using linear interpolation. For example, if two timestamps of a certain segment of point cloud data correspond to t1 and t3 on the reference time axis, and there is a reference time point t2 between t1 and t3, then the point cloud data at time t2 is calculated according to the time interval ratio based on the point cloud data at times t1 and t3, and finally all data are mapped to the same time series.

[0055] Using the 3D coordinates of LiDAR point cloud data as a spatial reference, the original coordinate characteristics of the image data are first clarified. Initially, the image data records pixel positions in the camera's own coordinate system. This pixel coordinates need to be converted to 3D coordinates in the camera coordinate system using camera intrinsic parameters, including focal length, principal point coordinates, and distortion coefficients, to eliminate positional deviations caused by camera optical characteristics. Then, combined with camera extrinsic parameters—the relative position and attitude parameters between the camera and the UAV, including translation vectors and rotation matrices—the coordinates in the camera coordinate system are further transformed to the UAV's body coordinate system. Finally, based on the transformation relationship between the UAV's body coordinate system and the geodetic coordinate system provided by the UAV's positioning system, the coordinates of the image data are mapped to the 3D coordinate system of the LiDAR point cloud data. Through this process, each pixel in the image can correspond to its 3D spatial position in the point cloud data, achieving precise matching between the image data and the point cloud data in physical space and avoiding spatial misalignment caused by differences in equipment installation location.

[0056] After completing spatiotemporal alignment, topographic features are extracted from point cloud data, including riverbed elevation changes, shoreline contours, and the boundary morphology between water and land. These features directly reflect the topographic foundation of the river channel. Water surface features are extracted from image data, such as water surface ripple texture and floating object distribution in visible light images, and water surface temperature gradient and cold / warm water zone boundaries in thermal imaging images. These features can help determine the state of water flow. Path features are extracted from flight trajectory data, including changes in UAV flight altitude, the relative direction of the flight path to the river channel, and the flight dwell time in key monitoring areas. These features reflect the key areas of data collection. Then, an attention mechanism is used to analyze the contribution of each feature to hydrological monitoring: for example, in the flow velocity calculation scenario, the importance of water surface ripple texture features is higher than that of flight path features, and the attention mechanism will assign them a higher weight; in the topographic modeling scenario, the weight of riverbed elevation change features will be increased. Finally, all features are weighted and superimposed according to the calculated weights, integrating topographic features, water surface features, and path features into unified fused data, which retains key information while avoiding data redundancy.

[0057] Step 103: Input the fused data into the U-Net model based on the Grey Wolf optimization algorithm, extract features, track the displacement of water surface feature points using the particle image velocimetry algorithm, and correct terrain errors by combining point cloud data;

[0058] In this embodiment, the key parameter ranges of the U-Net model are determined, including convolutional kernel size (e.g., 3×3 to 7×7), number of convolutional layers (e.g., 4 to 8), learning rate (e.g., 0.001 to 0.01), and activation function type (e.g., ReLU, Sigmoid). Within this range, 20-50 sets of parameter combinations are randomly generated, with each set representing a single gray wolf. The objective function is set as the feature extraction accuracy of the model on the validation set, i.e., the matching degree between the extracted water surface features and manually labeled features. The parameter combination corresponding to the historical best feature extraction result is set as the initial optimal solution. The difference between the objective function value and the optimal solution for each gray wolf is calculated. For individuals that are close to the optimal solution, parameters are fine-tuned proportionally, such as adjusting the learning rate towards the optimal learning rate. For individuals that are far from the optimal solution, the parameter adjustment range is appropriately increased, such as changing the convolutional kernel size, so that all parameter combinations gradually converge towards the optimal solution.

[0059] The hunting mechanism based on the gray wolf optimization algorithm updates the individual gray wolf positions. Specifically, each individual is sorted according to its distance from the optimal solution, and the top 30% of better parameter combinations (such as those with feature extraction accuracy higher than the current average) are retained. New parameter combinations are generated based on these combinations. By cross-referencing the parameters of different better combinations (e.g., combining the convolution kernel size of combination A with the learning rate of combination B), and introducing small-probability random mutations (e.g., randomly adjusting the value of a certain parameter), the model avoids getting trapped in local optima. After each update, the objective function value of all parameter combinations is recalculated. If the accuracy of the new combination is higher than the historical best value, it is set as the new optimal solution. The above update, screening, and evaluation process is repeated until the change in the optimal solution in 10 consecutive iterations is less than a set threshold (e.g., accuracy fluctuation is less than 1%). At this point, the parameters tend to stabilize, and the corresponding parameter combination is the optimized U-Net model parameter.

[0060] The fused data, which has undergone spatiotemporal alignment and weight fusion, contains multi-dimensional information such as point cloud terrain features and image water surface features. It is then input into the optimized U-Net model. The encoder part of the model uses optimized convolution kernels to extract key features from the data, such as the edge contours of water whirlpools and the direction of ripples, through multiple convolution operations. It also compresses the spatial dimension through pooling operations to obtain a low-dimensional feature matrix containing core information. This step can filter out redundant background noise, such as the shadows of trees on the shore, and focus on water-related features. The decoder part gradually restores the spatial resolution of the features through upsampling operations. At the same time, it fuses the feature information of the corresponding level of the encoder, such as combining the low-dimensional core features with the detailed texture of the original image, and finally generates a feature map that matches the size of the input data. This feature map accurately marks the position and shape of water surface features, such as the outline of floating objects and the velocity gradient region of water flow.

[0061] In this embodiment, an adaptive thresholding method is used to mark water surface features in the image and record the initial position. In consecutive frame images, the similarity between feature points and neighboring pixels is calculated using a normalized cross-correlation algorithm to determine the position of the same feature point in different frames. The position changes of feature points in the time series image are tracked and displacement data is recorded. Riverbank and riverbed points are extracted from the preprocessed point cloud data to construct an initial terrain model. The lidar point cloud data is compared with historical terrain data, and areas with a deviation of more than 0.3 meters are marked as terrain error areas. The coordinates of the error areas are corrected using the nearest neighbor interpolation method with the surrounding error-free point cloud data to obtain the corrected terrain data.

[0062] Step 104: Calculate the water surface velocity distribution and cross-sectional flow rate by combining the corrected topographic error and the displacement data of water surface feature points;

[0063] In this embodiment, the time interval corresponding to the displacement data of water surface feature points is determined according to the data acquisition frequency of the UAV. The displacement data of water surface feature points is divided by the time interval to obtain the flow velocity of a single feature point. The inverse distance weighted interpolation method is used to obtain the flow velocity distribution of the entire water surface area based on the flow velocity of a single point and its spatial location. According to the corrected topographic data, the cross section is divided into several equal-width sub-sections. The product of the water depth and width of each sub-section is calculated to obtain the sub-section area. The average flow velocity in the sub-section is multiplied by the sub-section area to obtain the sub-section flow rate. The total flow rate of the cross section is obtained by accumulating the flow rates of all sub-sections.

[0064] Step 105: Based on the velocity distribution and cross-sectional flow results, construct a three-dimensional dynamic water flow simulation scenario, combine UAV flight trajectory data to generate a spatiotemporally continuous hydrological monitoring map, and generate a visualization report.

[0065] In this embodiment, the UAV flight trajectory data is used as the spatial path reference to clarify the geographical coordinates of each sampling point on the trajectory and the discrete cross-sectional flow and velocity data associated with that location. When applying the Kriging interpolation method, the spatial correlation of these discrete data is first analyzed. By calculating the distance between different sampling points and the differences in their corresponding flow and velocity data, a semi-variogram model is established to quantify the spatial variation of the data. Subsequently, the area covered by the UAV flight trajectory is used as the interpolation range, and the area is divided into fine grid cells. For each grid cell, the weight relationship between it and the surrounding discrete sampling points is calculated according to the semi-variogram, and the flow and velocity values ​​of the grid cell are estimated by weighted averaging. In this way, the discrete data originally scattered across each cross-section is transformed into continuous spatial distribution data. Since the interpolation process references the spatial path of the UAV flight trajectory, it ensures that the continuous distribution results match the geographical boundaries and river direction of the actual monitoring area, avoiding unfounded spatial extrapolation errors.

[0066] After obtaining continuous spatial distribution data at different times, the data for each time moment is first marked with a corresponding timestamp, and then the data are arranged in chronological order of the timestamps. During the arrangement, the consistency of spatial coordinates is maintained, meaning that the flow and velocity data of the same geographical location at different times always correspond to the same spatial location in the map. The spatial distribution differences at different times are visually presented through color gradients, thus forming a spatiotemporally continuous monitoring map. This map can not only reflect the water flow status of the entire river channel at a certain moment, but also show the dynamic changes of water flow through the passage of time. Then, the three-dimensional dynamic water flow simulation scene, the spatiotemporal monitoring map, and the original cross-sectional flow data are summarized and integrated. During the integration process, the content is organized according to the logic of spatial distribution-temporal change-core data. Interactive tags are added to the three-dimensional scene, key time nodes are marked on the monitoring map, and bar charts or line charts are matched to the cross-sectional data. The final generated visualization report is presented in a combination of text and graphics, including both intuitive graphical displays and explanations of core data, making it easy for users to quickly grasp the spatiotemporal characteristics of river flow.

[0067] In this embodiment, as mentioned in step 103, feature extraction is performed using the U-Net model based on the Grey Wolf optimization algorithm. During this process, the U-Net model can identify and mark water surface features, such as the outline of floating objects and the gradient region of water flow velocity. These feature information are the basis for the subsequent use of the Particle Image Velocity (PIV) algorithm to track the displacement of water surface feature points. The feature map output by the U-Net model marks the position and shape of the water surface features. These positions are the points used by the PIV algorithm to track. That is, the relationship between them is that the water surface feature points extracted by the U-Net model provide initial position information for the PIV algorithm, so as to calculate the displacement of these feature points in different time frames.

[0068] Step 104 describes how to calculate the surface velocity distribution: First, the time interval for the UAV to collect data is determined. Then, the velocity of a single feature point is obtained by removing the position of the feature points on the surface using this time interval. After that, the inverse distance weighted interpolation method is used to estimate the velocity distribution of the entire surface area based on the velocity of each point and its spatial location. This step ensures that the approximate velocity can be estimated even in places where there is no direct measurement.

[0069] In step 105, the UAV flight trajectory is used as the spatial path, and the cross-sectional flow rate and velocity data are converted into a continuous spatial distribution using the Kriging interpolation method. Then, the data at different times are arranged in chronological order to form a spatiotemporally continuous monitoring map. To enhance understanding and practicality, the three-dimensional water flow dynamic simulation scene, spatiotemporal monitoring map and cross-sectional flow rate data are also summarized, and finally a visualization report is generated.

[0070] In this embodiment, the multi-sensor data carried by the UAV is fully utilized, combined with advanced data processing technology, to achieve efficient and accurate river flow monitoring, while minimizing human intervention and improving work efficiency and safety.

[0071] Please see Figure 2 A schematic diagram of the structure of the emergency flow measurement UAV data processing system provided in this embodiment of the invention. The system includes:

[0072] The data acquisition module is used to acquire multimodal data through the UAV's lidar, visible light camera, thermal imager and positioning system. The multimodal data includes point cloud data, image data and flight trajectory data.

[0073] The alignment module is used to preprocess the collected multimodal data, align the preprocessed data in terms of spatiotemporal dimensions, and introduce an attention mechanism to obtain fused data.

[0074] The extraction module is used to input the fused data into the U-Net model based on the Grey Wolf optimization algorithm, perform feature extraction, track the displacement of water surface feature points through the particle image velocimetry algorithm, and correct terrain errors by combining point cloud data;

[0075] The calculation module is used to calculate the water surface velocity distribution and cross-sectional flow rate by combining the corrected terrain error and the displacement data of water surface feature points;

[0076] The generation module is used to construct a three-dimensional dynamic water flow simulation scene based on the flow velocity distribution and cross-sectional flow results, combine UAV flight trajectory data to generate a spatiotemporally continuous hydrological monitoring map, and generate a visualization report.

[0077] Figure 3 This is a schematic diagram of the structure of an emergency flow measurement UAV data processing device 300 provided in an embodiment of the present invention. The emergency flow measurement UAV data processing device 300 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the emergency flow measurement UAV data processing device 300. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the emergency flow measurement UAV data processing device 300 to implement the method provided in the above embodiment.

[0078] The emergency flow measurement UAV data processing device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3 The structure of the emergency flow measurement UAV data processing equipment shown does not constitute a limitation on the computer equipment provided by the present invention. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0079] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the various steps of the emergency flow measurement UAV data processing method provided in the above embodiments.

[0080] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described equipment or apparatus / unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0081] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. An emergency flow measurement unmanned aerial vehicle data processing method, characterized in that, The method comprises the following steps: Collecting multi-modal data by laser radar, visible light camera, thermal imager and positioning system of the unmanned aerial vehicle, wherein the multi-modal data comprises point cloud data, image data and flight trajectory data; Preprocessing the collected multi-modal data, aligning the preprocessed data in time and space dimensions, and introducing an attention mechanism to obtain fusion data; Inputting the fusion data into a U-Net model based on a grey wolf optimization algorithm, extracting features, tracking the displacement of water surface feature points by a particle image velocimetry algorithm, and correcting terrain errors in combination with the point cloud data; Calculating water surface flow velocity distribution and cross-section flow rate in combination with the corrected terrain errors and the water surface feature point displacement data; Based on the flow velocity distribution and cross-section flow rate results, constructing a three-dimensional water flow dynamic simulation scene, generating a time and space continuous hydrological monitoring atlas in combination with the unmanned aerial vehicle flight trajectory data, and generating a visual report; The preprocessing of the collected multi-modal data comprises: Randomly selecting a point in the point cloud data in the multi-modal data, counting the number of neighboring points within a radius of 0.5 meters around the selected point, calculating the mean and standard deviation of the distance between the selected point and the neighboring points, and determining the selected point as a noise point and removing it if the deviation of the selected point from the mean exceeds 3 times the standard deviation, and completing point cloud denoising by traversing all point cloud data; Statistically analyzing the image pixel gray value distribution in the multi-modal data, calculating the cumulative distribution function, mapping the original image gray values to a new gray interval, and enhancing the image contrast; Traversing the image with a 3x3 pixel window, replacing the center pixel value with the median value of the pixels in the window, and removing salt and pepper noise; The preprocessing of the collected multi-modal data, the alignment of the preprocessed data in time and space dimensions, and the introduction of an attention mechanism to obtain fusion data comprise: Extracting the timestamps of the preprocessed point cloud, image and flight trajectory data, taking the timestamp of the unmanned aerial vehicle positioning system as the reference, and mapping to the same time sequence by linear interpolation; Taking the three-dimensional coordinates of the laser radar point cloud data as the reference, converting the image to the point cloud coordinate system through the camera intrinsic and extrinsic parameters, and matching the spatial positions; Extracting features from the aligned point cloud, image and flight trajectory data respectively, calculating the importance weights of the features through the attention mechanism, weighting and superimposing the features according to the weights, and obtaining the fusion data. The inputting of the fusion data into the U-Net model based on the grey wolf optimization algorithm comprises:

2. The unmanned aerial vehicle data processing method for emergency flow measurement according to claim 1, wherein, Randomly generating several groups of U-Net model parameters as grey wolf individuals, calculating the distance between each grey wolf individual and the optimal solution of the objective function, and adjusting the parameters to approach the optimal solution; Iteratively optimizing the parameters to be stable by updating the positions of the grey wolf individuals and retaining better parameter combinations, and obtaining the optimized U-Net model; Inputting the fusion data into the optimized U-Net model, extracting low-dimensional features by the encoder, restoring feature details by the decoder, and outputting water surface features. The tracking of the displacement of water surface feature points by the particle image velocimetry algorithm and the correction of terrain errors in combination with the point cloud data comprise: 3.The emergency flow measurement unmanned aerial vehicle data processing method of claim 1, wherein, ​ Adaptive threshold method is used to mark water surface features in the image, record the initial position, in the continuous frame image, the similarity of feature points and neighborhood pixels is calculated by normalized cross-correlation algorithm, the position of the same feature point in different frames is determined, the position change of feature points in time series image is tracked, and displacement data is recorded; The river bank and river bed points are extracted from the pre-processed point cloud data, the initial terrain model is constructed, the laser radar point cloud data and the historical terrain data are compared, the area with deviation exceeding 0.3 meters is marked as terrain error area, the nearest neighbor interpolation method is used to correct the coordinates of the error area with the surrounding error-free point cloud data, and the corrected terrain data is obtained. 4.The emergency flow measurement unmanned aerial vehicle data processing method of claim 1, wherein, The water surface flow velocity distribution and cross-section flow are calculated according to the corrected terrain error and water surface feature point displacement data, including: According to the unmanned aerial vehicle data acquisition frequency, the time interval corresponding to the water surface feature point displacement data is determined, the water surface feature point displacement data is divided by the time interval, and the flow velocity of a single feature point is obtained; The inverse distance weighted interpolation method is used to obtain the flow velocity distribution of the entire water surface area according to the single point flow velocity and spatial position; According to the corrected terrain data, the cross section is divided into several equal-width sub-sections, the product of the water depth and width of each sub-section is calculated, and the sub-section area is obtained; The average flow velocity in the sub-section is multiplied by the sub-section area to obtain the sub-section flow, and the total flow of all sub-sections is obtained.

5. The unmanned aerial vehicle data processing method for emergency flow measurement according to claim 1, wherein, Based on the flow velocity distribution and cross-section flow results, a three-dimensional water flow dynamic simulation scene is constructed, combined with the unmanned aerial vehicle flight trajectory data to generate a time and space continuous hydrological monitoring atlas, and a visual report is generated, including: Taking the unmanned aerial vehicle flight trajectory data as the spatial path, the discrete cross-section flow and flow velocity data are interpolated into continuous spatial distribution by using the Kriging interpolation method; The spatial distribution data of different time is arranged in time sequence to form a time and space continuous monitoring atlas, and the three-dimensional water flow dynamic simulation scene, time and space monitoring atlas and cross-section flow data are summarized to generate a visual report.

6. An emergency flow measurement drone data processing system, characterized by, The system comprises: The acquisition module is used for collecting multi-modal data by the laser radar, visible light camera, thermal imager and positioning system of the unmanned aerial vehicle, wherein the multi-modal data includes point cloud data, image data and flight trajectory data; The alignment module is used for preprocessing the collected multi-modal data, aligning the preprocessed data in time and space dimensions, and introducing an attention mechanism to obtain fusion data: a point in the point cloud data in the multi-modal data is randomly selected, the number of neighboring points within a radius of 0.5 meters around the selected point is counted, and the mean and standard deviation of the distance between the selected point and the neighboring points are calculated; if the deviation of the selected point from the mean exceeds 3 times the standard deviation, the selected point is determined as a noise point and is removed, and all point cloud data is traversed to complete point cloud denoising; the distribution of image pixel gray value in the multi-modal data is counted, the cumulative distribution function is calculated, the original gray value of the image is mapped to a new gray interval, and the contrast of the image is enhanced; a 3*3 pixel window is used to traverse the image, and the median value of the pixels in the window is used to replace the center pixel value to remove salt and pepper noise; the timestamps of the preprocessed point cloud, image and flight trajectory data are extracted, the timestamps of the unmanned aerial vehicle positioning system are used as the reference, and the timestamps are mapped to the same time sequence through linear interpolation; the three-dimensional coordinates of the laser radar point cloud data are used as the reference, the image is converted to the point cloud coordinate system through the camera intrinsic and extrinsic parameters, and the spatial position matching is performed; features are extracted from the aligned point cloud, image and flight trajectory data respectively, the importance weights of the features are calculated through the attention mechanism, the features are weighted and superimposed according to the weights, and the fusion data is obtained; The extraction module is used for inputting the fusion data into the U-Net model based on the gray wolf optimization algorithm, extracting features, and tracking the displacement of water feature points through the particle image velocimetry algorithm, and correcting the terrain error in combination with the point cloud data; The calculation module is used for calculating the water flow velocity distribution and cross-section flow based on the corrected terrain error and the water feature point displacement data; The generation module is used for constructing a three-dimensional water flow dynamic simulation scene based on the flow velocity distribution and cross-section flow results, generating a time and space continuous hydrological monitoring atlas in combination with the unmanned aerial vehicle flight trajectory data, and generating a visual report.

7. An emergency flow measurement unmanned aerial vehicle data processing device, characterized in that, The emergency flow measurement unmanned aerial vehicle data processing equipment includes a memory and at least one processor, and the memory stores instructions; the at least one processor calls the instructions in the memory, so that the emergency flow measurement unmanned aerial vehicle data processing equipment performs each step of the emergency flow measurement unmanned aerial vehicle data processing method in any one of claims 1-5.

8. A computer-readable storage medium having stored thereon instructions, the computer-readable storage medium comprising: The instructions are executed by the processor to implement each step of the emergency flow measurement unmanned aerial vehicle data processing method in any one of claims 1-5.

Images