Disaster risk heat map generation method based on deep attention mechanism

Abstract

The application discloses a kind of disaster risk thermodynamic diagram generation methods based on deep attention mechanism, comprising the following steps: S1, utilize the unmanned aerial vehicle of laser radar and multispectral camera are carried out and are scanned by preset flight route to landslide body, collect multiple original laser point cloud data and multiple multispectral image data of landslide body and transmit to edge computing node;S2, edge computing node first carries out point cloud sparsification processing to multiple original laser point cloud data by parameterized sparse module, then utilize deep attention mechanism to extract the texture feature of landslide body crack region, finally, the thermodynamic diagram of crack region is obtained by fusing point cloud geometric feature and texture feature;S3, the thermodynamic diagram of crack region obtained will be uploaded to geological disaster monitoring and early warning platform, and the platform automatically compares it with previous thermodynamic diagram.The application can realize the whole chain quick response of landslide crack from identification to early warning, with better timeliness, accuracy and applicability.

Description

Technical Field

[0001] This invention relates to the field of crack detection and monitoring technology, specifically to a method for generating disaster risk heat maps based on a deep attention mechanism. Background Technology

[0002] Currently, crack detection and monitoring technologies mainly rely on multi-source data fusion methods, such as combining lidar point clouds with multispectral imagery, to generate high-precision crack thermal maps. Meanwhile, existing patents employ traditional fusion algorithms to achieve crack identification by integrating point cloud data and image data.

[0003] Among them, single sensor technology relies solely on lidar or optical sensors, lacks depth estimation capabilities, and cannot fully reflect the three-dimensional characteristics of cracks. Moreover, these technologies have been widely used in civil engineering, geological monitoring, and infrastructure maintenance. The basic principle is to generate heat maps through data acquisition, preprocessing, feature extraction, and fusion to visualize the distribution and density of cracks.

[0004] However, the current measures have at least the following problems: First, high processing latency. Traditional multi-source fusion models have high algorithm complexity, resulting in data processing latency exceeding 30 minutes, which cannot meet the needs of real-time monitoring. Second, high consumption of computing resources. Point cloud data processing accounts for more than 85% of the total computing time. Due to its large data volume, it requires high-performance computing equipment, which limits its deployment on edge devices. Third, insufficient accuracy. Single-sensor methods lack multi-source complementarity, resulting in large errors in crack depth detection and an inability to effectively handle complex environments. Fourth, limited applicability. Existing technologies rely on strong network environments to transmit raw data and cannot operate stably in weak 5G network environments, thus limiting applications in field or remote scenarios. Summary of the Invention

[0005] To address the aforementioned issues, the present invention aims to propose a disaster risk heat map generation method based on a deep attention mechanism. This method utilizes a crack monitoring mode that combines drone data collection, lightweight edge computing node processing, and cloud-based fusion early warning to achieve a rapid response across the entire chain from landslide crack identification to early warning.

[0006] This was achieved through the following technical solutions: A method for generating disaster risk heatmaps based on a deep attention mechanism is proposed. The method includes: S1, using a drone equipped with a lidar and a multispectral camera to perform aerial scanning of a landslide along a preset flight path, and employing periodic and emergency data acquisition strategies to collect multiple raw lidar point cloud data and multiple multispectral image data of the landslide. The collected data is then transmitted via an onboard 5G module to an edge computing node deployed at a monitoring station near the landslide. S2, the edge computing node first performs point cloud sparsification processing on the multiple raw lidar point cloud data using a parameterized sparsity module, and then uses a lightweight Transformer... The ER network analyzes multiple multispectral image datasets and uses a deep attention mechanism to extract texture features of landslide crack areas. Finally, a depth weight allocator fuses the sparsed point cloud geometric features with the texture features of the landslide crack areas to obtain a heat map of the crack areas. S3. The obtained heat map of the crack areas is uploaded to a geological disaster monitoring and early warning platform. The platform automatically compares it with previous heat maps and calculates the expansion rate and depth change trend of the crack areas. If the platform determines that the crack area is expanding rapidly and the expansion rate reaches a preset threshold, an orange alert is triggered and an early warning message is sent. This invention enables a rapid response across the entire chain from landslide crack identification to early warning, offering better timeliness, accuracy, and applicability.

[0007] Preferably, in step S1, the drone is a DJI M300 RTK, the lidar is a Livox Mid-360, and the multispectral camera is a Sony ILX-F217. By configuring the models of the acquisition devices, high-precision point clouds and hyperspectral texture images can be generated.

[0008] Preferably, in step S1, in the periodic acquisition strategy, the multiple raw laser point cloud data collected by the UAV each time will generate a set of LAS point cloud files. The multiple multispectral image data collected include multiple TIFF multispectral images, and each multispectral image is labeled with GPS coordinates. The periodic acquisition strategy is used to capture the expansion rate and depth change trend of the crack region.

[0009] Preferably, in step S1, the emergency data acquisition strategy triggering conditions include: the real-time data transmitted by the rainfall sensor at the monitoring station exceeds a set threshold and the edge computing node detects an abnormal point cloud curvature; the response process is as follows: the monitoring station sends an emergency command to the UAV base station through the LoRa gateway, and after receiving the emergency command, the UAV uses a spiral flight path and a focused scanning mode to acquire data. The emergency data acquisition strategy ensures timely data acquisition.

[0010] Preferably, in step S1, the edge computing node sends reception progress frames to the drone in real time, displaying the number of packets received and the packet loss rate. Sending reception progress frames in real time ensures data integrity.

[0011] Preferably, in step S2, when performing point cloud sparsification, the landslide crack region is first identified based on the point cloud curvature. Then, different sampling methods are applied to the crack region, the stable landslide region, and the non-landslide background to perform dynamic downsampling. Dynamic downsampling is used to retain the point cloud density in the crack region and reduce the point cloud density in the non-crack region.

[0012] Preferably, in step S2, the point cloud curvature calculation method is as follows: first search any point cloud P j Construct a local point set S from the 20 nearest neighbors of (x, y, z). j ={P1,P2,...,P20}, then calculate the mean of the local point set of the current point cloud. Construct the corresponding covariance matrix C j And for the covariance matrix C j Perform eigenvalue decomposition to obtain the corresponding eigenvalues ​​λ. j 1. λ j 2 and λ j 3. Then, based on each characteristic value, apply the formula... Calculate the current point cloud curvature K j The crack region is identified by calculating the curvature of each point cloud element to accurately determine its location.

[0013] Preferably, in step S2, when extracting the texture features of the crack area of ​​the landslide body, each band B is first... i Normalization is performed, and the Normalized Difference Vegetation Index (NDVI) is calculated before feature extraction: first, patch embedding is performed, t patch =x patch ×W proj +W pos Then calculate the local attention Attn local Its formula is: Finally, the output is a dimensional texture feature map; where B i X represents the original pixel value of the i-th spectral band. patch W is a local image patch data of a multispectral image. proj W is the linear projection weight matrix. pos T is the position encoding vector. path QK is the embedded image patch feature vector. T For dot product attention, M is the window mask, and V is the value matrix. By utilizing a deep attention mechanism to extract texture features from the crack region of the landslide body, background interference such as vegetation and shadows can be effectively eliminated.

[0014] Preferably, in step S2, when performing feature fusion, the confidence level Conf of the point cloud geometric features of each crack region is first calculated, and the calculation formula is as follows: Then, after calculating resource allocation based on the confidence level (Conf), feature fusion is performed. The fusion formula is as follows: Where α is the balance coefficient, K is the average curvature of the point cloud, and W... attn For the attention weights of texture features, F fusion For fusion features, β is the weight of the point cloud geometric features, and F pc (x,y) represents the geometric features of the point cloud, F ms (x,y) represents the texture features. By using a depth weighted allocator for feature fusion, a high-precision heatmap of the crack region can be generated, clearly showing the location, length, density, and depth distribution of the cracks.

[0015] Preferably, in step S2, when generating the heat map, the formula is first used... The crack depth of each crack region is calculated, and then the Jet color mapping mechanism is used to map any crack depth in different ranges to a corresponding color, where k cal These are the calibration coefficients used in the field. By employing the Jet color mapping mechanism to generate heat maps, it is possible to easily monitor cracked areas.

[0016] The beneficial effects of this invention compared to the prior art are: The technical solution of this invention, by adopting a crack monitoring mode that includes UAV data collection, lightweight edge computing node processing, and cloud-based fusion early warning, can achieve rapid response across the entire chain from landslide crack identification to early warning, improve processing timeliness, meet real-time monitoring requirements, and significantly improve crack depth detection accuracy. At the same time, it can reduce computing resource consumption, lower hardware costs, and the lightweight design supports portable deployment, making it suitable for field and remote environments, and has better timeliness, accuracy, and applicability. Attached Figure Description

[0017] Figure 1 This is a flowchart of a disaster risk heatmap generation method based on a deep attention mechanism. Detailed Implementation

[0018] The following will refer to the appendices in the embodiments of the present invention. Figure 1 The technical solutions in the embodiments of the present invention will be described in detail below.

[0019] like Figure 1The diagram shows a flowchart of a disaster risk heat map generation method based on a deep attention mechanism. First, a drone equipped with a lidar and multispectral camera performs aerial scanning of the landslide body along a preset flight path, collecting multiple raw lidar point cloud data and multiple multispectral image data of the landslide body and transmitting them to an edge computing node. Then, the edge computing node first performs point cloud sparsification processing on the multiple raw lidar point cloud data using a parametric sparsity module, and then uses a deep attention mechanism to extract texture features of the crack areas in the landslide body. Finally, the point cloud geometric features and texture features are fused to obtain a heat map of the crack areas. The obtained heat map of the crack areas is then uploaded to a geological disaster monitoring and early warning platform, which automatically compares it with previous heat maps, thus enabling a rapid response across the entire chain from landslide crack identification to early warning, with better timeliness, accuracy, and applicability.

[0020] The method specifically includes the following: S1. Using a drone equipped with lidar and a multispectral camera, aerial scanning of the landslide body is carried out along a preset flight path. A regular and emergency data acquisition strategy is adopted to collect multiple raw lidar point cloud data and multiple multispectral image data of the landslide body. The collected data are transmitted to the edge computing node deployed in the monitoring station near the landslide body through the airborne 5G module. The edge computing node sends reception progress frames to the drone in real time to display the number of packets received and the loss rate. The main flight path is parallel to the direction of the landslide body, the spacing between the flight paths is about 5m, the lateral overlap rate is about 80%, the longitudinal overlap rate is about 70%, and the flight altitude is about 60m.

[0021] In step S1, the drone used is a DJI M300 RTK, which has a maximum flight time of 41 minutes under load, a positioning accuracy of ±1cm, and a wind resistance level of up to level 6; the lidar used is a Livox Mid-360, with a point cloud output format of LAS 1.4, an effective range of 5-200m, a point cloud density of 200 points / m², and a sampling frequency of 10Hz; the multispectral camera used is a Sony... The ILX-F217 has a resolution of 4096×3072 and includes five bands: blue (450-520nm), green (520-600nm), red (630-690nm), red-edge (705-745nm), and near-infrared (770-890nm). It also features 16-bit deep memory. The onboard 5G module uses the ME909s-821 industrial 5G module, with a downlink bandwidth of 1.6Gbps, an uplink bandwidth of 500Mbps, and a latency of <20ms. It supports NSA / SA dual-mode. NSA stands for Non-Standalone, and SA stands for Standalone. NSA / SA dual-mode refers to two deployment architecture modes of the 5G network, allowing devices to seamlessly switch between the two network architectures.

[0022] In this embodiment, in step S1, under the periodic acquisition strategy, the multiple raw laser point cloud data collected by the UAV each time will generate a set of LAS point cloud files. The multiple multispectral image data collected includes multiple TIFF multispectral images, and each multispectral image is labeled with GPS coordinates. Under the emergency acquisition strategy, the emergency acquisition triggering conditions include: the real-time data transmission of the rainfall sensor at the monitoring station exceeds a set threshold and the edge computing node detects abnormal point cloud curvature. The response process is as follows: the monitoring station sends an emergency command to the UAV base station through the LoRa gateway. After receiving the emergency command, the UAV will take off within a specified time and collect data using a spiral flight path and a focused scanning mode. At the same time, the emergency acquisition data is marked as high priority to ensure that the data upload is completed in a short time. Among them, LAS stands for LiDAR Aerial Survey, which is the standard format of lidar data; TIFF stands for Tagged Image File Format, which is a raster image file format that supports multiple spectral bands.

[0023] S2. The edge computing node first performs point cloud sparsification on multiple original laser point cloud data through a parameterized sparsity module, then uses a lightweight Transformer network to analyze multiple multispectral image data, and uses a deep attention mechanism to extract the texture features of the landslide crack area. Finally, the depth weight allocator fuses the geometric features of the sparsified point cloud with the texture features of the landslide crack area to obtain a heat map of the crack area.

[0024] In this embodiment, during point cloud sparsification in step S2, the landslide crack region is first identified based on the point cloud curvature. Then, different sampling methods are applied to the crack region, the stable landslide region, and the non-landslide background to perform dynamic downsampling. Specifically, voxel center sampling is used in the crack region, random sampling is used in the stable landslide region, and voxel filtering is used in the non-landslide background. High density is maintained in the identified crack region, while the point cloud density in other stable landslide regions and the non-landslide background region is reduced by 90%. This step reduces the original approximately 500MB point cloud data by 85%, and the processing can be completed in just about one minute.

[0025] Specifically, the curvature calculation method for each point cloud is as follows: first search any point cloud P j Construct a local point set S from the 20 nearest neighbors of (x, y, z). j ={P1,P2,...,P20}, then calculate the mean of the local point set of the current point cloud. Construct the corresponding covariance matrix C j And for the covariance matrix C j Perform eigenvalue decomposition to obtain the corresponding eigenvalues ​​λ. j 1. λ j 2 and λ j 3. Then, based on each characteristic value, apply the formula... Calculate the current point cloud curvature K j When the curvature of the point cloud is K j If the value exceeds a threshold, the current point cloud is marked as a crack candidate point. Finally, DBSCAN clustering is applied to all candidate points, and the clustering result represents the crack region. When clustering candidate points, the neighborhood radius is 0.3m, the minimum number of candidate points is 15, and the clustering result is output in vector SHP format, containing information such as crack boundary coordinates, crack length, and average crack curvature, as well as the eigenvalue λ. j 1≥λ j 2≥λ j3, j is the point cloud index; DBSCAN stands for Density-Based Spatial Clustering of Applications with Noise, which is a clustering algorithm based on data point density that can divide points in a dataset into several clusters and identify noise points.

[0026] In this embodiment, in step S2, when extracting the texture features of the landslide crack region using the depth attention mechanism, preprocessing is first performed, and each band B... i Normalization to [0,1] is performed using the following formula: Wherein, band refers to the i-th specific wavelength range in the image acquired by the multispectral camera, B i Let be the original pixel value of the i-th spectral band. For the normalized result of the original pixel value of the i-th spectral band, min(B i ) and max(B i The minimum and maximum gray values ​​of all pixels in the i-th spectral band are given by , respectively; and the Normalized Difference Vegetation Index (NDVI) is calculated using the following formula: NDVI, short for Normalized Difference Vegetation Index, has a value range of [-1, 1]. It is used to distinguish vegetated areas from bare or cracked areas, and to assist in crack boundary identification. When NDVI is greater than 0.3, the current area is marked as a vegetated area; B NIR This refers to the pixel value in the near-infrared band, where vegetation has high reflectivity. (B) Red The pixel values ​​are in the red light band, where vegetation has strong absorption; then feature extraction is performed: first, patch embedding is performed, t patch =x patch ×W proj +W pos Then calculate the local attention Attn local Its formula is: Finally, the output is a dimensional texture feature map; where local attention Attn local The Softmax function is used to calculate the contribution of different image patches to the crack texture features. X patch W represents local image patch data after multispectral imagery has been segmented. proj W is a linear projection weight matrix used to map image patch data to the feature space. pos T is a location-encoded vector used to supplement the spatial location information of image patches. path The embedded image patch feature vector, which integrates the original features and location information, QKT For dot product attention, M is the window mask and V is the value matrix.

[0027] In this embodiment, in step S2, when generating the heatmap using the depth weight allocator, the confidence level Conf of the point cloud geometric features of each crack region is first calculated. The calculation formula is as follows: Then, resource allocation is calculated based on the confidence level Conf. For regions with Conf ≥ 0.7, feature fusion is iterated 8 times; for regions with Conf < 0.7, feature fusion is iterated 2 times. After allocation, feature fusion is performed using the following formula: Where α = 0.6 is the balance coefficient, K is the average curvature of the point cloud, and W... attn F represents the attention weight for texture features, used to quantify the reliability of texture information from different regions in multispectral imagery, such as crack edges and background vegetation, for crack identification. fusion For fusion features, β is the weight of the point cloud geometric features, and F pc (x,y) represents the geometric features of the point cloud, F ms (x,y) represents the texture features.

[0028] In this embodiment, in step S2, when generating the heatmap, the formula is first used... The crack depth of each crack region is calculated, and then the Jet color mapping mechanism is used to map any crack depth in different ranges to a corresponding color, where k cal These are the field calibration coefficients.

[0029] Specifically, when D(x,y) is less than 5m, the crack depth corresponds to blue; when D(x,y) is greater than or equal to 5m and less than 15m, the crack depth corresponds to yellow; and when D(x,y) is greater than or equal to 15m, the crack depth corresponds to red. The generated heatmap is output in GeoTIFF format, with a built-in WGS84 coordinate system and a resolution of 0.01m / pixel. Figure 1 The metadata it outputs includes information such as collection time, drone number, processing node IP, and accuracy metrics.

[0030] S3. The obtained heat map of the crack area will be uploaded to the geological disaster monitoring and early warning platform. The platform will automatically compare it with previous heat maps and calculate the expansion rate and depth change trend of the crack area. At the same time, it will combine real-time rainfall data to determine the warning level. If the platform determines that the crack area is expanding rapidly and the expansion rate reaches a preset threshold, an orange warning will be triggered and a warning message will be sent. The warning message will be sent through platform SMS, APP push and on-site sound and light alarms, and will be sent to disaster prevention personnel, monitors and threatened people as soon as possible. The warning message includes suggested evacuation routes and safe area diagrams.

[0031] In summary, this invention, by employing a crack monitoring mode that incorporates UAV data collection, lightweight edge computing node processing, and cloud-based fusion early warning, enables a rapid response across the entire chain from landslide crack identification to early warning. This improves processing timeliness, meets real-time monitoring requirements, and significantly enhances crack depth detection accuracy. Simultaneously, it reduces computational resource consumption, lowers hardware costs, and its lightweight design supports portable deployment, making it suitable for field and remote environments. It offers better timeliness, accuracy, and applicability, demonstrating significant advancements in technology.

[0032] The above embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. A method for generating disaster risk heatmaps based on a deep attention mechanism, characterized in that, The method includes the following steps: S1. Using a drone equipped with lidar and a multispectral camera, aerial scanning of the landslide body is carried out along a preset flight path. A regular and emergency data collection strategy is adopted to collect multiple raw lidar point cloud data and multiple multispectral image data of the landslide body. The collected data is then transmitted to the edge computing node deployed in the monitoring station near the landslide body through the airborne 5G module. S2. The edge computing node first performs point cloud sparsification on multiple original laser point cloud data through the parameterized sparsity module, then uses a lightweight Transformer network to analyze multiple multispectral image data, and uses a deep attention mechanism to extract the texture features of the landslide crack area. Finally, the depth weight allocator fuses the geometric features of the sparsified point cloud with the texture features of the landslide crack area to obtain a heat map of the crack area. When performing point cloud sparsification, the landslide crack regions are first identified based on the point cloud curvature. Then, different sampling methods are applied to the crack regions, stable landslide regions, and non-landslide backgrounds for dynamic downsampling. Voxel center sampling is used in the crack regions, random sampling is used in the stable landslide regions, and voxel filtering is used in the non-landslide backgrounds. High density is maintained in the identified crack regions, while the point cloud density in other stable landslide regions and non-landslide background regions is reduced by 90%. When extracting texture features from crack areas in a landslide, first divide each band B... i Normalization is performed, and the Normalized Difference Vegetation Index (NDVI) is calculated before feature extraction: first, patch embedding is performed, t patch =x patch ×W proj +W pos Then calculate the local attention Attn local Its formula is: Finally, the output is a dimensional texture feature map; where B i X represents the original pixel value of the i-th spectral band. patch W is a local image patch data of a multispectral image. proj W is the linear projection weight matrix. pos For the position-encoded vector, t patch QK is the embedded image patch feature vector. T For dot product attention, M is the window mask and V is the value matrix; When performing feature fusion, first calculate the confidence score Conf of the point cloud geometric features for each crack region. The calculation formula is as follows: For regions with Conf≥0.7, feature fusion is iterated 8 times; for regions with Conf<0.7, feature fusion is iterated 2 times. Then, resource allocation is calculated based on the confidence level Conf, and feature fusion is performed again. The fusion formula is: Where α is the balance coefficient, K is the average curvature of the point cloud, and W... attn For the attention weights of texture features, F fusion For fusion features, β is the weight of the point cloud geometric features, and F pc (x,y) represents the geometric features of the point cloud, F ms (x,y) represents the texture features; When generating a heatmap, first follow the formula The crack depth of each crack region is calculated, and then the Jet color mapping mechanism is used to map any crack depth in different ranges to a corresponding color, where k cal For on-site calibration coefficients; S3. The obtained thermal map of the crack area will be uploaded to the geological disaster monitoring and early warning platform. The platform will automatically compare it with previous thermal maps and calculate the expansion rate and depth change trend of the crack area. If the platform determines that the crack area is expanding rapidly and the expansion rate reaches a preset threshold, an orange warning will be triggered and a warning message will be sent.

2. The disaster risk heatmap generation method based on deep attention mechanism according to claim 1, characterized in that, In step S1, the drone is a DJI M300 RTK, the lidar is a Livox Mid-360, and the multispectral camera is a Sony ILX-F217.

3. The disaster risk heatmap generation method based on deep attention mechanism according to claim 1, characterized in that, In step S1, in the periodic acquisition strategy, the multiple raw laser point cloud data collected by the UAV each time will generate a set of LAS point cloud files. The multiple multispectral image data collected contain multiple TIFF multispectral images, and each multispectral image is labeled with GPS coordinates.

4. The disaster risk heatmap generation method based on deep attention mechanism according to claim 1, characterized in that, In step S1, the emergency data acquisition strategy is triggered when the real-time data transmitted by the rainfall sensor at the monitoring station exceeds a set threshold and the edge computing node detects an abnormal point cloud curvature. The response process is as follows: the monitoring station sends an emergency command to the UAV base station through the LoRa gateway, and the UAV collects data using a spiral flight path and a key scanning mode after receiving the emergency command.

5. The disaster risk heatmap generation method based on deep attention mechanism according to claim 1, characterized in that, In step S1, the edge computing node sends a reception progress frame to the drone in real time, displaying the number of packets received and the loss rate.

6. The disaster risk heatmap generation method based on deep attention mechanism according to claim 1, characterized in that, In step S2, the point cloud curvature calculation method is as follows: first search any point cloud P j Construct a local point set S from the 20 nearest neighbors of (x, y, z). j ={P1,P2,...,P20}, then calculate the mean of the local point set of the current point cloud. Construct the corresponding covariance matrix C j And for the covariance matrix C j Perform eigenvalue decomposition to obtain the corresponding eigenvalues ​​λ. j 1. λ j 2 and λ j 3. Then, based on each characteristic value, apply the formula... Calculate the current point cloud curvature K j To identify the crack region, where j is the point cloud index.

Images