Unmanned aerial vehicle low-altitude environment landslide detection method, device and equipment and storage medium

Abstract

The invention relates to the technical field of landslide detection, and discloses an unmanned aerial vehicle low-altitude environment landslide detection method, device and equipment and a storage medium, and the method comprises the steps: obtaining a real-time video stream and real-time point cloud data collected by an unmanned aerial vehicle, then carrying out the preprocessing of the real-time video stream, obtaining an RGB image, carrying out the preprocessing of the real-time point cloud data, and obtaining an RGB image; obtaining target point cloud data, and converting the target point cloud data into a DEM image; image analysis is carried out based on the RGB image, the DEM image and a trained landslide detection model, a landslide detection result is obtained, and the landslide detection model comprises a data fusion module, a convolutional neural network, an attention mechanism module, a PPM module, a splicing module and a convolution module. The RGB image and the DEM image can be accurately predicted through the landslide detection model, real-time landslide detection and early warning are achieved, and a precious time window is provided for emergency rescue and evacuation work.

Description

Technical Field

[0001] The present invention relates to the technical field of landslide detection, and in particular to a method, device, equipment and storage medium for detecting landslides in a low-altitude environment of an unmanned aerial vehicle. Background Art

[0002] In recent years, the intensification of global climate change, the frequent occurrence of extreme weather events, and the continuous expansion of human activities have jointly led to a significant increase in the frequency of landslide disasters. As a highly destructive sudden geological disaster, once a landslide occurs, a large amount of soil will slide rapidly along a specific sliding surface, posing a serious threat to human living environment and production activities. In view of the frequency and potential harm of landslide disasters, it is particularly urgent and necessary to realize real-time monitoring and early warning of landslides.

[0003] However, current landslide detection has low accuracy and is not real-time, which is a serious problem for dealing with sudden natural disasters such as landslides. Landslides are often unpredictable and progress rapidly, so timely and effective landslide detection and early warning are crucial to reduce casualties and property losses. Summary of the invention

[0004] Based on this, it is necessary to propose a method, device, equipment and storage medium for landslide detection in a low-altitude environment using an unmanned aerial vehicle, in view of the low accuracy of current landslide detection in the existing technology and the significant technical problems of insufficient real-time performance.

[0005] In a first aspect, a method for detecting landslides in a low-altitude environment using an unmanned aerial vehicle is provided, the method comprising:

[0006] Obtain real-time video streams and real-time point cloud data collected by drones;

[0007] Preprocessing the real-time video stream to obtain an RGB image, preprocessing the real-time point cloud data to obtain target point cloud data, and converting the target point cloud data into a DEM image;

[0008] Image analysis is performed based on the RGB image, the DEM image and the trained landslide detection model to obtain a landslide detection result, wherein the landslide detection model includes a data fusion module, a convolutional neural network, an attention mechanism module, a PPM module, a splicing module and a convolution module, the output end of the data fusion module is connected to the input end of the convolutional neural network, the output end of the convolutional neural network is connected to the input end of the attention mechanism module, the output end of the convolutional neural network is connected to the input end of the PPM module, the output end of the attention mechanism module and the output end of the PPM module are both connected to the input end of the splicing module, and the output end of the splicing module is connected to the input end of the convolution module.

[0009] In a second aspect, a landslide detection device is provided, the device comprising:

[0010] The acquisition module is used to obtain the real-time video stream and real-time point cloud data collected by the drone;

[0011] A preprocessing module is used to preprocess the real-time video stream to obtain an RGB image, preprocess the real-time point cloud data to obtain target point cloud data, and convert the target point cloud data into a DEM image;

[0012] An image analysis module is used to perform image analysis based on the RGB image, the DEM image and the trained landslide detection model to obtain a landslide detection result, wherein the landslide detection model includes a data fusion module, a convolutional neural network, an attention mechanism module, a PPM module, a splicing module and a convolution module, the output end of the data fusion module is connected to the input end of the convolutional neural network, the output end of the convolutional neural network is connected to the input end of the attention mechanism module, the output end of the convolutional neural network is connected to the input end of the PPM module, the output end of the attention mechanism module and the output end of the PPM module are both connected to the input end of the splicing module, and the output end of the splicing module is connected to the input end of the convolution module.

[0013] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned UAV low-altitude environment landslide detection method when executing the computer program.

[0014] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned UAV low-altitude environment landslide detection method are implemented.

[0015] The method for detecting landslides in a low-altitude environment using a drone proposed in the present invention obtains real-time video streams and real-time point cloud data collected by a drone, then pre-processes the real-time video stream to obtain an RGB image, pre-processes the real-time point cloud data to obtain target point cloud data, and converts the target point cloud data into a DEM image. Finally, image analysis is performed based on the RGB image, the DEM image and a trained landslide detection model to obtain a landslide detection result, wherein the landslide detection model includes a data fusion module, a convolutional neural network, an attention mechanism module, a PPM module, a splicing module and a convolution module, the output end of the data fusion module is connected to the input end of the convolutional neural network, the output end of the convolutional neural network is connected to the input end of the attention mechanism module, the output end of the convolutional neural network is connected to the input end of the PPM module, the output end of the attention mechanism module and the output end of the PPM module are both connected to the input end of the splicing module, and the output end of the splicing module is connected to the input end of the convolution module. The landslide detection model can be used to accurately predict RGB images and DEM images, realize real-time landslide detection and early warning, and provide a valuable time window for emergency rescue and evacuation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] in:

[0018] Figure 1 This is an application environment diagram of a method for detecting landslides in a low-altitude environment using a drone in one embodiment;

[0019] Figure 2 Flow chart of a method for detecting landslides in a low-altitude environment using a drone in one embodiment;

[0020] Figure 3 A schematic diagram of the structure of a landslide detection model of a landslide detection method in a low-altitude environment of an unmanned aerial vehicle in an embodiment;

[0021] Figure 4 is a structural block diagram of a landslide detection device in one embodiment;

[0022] Figure 5 is a structural block diagram of a computer device in one embodiment;

[0023] Figure 6 It is a structural block diagram of a computer device in another embodiment. DETAILED DESCRIPTION

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0025] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The method for detecting landslides in a low-altitude environment provided by an unmanned aerial vehicle in an embodiment of the present invention can be applied in the following aspects: Figure 1In an application environment, the client 110 communicates with the server 120 through a network. The server 120 can obtain the real-time video stream and real-time point cloud data collected by the drone through the client 110, then pre-process the real-time video stream to obtain an RGB image, pre-process the real-time point cloud data to obtain target point cloud data, and convert the target point cloud data into a DEM image, and finally perform image analysis based on the RGB image, the DEM image and the trained landslide detection model to obtain a landslide detection result, wherein the landslide detection model includes a data fusion module, a convolutional neural network, an attention mechanism module, a PPM module, a splicing module and a convolution module, the output end of the data fusion module is connected to the input end of the convolutional neural network, the output end of the convolutional neural network is connected to the input end of the attention mechanism module, the output end of the convolutional neural network is connected to the input end of the PPM module, the output end of the attention mechanism module and the output end of the PPM module are both connected to the input end of the splicing module, and the output end of the splicing module is connected to the input end of the convolution module. The landslide detection model can accurately predict the RGB image and the DEM image, realize real-time detection of landslide and early warning, and provide a valuable time window for emergency rescue and evacuation. Among them, the client 110 can be but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server 120 can be implemented by an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0028] See also Figure 2 As stated, Figure 2 A flow chart of a method for detecting landslides in a low-altitude environment using an unmanned aerial vehicle according to an embodiment of the present invention includes the following steps:

[0029] Step S101: Acquire real-time video stream and real-time point cloud data collected by the drone;

[0030] Among them, the drone is equipped with an L2 laser radar, which has high-precision positioning and powerful flight performance, and is suitable for long-term flight and precise data collection in complex environments. The L2 laser radar can generate high-precision three-dimensional point cloud data with a wide coverage and high resolution. These data, combined with RGB images, can provide detailed information on the landslide area. The drone has strong wind resistance and long-term flight capabilities, and can work stably under severe weather conditions. It supports the mounting of multiple cameras and sensors and has flexible scalability. The L2 laser radar has strong anti-interference ability and environmental adaptability, can work stably in complex environments, has a fast data collection speed, and can meet the needs of real-time monitoring. By combining the drone and the L2 laser radar, the present invention can achieve real-time monitoring and early warning of the landslide area, providing strong technical support for the prevention and response of landslide disasters.

[0031] As an example, the DJI Skong 2 platform is used as a drone task management cloud platform to realize the automated management and scheduling of drone tasks. Skong 2 is a drone task management cloud platform launched by DJI Innovations. It provides powerful task release, execution and monitoring functions, which can effectively improve the efficiency and safety of drone operations. Through the Skong 2 platform, users can set the parameters of drone inspection tasks, such as flight altitude, flight speed, shooting interval, etc., and automatically plan the flight path of the drone to ensure that the drone conducts full coverage inspection in the target area. During the flight, the drone collects RGB images and laser point cloud data according to the preset parameters, and transmits the collected data to the Skong 2 platform in real time. The platform can monitor the status and data collection of the drone in real time to ensure the smooth progress of data collection. At the same time, the Skong 2 platform can store, manage and query the collected data for subsequent use. Through the Skong 2 platform, the automated management and scheduling of drone tasks are realized, the efficiency and accuracy of data collection are improved, and strong support is provided for landslide detection.

[0032] As another example, when performing landslide detection tasks, drones need to transmit two key data in real time: real-time video streams and real-time point cloud data. To achieve this goal, the present invention adopts two different technologies. For real-time video streams, WebRTC technology is used for transmission. WebRTC is an advanced real-time communication technology that enables direct video communication between clients and servers without going through traditional network infrastructure. This point-to-point communication method has the characteristics of low latency and high efficiency, which is very suitable for the transmission needs of real-time video streams. On DJI drones, through the customized VideoCapturer, high-definition video streams can be obtained in real time and transmitted to the server in a WebRTC-compatible format. This transmission method not only ensures the real-time and stability of the video stream, but also can greatly reduce the delay in the transmission process, providing high-quality input for subsequent landslide detection algorithms.

[0033] For real-time point cloud data, WebSocket technology is used for transmission. WebSocket is a full-duplex communication protocol that allows full-duplex communication on a single TCP connection and is suitable for transmitting real-time text data between the client and the server. On the drone, the precise point cloud data generated by the lidar is transmitted to the server in real time via WebSocket technology. This transmission method ensures the real-time and accuracy of the point cloud data, enabling the server to receive and process the point cloud data in real time, providing rich three-dimensional information for the landslide detection algorithm.

[0034] Through the combination of WebRTC and WebSocket technology, drones can efficiently transmit real-time video streams and point cloud data to the server, providing reliable data input for subsequent landslide detection algorithms. This data transmission mechanism not only improves the efficiency and accuracy of data transmission, but also provides strong technical support for the prevention and response of landslide disasters.

[0035] Step S102: preprocessing the real-time video stream to obtain an RGB image, preprocessing the real-time point cloud data to obtain target point cloud data, and converting the target point cloud data into a DEM image;

[0036] In this embodiment, in landslide detection, data preprocessing is a crucial step, which directly affects the accuracy and efficiency of subsequent analysis. In this step, the collected image data and laser point cloud data are mainly processed and converted to optimize the effect of subsequent analysis.

[0037] For real-time video streams, image data is captured from the real-time video streams collected by drones, and denoised and enhanced. The denoising process uses filters to remove noise in the image, such as Gaussian noise, salt and pepper noise, to improve image quality. The enhancement process adjusts image brightness, contrast, saturation and other parameters to improve the visibility of the image, making it more in line with the needs of subsequent analysis. After the preprocessing is completed, a high-quality RGB image can be obtained.

[0038] For real-time point cloud data, the first step is to perform a registration operation to transform and register the real-time point cloud data acquired at different times or from different sensors to ensure that they are in the same coordinate system. Subsequently, a denoising operation is performed to remove noise points in the point cloud, such as outliers and duplicate points, to improve the quality of the point cloud data. This step is crucial for subsequent analysis because it ensures the accuracy and consistency of the point cloud data. The real-time point cloud data after preprocessing can be used as the target point cloud data. Through these preprocessing steps, the quality of image data and laser point cloud data and the accuracy of analysis can be effectively improved, providing strong support for subsequent landslide detection. Finally, the target point cloud data needs to be converted into a DEM image.

[0039] In landslide detection systems, registration is required to ensure spatial consistency between DEM (Digital Elevation Model) images and RGB images. Registration is a key technology in the field of Geographic Information Systems (GIS) and remote sensing, which involves aligning two or more images or data sets from different sources into the same coordinate system. In landslide detection, DEM images provide accurate elevation information, while RGB images provide rich texture and color information. By registering these two images, accurate analysis and monitoring of landslide areas can be achieved. The registration process usually includes steps such as feature extraction, feature matching, transformation estimation, transformation application, and quality assessment to ensure spatial consistency between DEM images and RGB images, providing an accurate data basis for landslide detection and analysis.

[0040] Step S103: performing image analysis based on the RGB image, the DEM image and the trained landslide detection model to obtain a landslide detection result, wherein the landslide detection model includes a data fusion module, a convolutional neural network, an attention mechanism module, a PPM module, a splicing module and a convolution module, the output end of the data fusion module is connected to the input end of the convolutional neural network, the output end of the convolutional neural network is connected to the input end of the attention mechanism module, the output end of the convolutional neural network is connected to the input end of the PPM module, the output end of the attention mechanism module and the output end of the PPM module are both connected to the input end of the splicing module, and the output end of the splicing module is connected to the input end of the convolution module.

[0041] Among them, the PPM (Pyramid Pooling) model can capture multi-scale contextual information by performing pooling operations on feature maps of different scales. Specifically, it performs pooling operations of different sizes on feature maps (such as 1x1, 2x2, 3x3, 6x6). The main task of the data fusion module is to fuse the RGB image and the DEM image together to form an image containing four channels. In this way, the data fusion module enables the landslide detection model to understand and identify landslide areas from multiple angles, improving the accuracy and reliability of detection.

[0042] In this embodiment, the registered RGB image and DEM image are first input into the data fusion module, so that the registered RGB image and DEM image are fused together to create a comprehensive image data set. In this process, the elevation information of the DEM image is recorded in the transparency channel, so that the corresponding elevation value is superimposed on each pixel position of the RGB image. This fusion method makes full use of the visual information of the RGB image and the three-dimensional information of the DEM image, and provides more comprehensive and accurate data support for landslide detection. In this way, accurate analysis and monitoring of the landslide area can be achieved, thereby improving the accuracy and reliability of landslide detection.

[0043] The attention mechanism module can automatically identify key areas in the image that are relevant to the detection task and use them as the focus of model learning. This mechanism helps the model capture features related to landslides more accurately, thereby improving the accuracy and robustness of detection. The output of the attention mechanism module is fused with the output of the PPM module to enhance the focus on the landslide area.

[0044] like Figure 3 As shown in the figure, the RGB image and the DEM image are fused through the data module to obtain a four-channel image of RGB+DEM. Then it is input into the backbone network (CNN) for feature extraction to obtain the feature Figure 1 , and then the feature Figure 1 Input into the PPM module and attention mechanism module respectively to obtain features Figure 2 and Features Figure 3 , the feature Figure 1 , 2, 3 are combined to obtain the final feature map. Finally, the final feature map is convolved to obtain the final landslide detection result.

[0045] As an example, the data set used to train the landslide detection model is the landslide data set of City B, which is a data set for landslide events in City B, Province A. This data set contains information on landslide events in City B and surrounding areas, and is mainly used for geological disaster research, analysis and prediction. In this data set, there are 770 RGB images containing landslide areas, each RGB image has a corresponding DEM image, and there are 2003 images that do not contain landslide areas. Then, 70% of the images in the data set are used as training sets, and the other 30% of the images are used as test sets. This builds a solid data foundation for the training and verification of the model. Next, the landslide detection model can be trained in order to achieve better results in landslide identification and prediction.

[0046] After the model training is completed, the trained landslide detection model can be put into practical application, especially in areas with high landslide risk. In these areas, the drone uses a high-resolution optical camera and a lidar system to capture RGB images and laser point cloud data respectively, and transmits them to the server in real time. After receiving the data, the server goes through a series of preprocessing steps, including data fusion, to combine the RGB image and the DEM image into a fused image containing four channels. Subsequently, this fused image is input into the trained landslide detection model, which performs real-time landslide detection and prediction by analyzing the feature information in the fused image.

[0047] Once the model detects signs of landslide, the system will immediately trigger the early warning system, buying precious time for emergency rescue and evacuation. In addition, these detection results can also be used for landslide risk assessment and disaster prevention. By continuously monitoring the landslide area, potential risks can be discovered in a timely manner, providing a basis for decision makers to take preventive measures. Therefore, by actually applying the trained landslide detection model in landslide-prone areas, real-time monitoring and early warning of landslide areas can be achieved, providing strong technical support for the prevention and response of landslide disasters.

[0048] In one embodiment, the step of performing image analysis based on the RGB image, the DEM image and the trained landslide detection model to obtain a landslide detection result includes:

[0049] Step S201: inputting the RGB image and the DEM image into the data fusion model for image fusion to obtain a four-channel image output by the data fusion model;

[0050] Step S202: Inputting the four-channel image into the convolutional neural network to extract features, and obtaining a first feature map output by the convolutional neural network;

[0051] Step S203: inputting the first feature map into the attention mechanism module for processing to obtain a second feature map output by the attention mechanism module;

[0052] Step S204: inputting the first feature into the PPM module for processing to obtain a third feature map output by the PPM module;

[0053] Step S205: inputting the first feature map, the second feature map and the third feature map into the stitching module to merge the images to obtain a fourth feature map;

[0054] Step S206: input the fourth feature map into the convolution module for convolution processing to obtain target features.

[0055] See also Figure 4 In one embodiment, a landslide detection device is provided, the device comprising:

[0056] An acquisition module 10 is used to acquire real-time video stream and real-time point cloud data collected by the drone;

[0057] A preprocessing module 20 is used to preprocess the real-time video stream to obtain an RGB image, preprocess the real-time point cloud data to obtain target point cloud data, and convert the target point cloud data into a DEM image;

[0058] The image analysis module 30 is used to perform image analysis based on the RGB image, the DEM image and the trained landslide detection model to obtain a landslide detection result, wherein the landslide detection model includes a data fusion module, a convolutional neural network, an attention mechanism module, a PPM module, a splicing module and a convolution module, the output end of the data fusion module is connected to the input end of the convolutional neural network, the output end of the convolutional neural network is connected to the input end of the attention mechanism module, the output end of the convolutional neural network is connected to the input end of the PPM module, the output end of the attention mechanism module and the output end of the PPM module are both connected to the input end of the splicing module, and the output end of the splicing module is connected to the input end of the convolution module.

[0059] In one embodiment, the image analysis module 30 is further used for:

[0060] Inputting the RGB image and the DEM image into the data fusion model for image fusion to obtain a four-channel image output by the data fusion model;

[0061] Input the four-channel image into the convolutional neural network to extract features, and obtain a first feature map output by the convolutional neural network;

[0062] Inputting the first feature map into the attention mechanism module for processing to obtain a second feature map output by the attention mechanism module;

[0063] Inputting the first feature into the PPM module for processing to obtain a third feature map output by the PPM module;

[0064] Inputting the first feature map, the second feature map and the third feature map into the stitching module to merge the images to obtain a fourth feature map;

[0065] The fourth feature map is input into the convolution module for convolution processing to obtain the target feature.

[0066] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 The computer device comprises a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, the functions or steps on the service side of a method for detecting landslides in a low-altitude environment of an unmanned aerial vehicle are realized.

[0067] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 6 The computer device comprises a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, the functions or steps on the client side of a method for detecting landslides in a low-altitude environment of an unmanned aerial vehicle are realized.

[0068] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0069] Obtain real-time video streams and real-time point cloud data collected by drones;

[0070] Preprocessing the real-time video stream to obtain an RGB image, preprocessing the real-time point cloud data to obtain target point cloud data, and converting the target point cloud data into a DEM image;

[0071] Image analysis is performed based on the RGB image, the DEM image and the trained landslide detection model to obtain a landslide detection result, wherein the landslide detection model includes a data fusion module, a convolutional neural network, an attention mechanism module, a PPM module, a splicing module and a convolution module, the output end of the data fusion module is connected to the input end of the convolutional neural network, the output end of the convolutional neural network is connected to the input end of the attention mechanism module, the output end of the convolutional neural network is connected to the input end of the PPM module, the output end of the attention mechanism module and the output end of the PPM module are both connected to the input end of the splicing module, and the output end of the splicing module is connected to the input end of the convolution module.

[0072] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0073] Obtain real-time video streams and real-time point cloud data collected by drones;

[0074] Preprocessing the real-time video stream to obtain an RGB image, preprocessing the real-time point cloud data to obtain target point cloud data, and converting the target point cloud data into a DEM image;

[0075] Image analysis is performed based on the RGB image, the DEM image and the trained landslide detection model to obtain a landslide detection result, wherein the landslide detection model includes a data fusion module, a convolutional neural network, an attention mechanism module, a PPM module, a splicing module and a convolution module, the output end of the data fusion module is connected to the input end of the convolutional neural network, the output end of the convolutional neural network is connected to the input end of the attention mechanism module, the output end of the convolutional neural network is connected to the input end of the PPM module, the output end of the attention mechanism module and the output end of the PPM module are both connected to the input end of the splicing module, and the output end of the splicing module is connected to the input end of the convolution module.

[0076] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0077] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0078] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0079] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for detecting landslides in a low-altitude environment using an unmanned aerial vehicle, characterized in that: The method for detecting landslides in a low-altitude environment using an unmanned aerial vehicle comprises: Obtain real-time video streams and real-time point cloud data collected by drones; Preprocessing the real-time video stream to obtain an RGB image, preprocessing the real-time point cloud data to obtain target point cloud data, and converting the target point cloud data into a DEM image; Image analysis is performed based on the RGB image, the DEM image and the trained landslide detection model to obtain a landslide detection result, wherein the landslide detection model includes a data fusion module, a convolutional neural network, an attention mechanism module, a PPM module, a splicing module and a convolution module, the output end of the data fusion module is connected to the input end of the convolutional neural network, the output end of the convolutional neural network is connected to the input end of the attention mechanism module, the output end of the convolutional neural network is connected to the input end of the PPM module, the output end of the attention mechanism module and the output end of the PPM module are both connected to the input end of the splicing module, and the output end of the splicing module is connected to the input end of the convolution module.

2. The method for detecting landslides in a low-altitude environment using an unmanned aerial vehicle according to claim 1, characterized in that: The step of performing image analysis based on the RGB image, the DEM image and the trained landslide detection model to obtain the landslide detection result comprises: Inputting the RGB image and the DEM image into the data fusion model for image fusion to obtain a four-channel image output by the data fusion model; Input the four-channel image into the convolutional neural network to extract features, and obtain a first feature map output by the convolutional neural network; Inputting the first feature map into the attention mechanism module for processing to obtain a second feature map output by the attention mechanism module; Inputting the first feature into the PPM module for processing to obtain a third feature map output by the PPM module; Inputting the first feature map, the second feature map and the third feature map into the stitching module to merge the images to obtain a fourth feature map; The fourth feature map is input into the convolution module for convolution processing to obtain the target feature.

3. A landslide detection device, characterized in that: The landslide detection device comprises: The acquisition module is used to obtain the real-time video stream and real-time point cloud data collected by the drone; A preprocessing module is used to preprocess the real-time video stream to obtain an RGB image, preprocess the real-time point cloud data to obtain target point cloud data, and convert the target point cloud data into a DEM image; An image analysis module is used to perform image analysis based on the RGB image, the DEM image and the trained landslide detection model to obtain a landslide detection result, wherein the landslide detection model includes a data fusion module, a convolutional neural network, an attention mechanism module, a PPM module, a splicing module and a convolution module, the output end of the data fusion module is connected to the input end of the convolutional neural network, the output end of the convolutional neural network is connected to the input end of the attention mechanism module, the output end of the convolutional neural network is connected to the input end of the PPM module, the output end of the attention mechanism module and the output end of the PPM module are both connected to the input end of the splicing module, and the output end of the splicing module is connected to the input end of the convolution module.

4. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for detecting landslides in a low-altitude environment using an unmanned aerial vehicle as described in any one of claims 1 to 2 are implemented.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, the steps of the method for detecting landslides in a low-altitude environment using an unmanned aerial vehicle as claimed in any one of claims 1 to 2 are implemented.

Images