A three-dimensional unmanned multi-platform linkage landslide intelligent patrol system for land, water and air

Abstract

The invention discloses an amphibious, three-dimensional, unmanned, multi-platform linked landslide intelligent patrol system. The patrol system comprises an information acquisition system, an intelligent dock system and a monitoring and early warning system. The information acquisition system comprises a drone, a patrol robot dog and an unmanned boat; the intelligent dock system comprises a self-sensing hatch for the entry and exit of information acquisition system equipment, a wireless transmission module and a wireless charging platform; the monitoring and early warning system comprises a data processing module, a three-dimensional modeling module, a hazard prediction and early warning module, various data of the monitored slope are acquired through the unmanned boat, the patrol robot dog and the drone, and the data are sent to the monitoring and early warning system through the wireless transmission module; the three-dimensional modeling module establishes a three-dimensional model of the slope according to the acquired information and predicts the landslide situation of the slope; the hazard prediction and early warning module executes an early warning when the probability value of the landslide reaches a set limit, thereby performing a multi-dimensional evaluation of the slope and improving the accuracy of the precise early warning of the slope landslide.

Description

Technical Field

[0001] The present invention belongs to the field of geological disaster monitoring, and relates to an amphibious, three-dimensional, unmanned, multi-platform linked landslide intelligent patrol system. Background Art

[0002] In particular, large and medium-sized landslides are difficult to prevent and rescue, cause great harm, and are difficult to control.

[0003] The traditional method of landslide monitoring is mainly geodetic surveying, which takes manual monitoring as the main means and monitors deformation through total station wires and electromagnetic wave ranging triangulation height method. Although geodetic surveying has accumulated a lot of experience, its monitoring frequency is low, the monitoring gap period is long, and personnel are required to observe on-site, which results in a large workload. In complex environments, the safety of surveyors is difficult to guarantee.

[0004] In recent years, drone technology has developed rapidly, and monitoring methods such as laser scanning and digital photogrammetry have become more mature. Drone remote sensing has begun to be widely used in the field of natural disaster monitoring. In particular, high-resolution optical or radar remote sensing has begun to be widely used in landslide monitoring and analysis, but inspection robot dogs and unmanned boats are rarely used in landslide monitoring.

[0005] The current landslide monitoring system uses drones as the monitoring platform and does not adopt amphibious three-dimensional technology, making it impossible to effectively monitor landslides in the monitoring area. China's geological disaster sites are numerous and widespread, most of which have complex environments and are covered by vegetation, which restricts the flight of drones. Landslides often occur under extreme weather conditions. When the environmental wind force exceeds the drone's tolerance, it will cause the aircraft to drift and crash. UAV remote sensing measurement technology is also powerless to prevent landslides in the degraded zone of the drawdown area caused by seasonal rise and fall of reservoir water.

[0006] Taking into full consideration the geological environment and weather conditions in which landslides occur, it has become very difficult to monitor landslides in some areas using traditional landslide monitoring methods and previous unmanned monitoring systems. Therefore, it is of great practical significance for disaster prevention and mitigation to build an economical, practical, efficient and intelligent landslide monitoring system to achieve full-range normalized monitoring of landslide disasters under extreme conditions and surface deformation analysis. Summary of the invention

[0007] In view of the geological disasters such as landslides and collapses that are easily caused by extreme natural conditions such as heavy rainfall, and the landslides in the degraded zone of the drawdown area caused by the seasonal rise and fall of reservoir water, the present invention provides a three-dimensional unmanned multi-platform linkage landslide intelligent patrol system for land, water and air. The system uses a new land, water and air trinity surveying technology, that is, an intelligent unmanned boat equipped with a multi-beam depth sounder is used to obtain monitoring data of the seasonal rise and fall area of ​​the reservoir water; a true three-dimensional model of the monitoring area is obtained by using drones and patrol robot dogs equipped with laser radar, and then accurate land monitoring data is obtained; circular signs are set in areas with potential landslide geological disaster hazards, combined with binocular stereo vision measurement methods, for supplementary measurement of landslide deformation monitoring. These measures seamlessly connect land monitoring data and water monitoring data, greatly improving the efficiency of landslide monitoring while minimizing the risk of personnel, providing a new solution for landslide monitoring.

[0008] The objective of the present invention is achieved through the following technical solutions:

[0009] An amphibious, three-dimensional, unmanned, multi-platform linkage landslide intelligent patrol system, including an information acquisition system, an intelligent docking system, and a monitoring and early warning system, wherein:

[0010] The information acquisition system is an amphibious platform, including a drone (aerial platform), a patrol robot dog (ground platform) and an unmanned boat (underwater platform);

[0011] The drone is equipped with an airspace information acquisition module and an airspace attitude positioning module;

[0012] The inspection robot dog is equipped with a land information acquisition module and a land posture positioning module;

[0013] The unmanned boat is equipped with a water area information acquisition module and a water area attitude positioning module;

[0014] The water area information acquisition module is a multi-beam echo sounder and a storage device. The multi-beam echo sounder transmits wide-sector sound waves to the underwater area, uses a receiving transducer array to perform narrow-beam reception of the sound waves, forms an illumination footprint of the underwater terrain through the orthogonality of the transmitting and receiving sector directions, processes the illumination footprint, obtains point cloud data of the underwater terrain and stores it in the storage device.

[0015] The land information acquisition module and the air information acquisition module are a three-dimensional laser scanning radar, a binocular stereo vision camera and a memory. The three-dimensional laser scanning radar acquires the original point cloud data of the monitoring area and stores it in the memory. The binocular stereo vision camera acquires the identification point image of the potential landslide geological disaster hazard area and stores it in the memory.

[0016] The airspace attitude positioning module is used to obtain the coordinates, posture and speed information of the UAV, the land area attitude positioning module is used to obtain the coordinates, posture and speed information of the inspection robot dog, and the water area attitude positioning module is used to obtain the coordinates, posture and speed information of the unmanned boat;

[0017] The intelligent docking system includes a self-sensing door for information acquisition system equipment to enter and exit, a wireless transmission module, and a wireless charging platform;

[0018] A solar panel is also provided on one side of the wireless charging platform, and the solar panel is connected to the battery;

[0019] The monitoring and early warning system includes a data processing module, a three-dimensional modeling module, and a danger prediction and early warning module;

[0020] The data processing module performs denoising, inspection and filtering on the point cloud information transmitted from the intelligent docking system, and transmits the processed information to the three-dimensional modeling module and the danger prediction and warning module. The data processing module performs rain and fog removal processing on the image information transmitted from the intelligent docking system to obtain a clear marker point image, and then performs edge detection and ellipse fitting on the marker point image to obtain the three-dimensional coordinates of the marker point in the world coordinate system;

[0021] The three-dimensional modeling module generates a land DEM model and a water DEM model based on the acquired information, and splices the water and land DEM models to obtain an integrated above-water and underwater DEM model;

[0022] The hazard prediction and early warning module establishes a hazard prediction model based on the acquired information, and calculates the probability value of landslide occurrence in the monitored area through the hazard prediction model, and executes an early warning when the probability value of landslide occurrence reaches a set limit.

[0023] A method for using the above-mentioned amphibious three-dimensional unmanned multi-platform linkage landslide intelligent patrol system comprises the following steps:

[0024] Step 1: Preliminary preparation

[0025] According to the activity range of the information acquisition system, the slope is divided into multiple monitoring areas. An intelligent docking system for storage equipment and identification points for binocular stereo vision lenses are built in the central area of ​​the monitoring area. Target points for coordinate registration are set up in the area where water and land meet.

[0026] Step 2: Monitoring Operations

[0027] Step 2.1, use drones for remote sensing monitoring to inspect whether there are landslides with large deformations. The drone low-altitude remote sensing monitoring is used for small-scale slope surface deformation monitoring and records three-dimensional point cloud data. At the same time, the binocular stereo vision camera carried by the drone takes pictures of the landmark points in the potential landslide geological disaster risk area, collects the image information of the landmark points, and stores the three-dimensional point cloud data and image information in the storage;

[0028] Step 2.2, if there is an area in the monitoring area that the drone cannot enter, or the weather conditions are not suitable for drone flight, the inspection robot dog will monitor the area from the intelligent warehouse dock system. The robot dog is equipped with a 3D laser scanning radar to record 3D point cloud data; at the same time, the inspection robot dog uses a binocular stereo vision camera to shoot the landmark points in the potential landslide geological disaster risk area, collect image information of the landmark points, and store the 3D point cloud data and image information in the memory;

[0029] Step 2.3, if there is a slope that is submerged in water for a long time or a short time in the monitoring area, the intelligent dock system in the adjacent water area sends a signal, and the unmanned boat starts from the intelligent dock system to monitor the area. The unmanned boat is equipped with a multi-beam depth sounder, and uses the transmitting transducer array to transmit wide-sector sound waves to the bottom of the water, and uses the receiving transducer array to receive the sound waves in a narrow beam. Through the orthogonality of the transmitting and receiving sector directions, an illumination footprint of the bottom slope is formed, and these footprints are processed. One detection can give the water depth values ​​of hundreds or even more underwater measured points in the vertical plane perpendicular to the heading, and the water depth values ​​of the measured points are stored in the memory;

[0030] Step 2.4: After the inspection of the monitoring area is completed, the drone, inspection robot dog and unmanned boat return to the intelligent docking system, and transmit the monitoring information to the monitoring and early warning system through the wireless transmission module of the intelligent docking system;

[0031] Step 3: Image data processing

[0032] Step 3.1, the image processing module of the monitoring and early warning system processes the image collected by the binocular stereo vision camera through a dark channel prior defogging method and a morphological component analysis image decomposition deraining method to obtain a clear landmark point image;

[0033] Step 3.2: Filter, denoise and binarize the marker point image collected by the binocular stereo vision camera to obtain the circular marker point target parameters, then use the Canny algorithm to roughly locate the edge points, and use the Zernike moment algorithm to perform sub-pixel positioning of the edge to improve the positioning accuracy;

[0034] Step 3.3, using the least square method to fit the ellipse equation, perform ellipse fitting based on the obtained image sub-pixel edge, so as to locate the coordinates of the circle center;

[0035] Step 4: Point cloud data processing

[0036] The monitoring and early warning system performs data denoising, data verification and data filtering on the original point cloud data;

[0037] Step 5: 3D modeling

[0038] The 3D modeling module directly observes and analyzes the elevation information of the monitoring area by constructing a digital elevation model, generates a land DEM model and a water DEM model using ground point cloud data, and then splices the water and land DEM models through coordinate registration to obtain an integrated DEM model above and below the water;

[0039] Step 6: Hazard warning analysis

[0040] Step 6.1, based on the coordinates of the landmark points, combined with the integrated underwater and above-water DEM model, three-dimensional modeling of the landslide is performed, the point cloud data of the landslide body obtained from each monitoring period is compared, and the deformation and sliding data of the characteristic points, characteristic lines, and characteristic surfaces on the landslide body are extracted;

[0041] Step 6.2: The hazard prediction and warning module establishes a hazard prediction model based on the three-dimensional coordinate information of the landmark points and the integrated above-water and underwater DEM model, and calculates the probability value of landslides in the monitored area through the hazard prediction model. When the probability value of landslides reaches the set limit, an early warning is executed.

[0042] By adopting the above technical solution, the information acquisition system obtains various data of the monitored slope through unmanned boats, inspection robot dogs and drones, including three-dimensional point cloud data and landmark point photos, and sends them to the monitoring and early warning system through the wireless transmission module. The three-dimensional modeling module establishes a three-dimensional model of the slope according to the acquired information and predicts the landslide situation of the slope. The hazard prediction and early warning module executes an early warning when the probability value of the landslide reaches the set limit, thereby performing a multi-dimensional evaluation of the slope and improving the accuracy of the precise early warning of slope landslides.

[0043] Compared with the prior art, the present invention has the following advantages:

[0044] 1. The present invention is based on the three-in-one surveying and mapping technology of land, sea and air, and constructs an amphibious, three-dimensional, unmanned, multi-platform linked landslide intelligent patrol and prevention system. Unmanned aerial vehicles, patrol robot dogs and unmanned boats are used to collect information on the landslide area. The unmanned aerial vehicles and patrol robot dogs are equipped with laser radars and binocular stereo vision cameras, and the unmanned boats are equipped with multi-beam depth sounders. Remote, full-range real-time monitoring of the landslide monitoring area can be achieved without the need for personnel to take risks.

[0045] 2. The present invention overcomes the defects of traditional monitoring in extreme weather conditions, such as the inability to dynamically and real-time identify and track landslide potential hazards and sudden points, low efficiency, high difficulty, high risk of manual investigation, and difficulty in actively preventing and controlling potential hazards. It can achieve "intelligent warehouse" rapid response, "autonomous planning" of inspection routes, "identification and tracking" of disaster and dangerous conditions, "three-dimensional platform" intelligent linkage, and "data analysis" forecasting and early warning under extreme natural conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a block diagram of the amphibious, three-dimensional, unmanned, multi-platform linkage landslide intelligent patrol system;

[0047] Figure 2 A flow chart for landslide monitoring;

[0048] Figure 3 This is a flowchart for multivariate data processing. DETAILED DESCRIPTION

[0049] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0050] The present invention provides a three-dimensional unmanned multi-platform linkage landslide intelligent patrol system for land, water and air, such as Figure 1 As shown, the patrol system includes an information acquisition system, an intelligent docking system and a monitoring and early warning system, wherein:

[0051] The information acquisition system is divided into three parts: land, water and air: unmanned boats, inspection robot dogs and drones. Each part is also equipped with an information acquisition module and an attitude positioning module.

[0052] The information acquisition system is an amphibious platform, which is divided into three parts: unmanned boats, inspection robot dogs and drones;

[0053] The drone is equipped with an airspace information acquisition module and an airspace attitude positioning module to conduct a large-scale inspection of the monitoring area;

[0054] The inspection robot dog is equipped with a land information acquisition module and a land attitude positioning module. In extreme weather conditions that restrict the flight of drones, or in a geological environment that is not conducive to the flight of drones, the inspection robot dog monitors from the ground to make up for the drone's monitoring blind spots;

[0055] The unmanned boat is equipped with a water area information acquisition module and a water area attitude positioning module to monitor the seasonal rise and fall of reservoir water in the monitoring area, integrate the above-water and underwater data, and seamlessly connect the data;

[0056] The water area information acquisition module is a multi-beam echo sounder and a storage device. The multi-beam echo sounder transmits wide-sector sound waves to the underwater area, uses a receiving transducer array to receive the sound waves in a narrow beam, and forms an illumination footprint of the underwater terrain through the orthogonality of the transmitting and receiving sector directions. These footprints are properly processed, and a single detection can provide the water depth values ​​of hundreds or even more underwater measured points in a vertical plane perpendicular to the heading, thereby obtaining point cloud data of the underwater terrain and storing it in the storage device;

[0057] The land information acquisition module and the air information acquisition module are a three-dimensional laser scanning radar, a binocular stereo vision camera and a memory. The three-dimensional laser scanning radar is used to obtain the original point cloud data of the monitoring area and store it in the memory. The binocular stereo vision camera is used to obtain the identification point image of the potential landslide geological disaster hazard area and store it in the memory.

[0058] The airspace attitude positioning module is used to obtain the coordinates, posture and speed information of the UAV, the land area attitude positioning module is used to obtain the coordinates, posture and speed information of the inspection robot dog, and the water area attitude positioning module is used to obtain the coordinates, posture and speed information of the unmanned boat;

[0059] The intelligent docking system includes a self-sensing door for information acquisition system equipment to enter and exit, a wireless transmission module, and a wireless charging platform;

[0060] A solar panel is also provided on one side of the wireless charging platform, and the solar panel is connected to the battery;

[0061] The monitoring and early warning system includes a data processing module, a three-dimensional modeling module, and a danger prediction and early warning module;

[0062] The data processing module can perform denoising, inspection and filtering on the point cloud information transmitted from the intelligent docking system. The denoising process can remove the noise information in the data to make the point cloud data more accurate. The denoised information can be inspected to ensure that the point cloud density can meet the subsequent monitoring needs. The data filtering can realize the classification of the point cloud data, which is convenient for the construction of the subsequent model. The processed information is transmitted to the three-dimensional modeling module and the danger prediction and warning module.

[0063] The data processing module can perform rain and fog removal processing on the image information transmitted by the intelligent warehouse dock system to obtain a clear marker point image, and then perform edge detection and ellipse fitting on the marker point image to obtain the three-dimensional coordinates of the marker point in the world coordinate system;

[0064] The three-dimensional modeling module generates a land DEM model and a water DEM model based on the acquired information, and splices the water and land DEM models to obtain an integrated above-water and underwater DEM model;

[0065] The hazard prediction and early warning module establishes a hazard prediction model based on the acquired information, and calculates the probability value of landslide occurrence in the monitored area through the hazard prediction model, and executes an early warning when the probability value of landslide occurrence reaches a set limit.

[0066] In the present invention, the three-dimensional laser scanning radar is the core component of the system. Its working principle is to emit a narrow pulse through a laser generator, and the receiver receives the reflected wave signal after the narrow pulse contacts the target. By calculating the time interval from the pulse emission to the reception of the reflected wave signal, the distance between the laser and the object to be measured is obtained. The principle of calculating the spatial coordinates of the object to be measured is to obtain the spatial coordinates of the laser scanner as (x0, y0, z0) through the airborne differential GPS locator, and then the laser scanner obtains the distance between the laser scanner and the object to be measured as D, thereby calculating the spatial coordinates (x, y, z) of the object to be measured on the ground at this moment, as shown in formula (1):

[0067]

[0068] Where κ is the heading angle, ω is the roll angle, and ψ is the pitch angle.

[0069] In the present invention, the binocular stereo vision camera includes a left CCD camera and a right CCD camera, which are used to obtain images of specific identification points and store them in a storage. Due to the matching relationship between corresponding points on the left and right images, the three-dimensional coordinates of the measuring point in the world coordinate system can be obtained according to the geometric constraints of the two CCD cameras. The equation is as follows:

[0070]

[0071] In the formula, f l and f r are the effective focal lengths of the left and right CCD cameras, (x l ,y l ) is the coordinate of the measuring point in the image plane coordinate system of the left CCD camera, (x r ,y r ) is the coordinate of the measuring point in the image plane coordinate system of the right CCD camera, R and T are the rotation matrix and translation vector from the world coordinate system to the camera coordinate system respectively:

[0072]

[0073]

[0074] Where (r1, r2, r3), (r4, r5, r6) and (r7, r8, r9) are the representations of the basis vectors of the camera coordinate system in the world coordinate system; x ,t y,t z It is numerically equal to the coordinates of the origin of the world coordinate system in the camera coordinate system.

[0075] In the present invention, the wireless transmission module can adopt Bluetooth, WIFI, 5G, ZIGBE, GPRS and the like.

[0076] A method for using the above-mentioned amphibious three-dimensional unmanned multi-platform linkage landslide intelligent patrol system comprises the following steps:

[0077] Step 1: Preliminary preparation

[0078] According to the activity range of the information acquisition system (drones, inspection robot dogs and unmanned boats), the slope is divided into multiple monitoring areas. An intelligent docking system for storage equipment and identification points for binocular stereo vision lenses are built in the central area of ​​the monitoring area. Target points for coordinate alignment are set up in the area where water and land meet. Requirements:

[0079] 1) The monitoring area should completely cover the mountainous area to be monitored;

[0080] 2) The overlap rate of the monitoring area should be greater than 30% and less than 50%;

[0081] 3) The marking point is composed of a reflective plate and a reflective film. The marking point bracket adopts a steel structure and is set up at the measuring point according to the on-site conditions to support the marking point;

[0082] 4) Drones, inspection robot dogs and unmanned boats can share an intelligent docking system or set up separate intelligent docking systems depending on the specific conditions of the monitoring area.

[0083] Step 2: Monitoring Operations

[0084] After the preparation work is completed, start monitoring work, such as Figure 2 As shown, firstly, UAV is used for remote sensing monitoring to inspect whether there is a landslide with large deformation. The low-altitude remote sensing monitoring of the UAV is used to monitor the surface deformation of a small range of slopes and record three-dimensional point cloud data. At the same time, the binocular stereo vision camera carried by the UAV takes pictures of the landmark points in the potential landslide geological disaster risk area, collects the image information of the landmark points, and stores the three-dimensional point cloud data and image information in the storage.

[0085] If there are areas in the monitoring area that drones cannot enter, or the weather conditions are not suitable for drone flights, the inspection robot dog will monitor the area from the intelligent docking system. The robot dog is equipped with a three-dimensional laser scanning radar to record three-dimensional point cloud data; at the same time, the inspection robot dog uses a binocular stereo vision camera to photograph the landmark points in the potential landslide geological disaster risk area, collect image information of the landmark points, and store the three-dimensional point cloud data and image information in the memory.

[0086] If there are slopes that are flooded for a long time or short time (seasonally) in the monitoring area, the intelligent dock system in the nearby waters will send a signal, and the unmanned boat will monitor the area from the intelligent dock system. The unmanned boat is equipped with a multi-beam depth sounder, which uses a transmitting transducer array to transmit sound waves with wide sector coverage to the bottom of the water, and uses a receiving transducer array to receive the sound waves in a narrow beam. Through the orthogonality of the transmitting and receiving sector pointing, an illumination footprint of the underwater slope is formed. These footprints are properly processed, and a single detection can give the water depth values ​​of hundreds or even more underwater measured points in a vertical plane perpendicular to the heading, and the water depth values ​​of the measured points are stored in a memory.

[0087] After the inspection of the monitoring area is completed, the drone, inspection robot dog and unmanned boat return to the intelligent docking system, and transmit the monitoring information to the monitoring and early warning system through the wireless transmission module of the intelligent docking system.

[0088] Step 3: Image data processing

[0089] 3.1 Image Deraining and Defogging

[0090] The images collected by the binocular stereo vision camera may contain interference factors such as rain marks and rain fog. The image processing module of the monitoring and early warning system processes the images to a certain extent through a dark channel prior defogging method and a morphological component analysis (MCA) image decomposition deraining method to highlight image details, enhance contrast, make it look clearer, and obtain a clear landmark point image.

[0091] 3.2 Edge Detection

[0092] The collected marker point images are filtered, denoised and binarized to obtain the circular marker point target parameters. The Canny algorithm is then used to roughly locate the edge points, and the Zernike moment algorithm is used to perform sub-pixel positioning of the edge to improve the positioning accuracy.

[0093] 3.3 Ellipse fitting

[0094] The circular mark appears as an ellipse on the camera imaging plane, and the present invention selects the ellipse fitting method for positioning. The ellipse equation is fitted using the least square method, and the ellipse is fitted according to the obtained image sub-pixel edge to locate the coordinates of the circle center.

[0095] The equation of the ellipse is:

[0096] ax 2 +bxy+cy 2 +dx+ey+f=0 (5)

[0097] Where a, b, c, d, e, and f are the parameters of the fitted ellipse.

[0098] Convert the general form of an ellipse to matrix form:

[0099] f(u,v)=u*v=1 (6)

[0100] In the formula, u = [abcde 1], v = [x 2 xy y 2 xy 1]T, introducing the constraint condition ||u||=1 in the above problem, the objective function can be established:

[0101]

[0102] Where M is the penalty factor.

[0103] The optimal solution can be obtained by obtaining the coordinates of the center of the ellipse (x c ,y c ), the three-dimensional coordinates of the marker points in the world coordinate system can be obtained. By recording the three-dimensional coordinates of each marker point in different time periods, the deformation of the landslide body can be measured.

[0104]

[0105] Step 4: Point cloud data processing

[0106] The monitoring and early warning system performs data denoising, data verification and data filtering on the original point cloud data.

[0107] 4.1 Data Denoising

[0108] Denoising point cloud data is one of the necessary steps before using point cloud data. Removing noise information in the data can make the point cloud data more accurate. Noise is usually small amplitude noise and discrete points caused by defects in the instrument itself or interference from external factors. Discrete points usually have no obvious rules and appear to be disordered. The spatial topological relationship between the data can be established and the K-nearest neighbor method can be used for denoising. The present invention selects a Gaussian function based on kernel density to calculate the relationship between a point and its neighboring points, and screens point cloud data with low density and relatively discreteness as data noise. The influence of each point in the neighborhood data is expressed mathematically, and the function is the influence function, as shown in formula (9). Suppose the observation point P i The density function of can be defined as the sum of the influence functions of all K-neighboring points of the point. By selecting the Gaussian influence function, the point P is calculated. i The influence of the surrounding nearby points.

[0109]

[0110] Where (x, y) is the point cloud coordinate and σ is the density parameter.

[0111] Kernel density function D P(P i ) is shown in formula (10):

[0112]

[0113] Where k is P i The number of all K-nearest neighbor data points.

[0114] When the density value of an undetermined point is less than a preset threshold, it will be determined as a noise point.

[0115] 4.2 Data Verification

[0116] Data verification is to check the density of point cloud data. The greater the density of point cloud data, the better the ability to express micro-topography. Point cloud data is used for terrain analysis, mainly to help produce DEM. In theory, the greater the density of point cloud data, the higher the accuracy and quality of the generated DEM product. Conversely, the quality of DEM data produced by low-density point cloud data is relatively poor. For areas with greater terrain undulations and more complex objects, the required point cloud data density is higher. The present invention sets different density limits according to different monitoring areas to ensure that the point cloud data can meet subsequent monitoring requirements.

[0117] 4.3 Data Filtering

[0118] Data filtering can realize the classification of point cloud data, dividing the point cloud data into two categories: ground points and non-ground points, and then using the interpolation method to generate the DEM products required for monitoring from the classified ground point data. Nowadays, many point cloud filtering algorithms have been studied. The present invention selects different filtering algorithms according to different monitoring environments to optimize the monitoring effect and provide accurate data support for subsequent production requirements.

[0119] Step 5: 3D modeling

[0120] The 3D modeling module directly observes and analyzes the elevation information of the monitoring area by constructing a digital elevation model. These discrete ground point cloud data are used to generate land DEM models and water DEM models, and then the water and land DEM models are spliced ​​through coordinate registration. Coordinate registration requires setting target points in the adjacent areas of water and land, so that there are more than three target points with the same name in the adjacent area scanning points. Through the forced attachment of target points, the adjacent scanning data are unified together to obtain an integrated DEM model above and below the water.

[0121] Step 6: Hazard warning analysis

[0122] Based on the coordinates of the landmark points, combined with the integrated underwater and above-water DEM model, the three-dimensional modeling of the landslide is carried out. The point cloud data of the landslide body obtained from each monitoring period are compared, and the deformation and sliding data of the characteristic points, characteristic lines, and characteristic surfaces on the landslide body are extracted. The multi-source data processing flow is shown in Figure 3.

[0123] The hazard prediction and early warning module establishes a hazard prediction model based on the three-dimensional coordinate information of the landmark points and the integrated above-water and underwater DEM model, and calculates the probability value of landslides in the monitored area through the hazard prediction model. When the probability value of landslides reaches the set limit, an early warning is executed.

Claims

1. A three-dimensional unmanned multi-platform linkage landslide intelligent patrol system for land, water and air, characterized by The patrol system includes an information acquisition system, an intelligent docking system and a monitoring and early warning system, wherein: The information acquisition system is an amphibious platform, including drones, inspection robot dogs and unmanned boats; The drone is equipped with an airspace information acquisition module and an airspace attitude positioning module; The inspection robot dog is equipped with a land information acquisition module and a land posture positioning module; The unmanned boat is equipped with a water area information acquisition module and a water area attitude positioning module; The water area information acquisition module is a multi-beam echo sounder and a storage device. The multi-beam echo sounder transmits wide-sector sound waves to the underwater area, uses a receiving transducer array to perform narrow-beam reception of the sound waves, forms an illumination footprint of the underwater terrain through the orthogonality of the transmitting and receiving sector directions, processes the illumination footprint, obtains point cloud data of the underwater terrain and stores it in the storage device. The land information acquisition module and the air information acquisition module are a three-dimensional laser scanning radar, a binocular stereo vision camera and a memory. The three-dimensional laser scanning radar acquires the original point cloud data of the monitoring area and stores it in the memory. The binocular stereo vision camera acquires the identification point image of the potential landslide geological disaster hazard area and stores it in the memory. The airspace attitude positioning module is used to obtain the coordinates, posture and speed information of the UAV, the land area attitude positioning module is used to obtain the coordinates, posture and speed information of the inspection robot dog, and the water area attitude positioning module is used to obtain the coordinates, posture and speed information of the unmanned boat; The intelligent docking system includes a self-sensing door for information acquisition system equipment to enter and exit, a wireless transmission module, and a wireless charging platform; The monitoring and early warning system includes a data processing module, a three-dimensional modeling module, and a danger prediction and early warning module; The data processing module performs denoising, inspection and filtering on the point cloud information transmitted from the intelligent docking system, and transmits the processed information to the three-dimensional modeling module and the danger prediction and warning module. The data processing module performs rain and fog removal processing on the image information transmitted from the intelligent docking system to obtain a clear marker point image, and then performs edge detection and ellipse fitting on the marker point image to obtain the three-dimensional coordinates of the marker point in the world coordinate system; The three-dimensional modeling module generates a land DEM model and a water DEM model based on the acquired information, and splices the water and land DEM models to obtain an integrated above-water and underwater DEM model; The hazard prediction and early warning module establishes a hazard prediction model based on the acquired information, and calculates the probability value of landslide occurrence in the monitored area through the hazard prediction model, and executes an early warning when the probability value of landslide occurrence reaches a set limit.

2. The amphibious, three-dimensional, unmanned, multi-platform linkage landslide intelligent patrol system according to claim 1 is characterized in that The binocular stereo vision camera comprises a left CCD camera and a right CCD camera, which are used to obtain images of specific marking points and store them in a storage.

3. The amphibious, three-dimensional, unmanned, multi-platform linkage landslide intelligent patrol system according to claim 1 is characterized in that The wireless transmission module adopts Bluetooth, WIFI, 5G, ZIGBE or GPRS.

4. The amphibious, three-dimensional, unmanned, multi-platform linkage landslide intelligent patrol system according to claim 1 is characterized in that A solar panel is also provided on one side of the wireless charging platform, and the solar panel is connected to the battery.

5. A method for using the amphibious, three-dimensional, unmanned, multi-platform linkage landslide intelligent patrol system according to any one of claims 1 to 4, characterized in that The method comprises the following steps: Step 1: Preliminary preparation According to the activity range of the information acquisition system, the slope is divided into multiple monitoring areas. An intelligent docking system for storage equipment and identification points for binocular stereo vision lenses are built in the central area of ​​the monitoring area. Target points for coordinate registration are set up in the area where water and land meet. Step 2: Monitoring Operations Step 2.1, use drones for remote sensing monitoring to inspect whether there are landslides with large deformations. The drone low-altitude remote sensing monitoring is used for small-scale slope surface deformation monitoring and records three-dimensional point cloud data. At the same time, the binocular stereo vision camera carried by the drone takes pictures of the landmark points in the potential landslide geological disaster risk area, collects the image information of the landmark points, and stores the three-dimensional point cloud data and image information in the storage; Step 2.2, if there is an area in the monitoring area that the drone cannot enter, or the weather conditions are not suitable for drone flight, the inspection robot dog will monitor the area from the intelligent warehouse dock system. The robot dog is equipped with a 3D laser scanning radar to record 3D point cloud data; at the same time, the inspection robot dog uses a binocular stereo vision camera to shoot the landmark points in the potential landslide geological disaster risk area, collect image information of the landmark points, and store the 3D point cloud data and image information in the memory; Step 2.3, if there is a slope that is submerged in water for a long time or a short time in the monitoring area, the intelligent dock system in the adjacent water area sends a signal, and the unmanned boat starts from the intelligent dock system to monitor the area. The unmanned boat is equipped with a multi-beam depth sounder, and uses the transmitting transducer array to transmit wide-sector sound waves to the bottom of the water, and uses the receiving transducer array to receive the sound waves in a narrow beam. Through the orthogonality of the transmitting and receiving sector directions, an illumination footprint of the bottom slope is formed, and these footprints are processed. One detection can give the water depth values ​​of hundreds or even more underwater measured points in the vertical plane perpendicular to the heading, and the water depth values ​​of the measured points are stored in the memory; Step 2.4: After the inspection of the monitoring area is completed, the drone, inspection robot dog and unmanned boat return to the intelligent docking system, and transmit the monitoring information to the monitoring and early warning system through the wireless transmission module of the intelligent docking system; Step 3: Image data processing Step 3.1, the image processing module of the monitoring and early warning system processes the image collected by the binocular stereo vision camera through a dark channel prior defogging method and a morphological component analysis image decomposition deraining method to obtain a clear landmark point image; Step 3.2: Filter, denoise and binarize the marker point image collected by the binocular stereo vision camera to obtain the circular marker point target parameters, then use the Canny algorithm to roughly locate the edge points, and use the Zernike moment algorithm to perform sub-pixel positioning of the edge to improve the positioning accuracy; Step 3.3, using the least square method to fit the ellipse equation, perform ellipse fitting based on the obtained image sub-pixel edge, so as to locate the coordinates of the circle center; Step 4: Point cloud data processing The monitoring and early warning system performs data denoising, data verification and data filtering on the original point cloud data; Step 5: 3D modeling The 3D modeling module directly observes and analyzes the elevation information of the monitoring area by constructing a digital elevation model, generates a land DEM model and a water DEM model using ground point cloud data, and then splices the water and land DEM models through coordinate registration to obtain an integrated DEM model above and below the water; Step 6: Hazard warning analysis Step 6.1, based on the coordinates of the landmark points, combined with the integrated underwater and above-water DEM model, three-dimensional modeling of the landslide is performed, the point cloud data of the landslide body obtained from each monitoring period is compared, and the deformation and sliding data of the characteristic points, characteristic lines, and characteristic surfaces on the landslide body are extracted; Step 6.2: The hazard prediction and warning module establishes a hazard prediction model based on the three-dimensional coordinate information of the landmark points and the integrated above-water and underwater DEM model, and calculates the probability value of landslides in the monitored area through the hazard prediction model. When the probability value of landslides reaches the set limit, an early warning is executed.

6. The method for using the amphibious, three-dimensional, unmanned, multi-platform linkage landslide intelligent patrol system according to claim 5 is characterized in that The monitoring area should completely cover the mountainous area to be monitored.

7. The method for using the amphibious, three-dimensional, unmanned, multi-platform linkage landslide intelligent patrol system according to claim 5 or 6 is characterized in that The overlap rate of the monitoring areas should be greater than 30% and less than 50%.

8. The method for using the amphibious, three-dimensional, unmanned, multi-platform linkage landslide intelligent patrol system according to claim 5 is characterized in that The marking point is composed of a reflective plate and a reflective film. The marking point bracket adopts a steel structure and is erected at the measuring point according to the on-site conditions to support the marking point.

9. The method for using the amphibious, three-dimensional, unmanned, multi-platform linkage landslide intelligent patrol system according to claim 5 is characterized in that The drone, inspection robot dog and unmanned boat can share an intelligent docking system according to the specific conditions of the monitoring area, or they can be provided with separate intelligent docking systems.

Images