Method and device for emergency monitoring of reservoir area landslide based on side-looking camera of unmanned aerial vehicle

Abstract

The present invention provides a method and device for emergency monitoring of reservoir area landslides based on a side-view camera of an unmanned aerial vehicle, which are applied to the field of remote sensing technology. The method includes: obtaining at least one exposure image captured by a side-view camera group in a fixed-wing unmanned aerial vehicle; performing first-level landslide identification on the central camera image of the at least one exposure image through a plurality of preset models to obtain a candidate landslide image group and a list of candidate exposure positions; constructing a post-temporal digital surface model for each candidate landslide image in the candidate landslide image group based on the at least one exposure image; obtaining a pre-constructed pre-temporal digital surface model corresponding to the list of candidate exposure positions; performing second-level landslide identification based on the post-temporal digital surface model and the pre-temporal digital surface model to obtain a target landslide area; determining the three-dimensional spatial volume change of the ground objects in the target landslide area as the landslide volume of the target landslide area; through the present invention, the data acquisition efficiency and the landslide identification efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing technology, and particularly to a method and device for emergency monitoring of reservoir area landslides based on an unmanned aerial vehicle (UAV) side-looking camera. Background Art

[0002] In the field of remote sensing, for high-precision landslide identification and landslide volume estimation, three-dimensional data products of two time phases before and after are required. Data acquisition is a crucial prerequisite and the top priority, and the production of three-dimensional data products has relatively high requirements for data acquisition.

[0003] The satellite SAR (Synthetic Aperture Radar) data used should be able to form an interference, and the optical data should be able to form a stereo pair. However, the spatial resolution of satellite lidar data needs to be greatly improved. Obtaining data through manned aircraft aerial remote sensing can easily obtain coherent data or stereo pairs, but the data acquisition cost is high.

[0004] Therefore, whether it is satellite remote sensing or manned aircraft aerial remote sensing, when constructing corresponding three-dimensional data products to achieve rapid landslide identification, the implementation difficulty is relatively large, the implementation period has uncontrollable risks, and the cost is relatively high. Summary of the Invention

[0005] The present invention provides a method and device for emergency monitoring of reservoir area landslides based on an unmanned aerial vehicle (UAV) side-looking camera, which is used to solve the defect of difficult three-dimensional data acquisition in the existing landslide identification method, and can improve the data acquisition efficiency and landslide identification efficiency.

[0006] The present invention provides a method for emergency monitoring of reservoir area landslides based on an unmanned aerial vehicle (UAV) side-looking camera, including the following steps.

[0007] Obtain at least one exposure image captured by a side-looking camera group in a fixed-wing unmanned aerial vehicle (UAV), where the side-looking camera group includes multiple stitching cameras with zero inclination; based on the established texture features, perform first-level landslide identification on the central camera image of the at least one exposure image through multiple preset models to obtain a candidate landslide image group and a candidate exposure position list, where the central camera image is captured by the central position camera of the side-looking camera group; based on the at least one exposure image, construct a post-temporal digital surface model for each candidate landslide image in the candidate landslide image group; obtain a pre-temporal digital surface model constructed in advance corresponding to the candidate exposure position list; perform second-level landslide identification based on the post-temporal digital surface model and the pre-temporal digital surface model to obtain a target landslide area, where the elevation difference value between the post-temporal digital surface model and the pre-temporal digital surface model of the target landslide area is greater than an elevation difference threshold; determine the three-dimensional spatial volume change of the ground objects in the target landslide area as the landslide volume of the target landslide area.

[0008] According to a method for emergency monitoring of reservoir area landslides based on a UAV side-looking camera provided by the present invention, the obtaining at least one exposure image captured by a side-looking camera group in a fixed-wing UAV includes: calling the fixed-wing UAV to fly a single route, and using the side-looking camera group located on the side of the fuselage of the fixed-wing UAV to collect ground object images of a target location when the fixed-wing UAV is at a preset target exposure point to obtain at least one exposure image; where the side-looking camera group includes a target number of fixed-position cameras, and the lens spacing between the target number of fixed-position cameras is less than a preset lens spacing threshold.

[0009] According to a method for emergency monitoring of reservoir area landslides based on a UAV side-looking camera provided by the present invention, the target exposure points include a front exposure point, a middle exposure point, and a rear exposure point. The constructing a post-temporal digital surface model for each candidate landslide image in the candidate landslide image group based on the at least one exposure image includes: determining a first exposure image when the fixed-wing UAV is at the front exposure point, a second exposure image when the fixed-wing UAV is at the middle exposure point, and a third exposure image when the fixed-wing UAV is at the rear exposure point; determining the pixels for the target location in the first exposure image, the second exposure image, and the third exposure image as the overlapping coverage area; constructing a post-temporal digital surface model for each candidate landslide image in the candidate landslide image group based on the overlapping coverage area.

[0010] A method for emergency monitoring of reservoir area landslides based on an unmanned aerial vehicle side-looking camera. After constructing the post-temporal digital surface model of each candidate landslide image in the candidate landslide image group based on the at least one exposure image, the method further includes: when there is no pre-constructed pre-temporal digital surface model in the candidate exposure position list, determining the candidate landslide area boundary of the candidate landslide image based on the candidate exposure position list to obtain a candidate landslide area; acquiring a target post-temporal digital surface model corresponding to the candidate landslide area; gridifying the candidate landslide area based on the grid size of the target post-temporal digital surface model to obtain a grid landslide area; simulating the pre-temporal digital surface model of the candidate landslide area based on the simulated elevation values of each row of grids in the grid landslide area; simulating the current temporal digital surface model of the candidate landslide area based on the average elevation value of each grid in the grid landslide area according to the boundary growth method; performing a comparative analysis based on the pre-temporal digital surface model and the current temporal digital surface model to obtain the elevation change difference of the candidate landslide area; when the elevation change difference of the candidate landslide area is greater than the elevation change threshold, taking the candidate landslide area as the target landslide area.

[0011] A method for emergency monitoring of reservoir area landslides based on an unmanned aerial vehicle side-looking camera. Before simulating the pre-temporal digital surface model of the candidate landslide area based on the simulated elevation values of each row of grids in the grid landslide area, the method further includes: acquiring the upper grid elevation, lower grid elevation, first grid elevation, and last grid elevation of each row of grids in the grid landslide area based on the target post-temporal digital surface model; taking the average value among the upper grid elevation, lower grid elevation, first grid elevation, and last grid elevation as the simulated elevation value of each row of grids in the grid landslide area.

[0012] A method for emergency monitoring of reservoir area landslides based on an unmanned aerial vehicle side-looking camera. Before simulating the current temporal digital surface model of the candidate landslide area based on the average elevation value of each grid in the grid landslide area according to the boundary growth method, the method further includes: taking each grid in the grid landslide area as a grid to be estimated, and performing the following steps to obtain the average elevation value of the grid to be estimated: acquiring the grid elevation values of the four adjacent areas of the grid to be estimated based on the target post-temporal digital surface model; determining the average elevation value of the grid to be estimated based on the grid elevation values of the four adjacent areas of the grid to be estimated:

[0013]

[0014] wherein, represents the average elevation value of the grid to be estimated, represents the elevation value of the grid directly to the right of the grid to be estimated, represents the elevation value of the grid directly to the left of the grid to be estimated, represents the elevation value of the grid directly above the grid to be estimated, represents the elevation value of the grid directly below the grid to be estimated, represents the number of grids participating in the calculation.

[0015] The present invention also provides a reservoir area landslide emergency monitoring device based on a drone side-looking camera, including the following modules: an acquisition module, configured to acquire at least one exposure image captured by a side-looking camera group in a fixed-wing drone, wherein the side-looking camera group includes a plurality of stitching cameras with zero inclination; a first recognition module, configured to perform a first-level landslide recognition on the central camera image of the at least one exposure image based on a plurality of preset models according to the established texture features, to obtain a candidate landslide image group and a candidate exposure position list, wherein the central camera image is captured by a camera at the central position of the side-looking camera group; a construction module, configured to construct a post-temporal digital surface model of each candidate landslide image in the candidate landslide image group based on the at least one exposure image; the acquisition module is further configured to acquire a pre-constructed pre-temporal digital surface model corresponding to the candidate exposure position list; a second recognition module, configured to perform a second-level landslide recognition based on the post-temporal digital surface model and the pre-temporal digital surface model, to obtain a target landslide area, wherein the elevation difference value between the post-temporal digital surface model and the pre-temporal digital surface model of the target landslide area is greater than an elevation difference threshold; a determination module, configured to determine the three-dimensional spatial volume change of the ground objects in the target landslide area as the landslide volume of the target landslide area.

[0016] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, it implements the method for reservoir area landslide emergency monitoring based on a drone side-looking camera as described in any one of the above.

[0017] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the method for reservoir area landslide emergency monitoring based on a drone side-looking camera as described in any one of the above.

[0018] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the method for reservoir area landslide emergency monitoring based on a drone side-looking camera as described in any one of the above.

[0019] The method and device for emergency monitoring of reservoir area landslides based on a drone side-view camera provided by the present invention obtain at least one exposure image through a side-view camera group carried by a fixed-wing drone. Among the side-view camera group, multiple zero-inclination stitching cameras can increase the number of homologous points of the same ground object that can be obtained by a single exposure point without increasing the flight workload, thereby improving the point cloud density of the subsequent digital surface model and enhancing the ability to depict the terrain. Using the established texture features and multiple preset models, only the images obtained by the central position camera are used for the first-level landslide identification, greatly reducing the single and total input data volume of the data to be detected. Multiple models run in parallel, which not only improves the recognition accuracy but also reduces the omission rate of recognition. Based on at least one exposure image, constructing a post-temporal digital surface model for each candidate landslide image in the candidate landslide image group can provide detailed terrain information of the suspected landslide area. Based on the post-temporal digital surface model and the pre-temporal digital surface model, the second-level landslide identification is carried out to screen out the target landslide areas with elevation difference values greater than the threshold, which can exclude misidentified areas and improve the accuracy and reliability of landslide identification. By calculating the three-dimensional spatial volume change of the ground objects in the target landslide area to determine the landslide volume, the scale of the landslide can be quantified, and efficient and accurate landslide identification is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art one by one. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 It is a schematic flowchart of the method for emergency monitoring of reservoir area landslides based on a drone side-view camera provided by the present invention.

[0022] Figure 2 It is a schematic overall flowchart of the method for emergency monitoring of reservoir area landslides based on a drone side-view camera provided by the present invention.

[0023] Figure 3 It is a schematic diagram of the installation of the five-stitch camera of the drone provided by the present invention and the data acquisition overlap effect diagram obtained thereby.

[0024] Figure 4 It is a schematic diagram of the homologous point area of three adjacent exposure points provided by the present invention.

[0025] Figure 5 It is a schematic diagram of the installation of the right side-view fixed-wing drone and the five-stitch camera provided by the present invention.

[0026] Figure 6 It is a schematic diagram of the data acquisition of the fixed-wing drone side-view system provided by the present invention.

[0027] Figure 7 It is a schematic diagram of the grid of candidate landslide areas provided by the present invention.

[0028] Figure 8 It is a schematic structural diagram of the emergency monitoring device for reservoir landslides based on the side-looking camera of an unmanned aerial vehicle provided by the present invention.

[0029] Figure 9 It is a schematic physical structure diagram of the electronic device provided by the present invention. Detailed implementation manners

[0030] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0031] Landslide disasters are one of the natural disasters that cause significant economic losses second only to earthquakes. When the water level in the reservoir area of a hydropower station rises and falls due to water storage and flood discharge, it will directly affect the stability of the reservoir banks upstream and downstream of the reservoir area, and thus pose potential safety hazards to the cruise ships, dam bodies and power generation facilities in the reservoir area. Therefore, carrying out disaster body identification and key landslide monitoring on the reservoir banks within a certain range upstream and downstream of the hydropower station is an important part of the emergency monitoring of geological disasters in the reservoir area. In recent years, many detection and monitoring technologies have been widely applied in the field of landslide research, covering many aspects such as landslide area investigation, landslide displacement monitoring, landslide disaster assessment and landslide hazard assessment. Among them, the investigation and monitoring of landslides are the basic links of the entire research work.

[0032] At present, the methods for emergency monitoring of geological disasters such as reservoir landslides include ground monitoring methods based on the Global Navigation Satellite System (GNSS) and remote sensing monitoring methods, etc.

[0033] The GNSS automatic deformation observation method is to set up GNSS deformation monitoring stations at key positions such as the mountain body in the reservoir area and a reference station outside the landslide body. The monitoring station is remotely controlled by wireless transmission or through a computer in the Internet management center to monitor the real-time deformation conditions of the mountain body in the reservoir area, so as to continuously monitor the landslide body for 24 hours. This method has a high time resolution of monitoring, good effects and strong reliability. However, its disadvantages are also relatively obvious, that is, limited by the construction cost of the deformation station, it cannot be arranged in a large area in large quantities, and the monitoring grid size is limited.

[0034] Due to its characteristics such as large monitoring range, less restricted by ground conditions, all-day and all-weather operation, high precision, and high resolution, remote sensing is one of the important technical means for rapid landslide monitoring at present. The basic principle of remote sensing landslide identification is to obtain changes (deformations) in the three-dimensional space of the same area through data of two different time phases, which are used as the identification results of suspected landslides. It mainly includes methods based on SAR (Synthetic Aperture Radar), optical remote sensing, lidar, and the combination of multiple remote sensing means.

[0035] The method based on SAR mainly calculates (obtains) the SAR interferogram. Differential interferometry can be carried out using the data of a digital elevation model (DEM, Digital Elevation Model) external to a single-phase SAR, or directly using interferometric radar data to obtain the differential interferometry results (deformation map, intensity map, coherence map, and layover shadow map), which are used as the basis for identifying and judging landslides.

[0036] The main idea of the method based on optical remote sensing is to obtain high-overlap images of the target area, and generate a three-dimensional numerical model of the area (such as a digital surface model product (DSM, Digital Surface Model) and a digital elevation model) through photogrammetry methods. By comparing the three-dimensional model products of the two different time phases, landslides in the changed area can be identified.

[0037] The method based on lidar is different from the method of generating a three-dimensional numerical model of the area through photogrammetry in optical remote sensing. Instead, it directly obtains three-dimensional point cloud data of the target area through sensors and generates a three-dimensional numerical model product. By comparing this product with the three-dimensional model product of the previous time phase, landslides in the changed area can be identified.

[0038] The remote sensing monitoring method combining multiple means mainly integrates the advantages of different technical methods and makes up for their disadvantages to improve the monitoring efficiency and accuracy. For example, by combining lidar, optical remote sensing, and drones, using drones equipped with LiDAR (Light Detection and Ranging) systems and optical lenses, combined with PPP-RTK technology (a combination of precise point positioning and real-time kinematic positioning), high-precision three-dimensional terrain data can be obtained through the tilt photography post-processing method and compared with data of other time phases to identify suspected landslide areas.

[0039] With the development of technology, in addition to the above method of identifying suspected landslide areas relying on three-dimensional information, methods have also been developed to directly use two-dimensional remote sensing images without constructing three-dimensional data product information, and comprehensively use multi-feature information such as spectrum, shape, and texture, and conduct landslide identification through spatial statistics, analysis techniques, deep learning techniques, etc.

[0040] As can be seen from the above analysis, one of the keys to using remote sensing methods for landslide monitoring is to timely obtain specific types of remote sensing data for the target area. Such data can be acquired through satellite remote sensing, manned aircraft, and unmanned aerial vehicle (UAV) aerial remote sensing. Satellite remote sensing data has stable quality and a large acquisition coverage area, but it is greatly affected by local weather conditions during acquisition, especially optical images. The aerial remote sensing method is relatively more flexible than satellite remote sensing. Especially for UAVs, due to their flexible mobility and low usage cost, they have obvious advantages in obtaining remote sensing data for small areas and are suitable for landslide monitoring in a certain area before and after the dam in the reservoir area of hydropower stations.

[0041] To improve the acquisition efficiency of aerial remote sensing data, one feasible method is to use a multi-stitching camera. Through the external field-of-view angle stitching technology, a virtual stitched image can be obtained, expanding the ground coverage area corresponding to the image obtained in one exposure, thereby improving the aerial photography efficiency.

[0042] In the field of remote sensing, for high-precision landslide identification and estimating landslide volume, data acquisition is a crucial prerequisite and the top priority. When producing such products, there are relatively high requirements for data acquisition. The satellite SAR data used should be able to form an interferogram (InSAR), the optical data should be able to form a stereo pair, and the spatial resolution of satellite lidar data needs to be significantly improved. It is easy to obtain coherent data or stereo pairs through manned aircraft aerial remote sensing for data acquisition, but the data acquisition cost is high. Therefore, whether it is satellite remote sensing or manned aircraft aerial remote sensing, especially when it comes to realizing rapid landslide identification and efficient early warning in the reservoir area by constructing corresponding three-dimensional data products for two different times before and after, the implementation difficulty is relatively large, the implementation cycle has uncontrollable risks, and the cost is relatively high.

[0043] Compared with satellite remote sensing and manned aircraft aerial remote sensing, using UAVs to conduct landslide monitoring in small areas is a feasible technical solution with low cost. However, for the conventional UAV flight mode (heading overlap and side overlap), due to the limited image overlap, the point cloud density of the produced digital surface model is not sufficient to finely depict the surface topography. It is necessary to increase the image overlap by encrypting the flight line and increase the number of homologous image pairs to improve the point cloud density of the digital surface model. However, such a design will significantly reduce the data acquisition efficiency. And carrying lidar and other devices that can directly obtain three-dimensional data increases the requirements for the performance and reliability of the UAV itself, and the flight risk and cost increase significantly. Moreover, the UAVs suitable for installing this type of equipment (mostly multi-rotor aircraft) are often unable to carry out relatively large-scale and long-distance data acquisition tasks within one flight compared to fixed-wing UAVs.

[0044] In addition, landslides may cause great harm to hydropower station dams. In application, not only should we focus on improving the efficiency and ability to detect landslides, but more importantly, reduce the omission rate of landslide identification.

[0045] In the embodiments of the present invention, in view of these problems and the requirements of emergency monitoring of landslide geological disasters in the reservoir area of hydropower stations, taking into account both efficiency and effect, a fixed-wing UAV five-piece camera side-looking system is proposed to solve the problem of data acquisition. Without adding extra flights, the present invention enables the UAV to obtain multiple pairs of homologous images of the same target during a single flight, meeting the production of a digital surface model with a high point cloud density under the same acquisition conditions; using a hierarchical data processing technical solution to compress the data processing volume, and realizing the identification of suspected landslide areas under the parallel operation of multiple methods based on two-dimensional images, improving the identification effect and reducing the identification omission rate; separately completing the production of the digital surface model for the area (image group) where the suspected landslide is located, reducing the production cycle of the digital surface model of a large amount of data, and improving the time efficiency of early warning; using the boundary addition method to solve the problem of estimating the landslide volume in the case of no previous-phase digital surface model; finally, realizing the rapid identification of landslides along the reservoir area shore and the estimation of the landslide volume during a single flight along a single route, and giving an early warning based on the identification results.

[0046] The present invention solves the problems of three-dimensional data acquisition required for identification and hierarchical rapid identification through specific hardware devices and a hierarchical concept, reduces the cost of emergency monitoring of landslides in the reservoir area of hydropower stations, improves the identification efficiency, reduces the identification omission rate, and enhances the ability of rapid landslide early warning.

[0047] Optionally, the method for emergency monitoring of reservoir area landslides based on the UAV side-looking camera in the embodiments of the present application can be executed by a server, or by a terminal device, or jointly executed by a server and a terminal device. Taking the execution of the method for emergency monitoring of reservoir area landslides based on the UAV side-looking camera in this embodiment by the terminal device as an example.

[0048] Figure 1 is a schematic flow chart of the method for emergency monitoring of reservoir area landslides based on the UAV side-looking camera provided by the present invention, as Figure 1 shown, the method includes the following:

[0049] Step 101, obtain at least one exposure image captured by the side-looking camera group in the fixed-wing UAV.

[0050] Among them, the side-looking camera group includes multiple stitching cameras with zero inclination angles;

[0051] The present invention uses a different idea from existing aerial remote sensing multi-piece cameras to increase the coverage area of a single exposure point image. Instead, by adopting multiple stitching cameras with zero inclination angles and a large overlap degree, without increasing the flight workload of the UAV, the number of homologous points of the same ground object that can be obtained by a single exposure point is increased, thereby improving the point cloud density of subsequent digital surface model products and enhancing the ability to depict the terrain.

[0052] Fixed-wing UAVs have the characteristics of fast flight speed and wide coverage, and are suitable for image acquisition tasks in large-scale areas; fixed-wing UAVs are equipped with a side-looking camera group, which can take pictures of the target area from a side-looking angle during flight to obtain multi-perspective image data.

[0053] The side-looking camera group consists of multiple stitching cameras with zero inclination. Zero inclination means that the optical axis of the camera is parallel to the horizontal plane, ensuring the consistency of the images taken in the vertical direction. Multiple cameras are arranged in a stitching manner, which can simultaneously obtain multi-angle images of the target area, increasing the coverage and resolution of the images.

[0054] It should be noted that through the zero-inclination and high-overlap multi-stitching camera mode, the images obtained by one exposure can meet the requirements for producing DSM data with a high point cloud density.

[0055] Through the side-looking observation method of the camera, the ability to collect the texture information on the side of the target can be improved to meet the requirements for obtaining geological disaster information on the side of the reservoir bank, mountain body, and dam body.

[0056] Step 102, based on the established texture features, perform first-level landslide identification on the central camera image of at least one exposure image through multiple preset models to obtain a candidate landslide image group and a candidate exposure position list.

[0057] Among them, the central camera image is taken by the camera at the central position of the side-looking camera group.

[0058] In the embodiment of the present invention, landslide identification adopts a hierarchical mode (including first-level landslide identification and second-level landslide identification), uses image recognition or target recognition algorithms, and only performs landslide identification on the images obtained by the central position camera, greatly reducing the single and total input data volume of the data to be detected. Multiple models run in parallel, which not only improves the recognition accuracy rate but also reduces the recognition omission rate.

[0059] Applying the UAV remote sensing method, the main method for high-precision landslide identification is to use the comparison and analysis of three-dimensional data (DSM / DEM) in two periods before and after. However, directly carrying out DSM product production on the obtained UAV images is costly in terms of time and manpower, and most of the time it is also unnecessary.

[0060] Therefore, in the embodiment of the present invention, the first-level landslide identification directly targets the texture features of at least one exposure image (landslide image) collected, and uses methods such as deep learning and maximum likelihood models to carry out the extraction of suspected landslides.

[0061] First, prepare the input data. Only call the images captured by the central camera. The images are not processed for color or tone, nor are geometric correction, distortion correction, or mosaicking performed. Directly input at least two preset models by scene (image). Based on the established texture features, identify potential landslides, and extract and record the candidate landslide images and candidate exposure positions (lists) corresponding to the potential landslide areas.

[0062] Secondly, merge the potential landslide areas identified by multiple methods, and establish a group of candidate landslide images where the potential landslide areas are located and their corresponding list of candidate exposure positions, which serves as the basis for data screening in secondary landslide identification.

[0063] The accuracy of potential landslides identified by the above methods is very low, and there are a large number of misjudgments. However, using this method, only a single image is input each time, the data volume is small, the data analysis pressure is small, and the saved time can be used for the method of parallel repeated detection by multiple models to identify potential landslides simultaneously, thereby greatly reducing the omission rate of identification.

[0064] In some embodiments, obtain the texture features of the pre-established landslide areas as the basis for subsequent identification. Texture features are important bases for landslide identification and usually include information such as color, shape, edge, and roughness in the images.

[0065] The images captured by the central camera in the side-looking camera group are used as the core data source. Because its viewing angle is centered, it can cover the main part of the target area, and the image quality is relatively stable; the resolution and coverage range of the central camera images are moderate, making them suitable as the basic data for the first-level landslide identification.

[0066] Based on the established texture features, multiple preset models (such as convolutional neural networks, support vector machines, random forests, etc.) analyze the central camera images pixel by pixel or region by region to identify areas where landslides may occur. The identification results include a group of candidate landslide images (i.e., image segments of potential landslides) and their corresponding list of candidate exposure positions (i.e., the geographical locations of potential landslides).

[0067] Through the embodiments of the present invention, the first-level landslide identification quickly screens out areas where landslides may occur through multiple preset models, greatly reducing the data volume for subsequent processing and improving the overall identification efficiency.

[0068] Step 103, based on at least one exposure image, construct a post-temporal digital surface model for each candidate landslide image in the group of candidate landslide images.

[0069] In the embodiments of the present invention, at least one exposure image provides image data of the target area at different times or under different lighting conditions, which can more comprehensively reflect the characteristics of the ground surface.

[0070] For the candidate landslide image group obtained from the first-level landslide recognition, each candidate landslide image is processed one by one, and the three-dimensional point cloud data of the ground objects is extracted from at least one exposure image using photogrammetry techniques (such as stereo matching, dense matching, etc.).

[0071] Based on the extracted three-dimensional point cloud data, a post-temporal digital surface model (DSM) is generated through interpolation or fitting algorithms (such as triangulation interpolation, Kriging interpolation, etc.). The post-temporal digital surface model reflects the surface elevation and terrain features after the target time (for example, after the landslide occurs).

[0072] Through the embodiments of the present invention, a high-precision post-temporal digital surface model can be constructed through at least one exposure image, accurately reflecting the surface changes after the landslide occurs. Only the images in the candidate landslide image group are processed, reducing the computational amount and improving the efficiency of model construction.

[0073] Step 104: Obtain the pre-constructed pre-temporal digital surface model corresponding to the candidate exposure position list.

[0074] In the embodiments of the present invention, the candidate exposure position list is obtained from the first-level landslide recognition and records the geographical location information of the suspected landslide area. According to the coordinate range of the candidate exposure position list, the corresponding pre-temporal digital surface model is retrieved and extracted from the existing digital surface model database.

[0075] The pre-temporal digital surface model is pre-constructed through historical image data (such as images taken by drones, satellites, or other remote sensing devices), reflecting the surface elevation and terrain features at a specific time (before the landslide occurs).

[0076] Step 105: Perform a second-level landslide recognition based on the post-temporal digital surface model and the pre-temporal digital surface model to obtain the target landslide area.

[0077] Among them, the elevation difference value between the post-temporal digital surface model and the pre-temporal digital surface model of the target landslide area is greater than the elevation difference threshold.

[0078] In the embodiments of the present invention, first, according to the first-level suspected landslide recognition result, an image group containing suspected landslides (each candidate landslide image in the candidate landslide image group) is extracted, and only the images in the image group are used to generate a post-temporal digital surface model based on the photogrammetry method.

[0079] Secondly, a quantitative comparison and analysis is performed with the pre-temporal DSM product. If an abnormal area is found, it is identified as a suspected landslide area, and the three-dimensional spatial volume change of the ground objects in the suspected landslide area is calculated as the suspected landslide volume.

[0080] Finally, a result set (list) of suspected landslide results to be verified is established. Using this method, the number of images to be processed input each time is greatly reduced, the system overhead is small, and it is easy to complete the processing quickly.

[0081] In some embodiments, the post-temporal digital surface model (reflecting the surface state after the landslide occurred) is compared with the pre-temporal digital surface model (reflecting the surface state before the landslide occurred) pixel by pixel or region by region. The comparison content includes changes in elevation values, terrain deformation, etc.

[0082] Calculate the elevation difference value between the post-temporal digital surface model and the pre-temporal digital surface model, that is, the post-temporal elevation value minus the pre-temporal elevation value. The elevation difference value reflects the change amount of the surface before and after the landslide occurred.

[0083] According to the characteristics of the landslide and the geological conditions of the region, a reasonable elevation difference threshold is set. The elevation difference threshold is used to distinguish normal terrain changes and terrain changes caused by landslides. Mark the regions where the elevation difference value is greater than the elevation difference threshold as the target landslide areas. The target landslide areas are the core areas where the landslide occurred and have significant surface change characteristics.

[0084] Through the embodiments of the present invention, based on the post-temporal digital surface model and the pre-temporal digital surface model for the second-level landslide identification, the target landslide areas can be accurately identified through the calculation of the elevation difference value and threshold screening.

[0085] Step 106, determine the three-dimensional spatial volume change of the target landslide area as the landslide volume of the target landslide area.

[0086] In the embodiments of the present invention, based on the pre-temporal digital surface model and the post-temporal digital surface model of the target landslide area, three-dimensional spatial comparison data of the target landslide area is extracted, where the pre-temporal DSM reflects the surface elevation before the landslide occurred, and the post-temporal DSM reflects the surface elevation after the landslide occurred.

[0087] Based on the three-dimensional spatial comparison data, by comparing the elevation values of the pre-temporal DSM and the post-temporal DSM pixel by pixel, calculate the elevation difference of each pixel or region; multiply the elevation difference value by the area of the pixel or region to obtain the volume change amount of each pixel or region.

[0088] Accumulate the volume change amounts of all pixels or regions within the target landslide area to obtain the three-dimensional spatial volume change of the entire target landslide area; this volume change is the landslide volume of the target landslide area.

[0089] Reference Figure 2 , Figure 2It is a schematic diagram of the overall process of the reservoir area landslide emergency monitoring method provided by the present invention based on a drone side-looking camera. It includes: task startup; drone reservoir area landslide monitoring (cruise mode); the drone carrying a five-piece camera side-looking system to obtain data; importing data into the system after the flight ends; quickly identifying suspected landslide textures based on the data of the central camera; retrieving the data of the exposure points where the suspected landslide textures are located; producing DSM products based on photogrammetry methods; judging whether there is a previous DSM; if not, estimating the DSM through the boundary growth method; if so, conducting quantitative comparative analysis with the previous DSM products; secondary landslide determination; if not, excluding the landslide risk; if so, calculating the landslide volume; uploading the results to the system and starting or terminating the early warning according to regulations; task end.

[0090] First of all, the present invention uses an idea different from that of existing aerial remote sensing multi-piece cameras to increase the coverage area of a single exposure point image. By adopting a five-piece camera mode with a large overlap degree, without increasing the flight workload, the number of homologous points of the same ground object that can be obtained by a single exposure point is increased, thereby improving the point cloud density of the DSM product and enhancing the ability to depict the terrain.

[0091] Secondly, due to the change in the camera splicing method, aiming at the data requirement characteristics of the reservoir area shore, the multi-route flight is changed to a single-route flight, that is, the data required for the production of the DSM three-dimensional product is obtained in one flight. Combining with the route overlap rate (usually set to 60% for flight), dozens of groups of homologous image pairs can be obtained for the same target.

[0092] Thirdly, the acquisition mode of the fixed-wing drone's directly downward image (the observation port is located on the belly of the aircraft) is changed, and an observation port is set up on the side of the fuselage. The five-piece camera obtains the images of the ground objects along the reservoir area shore in a side-looking manner.

[0093] Fourthly, the landslide identification adopts a hierarchical mode. Using image recognition or target recognition algorithms, landslide identification is only carried out on the images obtained by the central camera, which greatly reduces the single and total input data volume of the data to be detected. Multiple models run in parallel, which not only improves the recognition accuracy rate but also reduces the recognition omission rate. For the identified suspected results, the image group of the exposure point where they are located is retrieved, and the DSM product of this point is produced using photogrammetry methods, and quantitative comparative analysis is carried out with the previous DSM products to further determine the landslide area and obtain the landslide volume; because the images in a small area where the landslide is located are used as the input, the quantity of the input DSM production model is small, the workload of completing the DSM production is small, and the time is fast, which improves the time efficiency of the entire recognition process.

[0094] Fifthly, when the system is idle, based on the data obtained by the drone, the DSM product is produced as the reference data for the next-time phase detection.

[0095] Sixth, when there is no DSM product in the previous time phase, all recognition results are submitted for manual interpretation. For the data of the exposure points where suspected landslide areas are interpreted, DSM products are produced, and the DSM of the previous time phase is estimated according to the boundary growth method, and the landslide volume is estimated.

[0096] In the present invention, DSM (Digital Surface Model product) is uniformly used. On the one hand, the DSM product is a three-dimensional product without removing the ground coverage, which can reflect the surface conditions more comprehensively and meticulously than the DEM (Digital Elevation Model) product. On the other hand, additional manpower and time costs are required to produce DEM from DSM. Obviously, in terms of the application scenario, using DSM is more appropriate both in terms of analysis effect and cost.

[0097] Through the above steps of the embodiments of the present invention, at least one exposure image is obtained by the side-looking camera group carried by a fixed-wing unmanned aerial vehicle. Multiple zero-inclination stitching cameras in the side-looking camera group can increase the number of homologous points of the same ground object that can be obtained by a single exposure point without increasing the flight workload, thereby improving the point cloud density of the subsequent digital surface model and enhancing the ability to depict the terrain; using the established texture features and multiple preset models, only the images obtained by the central position camera are used for the first-level landslide recognition, greatly reducing the single and total input data volume of the data to be detected. Multiple models run in parallel, which not only improves the recognition accuracy rate but also reduces the recognition omission rate; based on at least one exposure image, constructing the post-time-phase digital surface model of each candidate landslide image in the candidate landslide image group can provide detailed terrain information of the suspected landslide area; based on the post-time-phase digital surface model and the pre-time-phase digital surface model for the second-level landslide recognition, screening out the target landslide areas with elevation difference values greater than the threshold can exclude misrecognized areas and improve the accuracy and reliability of landslide recognition; by calculating the three-dimensional spatial volume change of the ground objects in the target landslide area to determine the landslide volume, the scale of the landslide can be quantified, and efficient and accurate landslide recognition is achieved.

[0098] According to a method for emergency monitoring of reservoir area landslides based on a side-looking camera of an unmanned aerial vehicle provided by the present invention, at least one exposure image obtained by shooting with a side-looking camera group in a fixed-wing unmanned aerial vehicle is acquired, including:

[0099] Calling a fixed-wing unmanned aerial vehicle to fly a single route, and through a side-looking camera group located on the side of the fuselage of the fixed-wing unmanned aerial vehicle, when the fixed-wing unmanned aerial vehicle is at a preset target exposure point, collecting ground object images of the target location to obtain at least one exposure image;

[0100] Wherein, the side-looking camera group includes a target number of fixed-position cameras, and the lens spacing between the target number of fixed-position cameras is less than a preset lens spacing threshold.

[0101] As mentioned above, when currently producing digital orthophotos in general aerial remote sensing, an optical camera is used to design the flight route according to the specified overlap degrees in the forward (flight direction) and lateral (perpendicular to the flight route direction), such as 60% in the forward direction and 30% in the lateral direction. At this time, it can not only meet the accuracy requirements of the digital orthophoto product, but also achieve a balance between the product accuracy, quality and data acquisition efficiency.

[0102] However, when producing a high-quality digital surface model, a larger proportion of image overlap is required to increase the number of homologous image pairs (to construct stereo image pairs). The common practice is to encrypt the flight route, which brings the problems of a significant increase in data acquisition time and data volume.

[0103] In some embodiments, the landslide occurs on the mountain body. In the reservoir area of the hydropower station, the heights of the mountain (bank) bodies on both sides of the reservoir bank are generally in the range of dozens to hundreds of meters. Therefore, as long as the data obtained by the UAV can reach this coverage ability, that is, the image width is dozens - hundreds of meters.

[0104] In the embodiments of the present invention, in view of this problem and in combination with the characteristics of landslide monitoring in the reservoir area, that is, only focusing on the landslide situation within a certain range of the reservoir bank, the UAV camera is redesigned. Under the condition of meeting the requirements of the single-image width, the original method of increasing the image width by camera splicing is changed to the method of zero-inclination camera splicing, increasing the coverage area of the overlap area during one exposure, achieving an increase in the number of homologous image pairs and realizing the effect of enhancing the point cloud density of the digital surface model.

[0105] Reference Figure 3 , Figure 3 is the installation schematic diagram of the five-spliced camera of the UAV provided by the present invention and the data acquisition overlap effect diagram.

[0106] As Figure 3 shown, the blue frame is the camera installation transition plate, made of carbon steel, with light weight and sufficient mechanical strength. An image window is opened in the center of the plate, and the camera lens obtains external images through this window. The frames marked with numbers 1 - 5 and different colors represent the spliced cameras. Considering the performance of the UAV and the relatively low flight altitude in the mission, a lightweight card-type digital camera with many pixels is selected as the component camera. Each camera is fixed rigidly on the transition plate with zero inclination to ensure the stable geometric position between the cameras remains unchanged. On the premise that the camera size and the installation conditions of the transition plate permit, the lenses of each camera are installed as close as possible (less than the preset lens spacing threshold) to increase the overlap area of the images obtained by each sub-camera and increase the number of effective homologous image points.

[0107] Through the embodiments of the present invention, the lens spacing between the fixed-position cameras in the side-view camera group is less than the preset lens spacing threshold, ensuring that the overlap rate between the images meets the subsequent processing requirements. The control of the lens spacing can improve the accuracy of image splicing and 3D reconstruction.

[0108] A method for emergency monitoring of reservoir area landslides based on a side-looking camera of an unmanned aerial vehicle. The target exposure points include a front exposure point, a middle exposure point, and a rear exposure point. Based on at least one exposure image, a post-temporal digital surface model of each candidate landslide image in the candidate landslide image group is constructed, including:

[0109] Determine the first exposure image when the fixed-wing unmanned aerial vehicle is at the front exposure point, the second exposure image when the fixed-wing unmanned aerial vehicle is at the middle exposure point, and the third exposure image when the fixed-wing unmanned aerial vehicle is at the rear exposure point;

[0110] Determine the pixels of the target location in the first exposure image, the second exposure image, and the third exposure image as the repeated coverage area;

[0111] Based on the repeated coverage area, construct a post-temporal digital surface model of each candidate landslide image in the candidate landslide image group.

[0112] Reference Figure 4 , Figure 4 is a schematic diagram of the homologous point area of three adjacent exposure points provided by the present invention.

[0113] Through a high-overlap five-patch camera, with one exposure, 10 pairs of homologous image points can be generated for one target ground object viewpoint in the data overlap area.

[0114] Design the flight path according to a 60% cross-track overlap degree. For the adjacent front, middle, and rear three exposure points, more homologous image points can be generated for one target ground object viewpoint in the data overlap area ( Figure 4 shown). The blue grid frame indicates the overlapping coverage area of the front and rear exposure images, and the red diagonal frame is the overlapping coverage area of the middle and rear exposure images. In the overlapping coverage area, the same ground object has image points on 10 images, while in the area simultaneously covered by the front-middle-rear exposure point images, there are 15 images. In this way, the data requirements for producing high-quality DSM products with an unmanned aerial vehicle optical camera are solved.

[0115] Based on the multi-view images of the repeated coverage area, through photogrammetry techniques such as stereo matching and dense matching, extract the three-dimensional point cloud data of the target location. Using the three-dimensional point cloud data, through interpolation or fitting algorithms (such as triangulation interpolation, Kriging interpolation, etc.), construct a post-temporal digital surface model (DSM) of each candidate landslide image in the candidate landslide image group.

[0116] Here, the repeated coverage area can be the overlapping coverage area of the images obtained by a single exposure, or the overlapping coverage area of the images obtained by the three exposure points.

[0117] It should be noted that for the front exposure point, the middle exposure point, and the rear exposure point, due to the zero-inclination and high-overlap multi-patch camera mode, the image group obtained by only a single exposure point also has a sufficient number of repeated coverage areas.

[0118] Through the embodiments of the present invention, comprehensive three-dimensional information of the target location can be obtained through multi-view images of the front, middle, and rear exposure points, improving the accuracy of the digital surface model; based on the multi-view images of the overlapping area, the post-temporal digital surface model constructed by photogrammetry technology has high accuracy and can accurately reflect the surface state after the landslide occurs.

[0119] In general unmanned aerial vehicle (UAV) aerial remote sensing tasks, such as DOM production and oblique photogrammetry, the overall lens module remains vertically downward. To meet such requirements, the attitude stability of the lens module is often maintained by installing a gimbal. Therefore, the UAV lens window is usually located on the belly of the aircraft.

[0120] In the landslide monitoring of the reservoir area of a hydropower station, to obtain the topography and geomorphology of the embankment, it is necessary to fly at a high altitude. At this time, the spatial resolution of the image decreases, and the test texture (potential landslide area) of the ground objects is compressed on the image, resulting in a poor effect. By means of side viewing, reducing the flight altitude can solve the problem of such data acquisition.

[0121] The aircraft attitude can be controlled, that is, the fuselage is tilted, to achieve side viewing of the camera. However, the disadvantage of this method is the accuracy problem of aircraft attitude control, resulting in unstable tilting, causing image coverage deviation and increasing the difficulty of subsequent data processing.

[0122] Side viewing can also be achieved by installing a gimbal and adjusting the camera attitude by controlling the gimbal. For fixed-wing aircraft, due to their high flight speed, large wind resistance, and the fact that the fuselage is externally hung on the gimbal for long endurance, this method affects the fuselage stability, with weak safety and low reliability.

[0123] Therefore, in the present invention, it is achieved by modifying the fuselage of a fixed-wing UAV.

[0124] First, the UAV lens window is modified from the belly of the aircraft to one side of the fuselage. At the position on the right side of the fuselage where there is no wing, connecting rod, or support rod blocking, a lens window is opened with a size slightly larger than the image hole size of the transition plate.

[0125] Secondly, a connecting rod is installed at the belly position. One end of the connecting rod is rigidly connected perpendicular to the belly, and the other end is a mounting plate with an angle adjustment device, and the mounting plate is connected to the transition plate of the five-piece camera.

[0126] The connecting rod is fixedly installed with the camera transition plate through bolts, and then the connecting rod is fixedly installed with the belly bottom plate through bolts.

[0127] Reference Figure 5 , Figure 5This is a schematic diagram of the installation of a right-side view fixed-wing drone and a five-piece camera provided by the present invention. It includes: a belly bottom plate, a connecting rod mounting plate, a mounting connecting rod, an angle adjuster, a transition plate mounting plate, a five-piece camera transition plate, and a camera side view angle.

[0128] Before the mission, first investigate and analyze the terrain of the monitoring target area in the reservoir area of the hydropower station. After determining the side view angle, set the tilt angle of the camera transition plate through the angle adjuster and fix it. During the same flight mission, this angle does not change. The angle adjustment range is limited by the cabin size. Figure 5 In the figure, the dotted-line graph represents the adjusted position state of the camera.

[0129] Reference Figure 6 , Figure 6 This is a schematic diagram of data acquisition of the side view system of the fixed-wing drone provided by the present invention. It includes: water surface, relative flight altitude, fuselage (including the five-piece camera system), image coverage area, maximum horizontal safety distance corresponding to the flight altitude in the flight area, bank target, and average distance between the aircraft and the ground object.

[0130] During the mission, the round-trip single flight path mode is adopted. Set the flight altitude (relative flight altitude) according to the image width. Usually, it is determined based on the ground (water surface) coverage (long side) corresponding to the maximum image width (i.e., the image coverage area) and the maximum horizontal safety distance corresponding to the flight altitude in the flight area, as shown in Figure 6 The maximum horizontal safety distance is defined as: at this flight altitude, the minimum horizontal distance between the aircraft and the ground object (bank target) (or the average distance between the aircraft and the ground object). When this distance is less than the minimum horizontal safety distance during the flight of the drone, the minimum safe flight distance of the drone is used instead.

[0131] When the space in the flight area cannot meet the maximum horizontal safety distance, adjust the camera side view angle and change the flight altitude to solve the image coverage problem (as shown by the dotted line in Figure 6 After adjusting the side view angle and flight altitude, when the images obtained from one flight path still cannot completely cover one side of the bank (in the vertical direction), then reuse the outbound flight path, adjust the flight altitude, and obtain the image data of the other half of the same-side bank. At this time, the flight altitude can be set based on the remaining bank situation on the basis of the previous flight altitude.

[0132] Since the system is a right-side view, the outbound flight path obtains the data of the right bank, and the return flight path obtains the data of the left bank.

[0133] According to a method for emergency monitoring of reservoir landslides based on a drone side view camera provided by the present invention, after constructing the digital surface model of the later phase of each candidate landslide image in the candidate landslide image group based on at least one exposure image, the above method further includes:

[0134] When there is no pre - constructed pre - temporal digital surface model in the candidate exposure position list, based on the candidate exposure position list, determine the candidate landslide area boundary of the candidate landslide image to obtain the candidate landslide area;

[0135] Obtain the target post - temporal digital surface model corresponding to the candidate landslide area;

[0136] Based on the grid size of the target post - temporal digital surface model, grid the candidate landslide area to obtain the grid landslide area;

[0137] Based on the simulated elevation values of each row of grids in the grid landslide area, simulate the pre - temporal digital surface model of the candidate landslide area;

[0138] According to the boundary growth method, based on the average elevation value of each grid in the grid landslide area, simulate the current - temporal digital surface model of the candidate landslide area;

[0139] Based on the comparison and analysis between the pre - temporal digital surface model and the current - temporal digital surface model, obtain the elevation change difference of the candidate landslide area;

[0140] When the elevation change difference of the candidate landslide area is greater than the elevation change threshold, take the candidate landslide area as the target landslide area.

[0141] In the embodiment of the present invention, when there is no pre - temporal digital surface model, it is necessary to use the landslide volume estimation algorithm without a reference digital surface model to complete the discrimination and volume estimation. Here, it is considered that the surface space is continuous, so the elevation information of the upper, lower, left, and right points can be used to simulate the ground elevation information of its center point.

[0142] When there is no pre - constructed pre - temporal digital surface model in the candidate exposure position list, based on the candidate exposure position list, determine the candidate landslide area boundary of the candidate landslide image through image analysis or a machine learning model to obtain the candidate landslide area.

[0143] Obtain the target post - temporal digital surface model (DSM) corresponding to the candidate landslide area, which reflects the surface elevation after the landslide occurs.

[0144] Reference Figure 7 , Figure 7 is the schematic diagram of the grid of the candidate landslide area provided by the present invention.

[0145] Based on the grid size of the target post - temporal digital surface model, perform grid processing on the candidate landslide area to obtain the grid landslide area. Each grid represents a fixed - size area.

[0146] Based on the simulated elevation values of each row of grids in the grid landslide area, a digital surface model of the previous time phase of the candidate landslide area is simulated through an interpolation or fitting algorithm. This model assumes the surface state before the landslide occurred.

[0147] Based on the boundary growth method, using the average elevation value of each grid in the grid landslide area, a digital surface model of the current time phase of the candidate landslide area is simulated. This model reflects the surface state after the landslide occurred.

[0148] The simulated digital surface model of the previous time phase is compared with the digital surface model of the current time phase grid by grid, and the elevation change difference of each grid is calculated. When the elevation change difference of the candidate landslide area is greater than the preset elevation change threshold, this area is marked as the target landslide area.

[0149] In some embodiments, the DSMs of the suspected landslide areas before and after quantitative analysis are compared, the change difference is calculated, and it is included in the list to be excluded. According to the list to be excluded, human-computer interaction is used to visually interpret the suspected landslide areas, identify errors are excluded, and the identified or suspected landslide areas and their related information (such as coordinates, suspected landslide volume, etc.) are added to the result set (list) to be verified.

[0150] In some embodiments, the recognition results of all suspected landslide areas are automatically merged to generate a result set (list) to be verified, and the system issues a warning to remind the staff to organize a higher-level expert interpretation or conduct on-site verification.

[0151] In some embodiments, during idle periods, flight data is automatically extracted to complete the generation of DSM products for the entire flight area.

[0152] When using this method for estimation, it can solve the problem of insufficient data. Combined with human-computer interaction interpretation, it can play a warning role to a certain extent. Especially when the target monitoring area is carried out for the first time and there is no DSM product, it can be applied. After the first monitoring is completed, the system can use the idle time to carry out the production of DSM products for the whole area as a reference value for subsequent monitoring.

[0153] Through the embodiments of the present invention, when there is no pre-constructed digital surface model of the previous time phase, the digital surface model of the previous time phase is generated through a simulation method. The grid processing divides the candidate landslide area into grids of a fixed size, simplifies the calculation process of the elevation change difference, and improves the processing efficiency; the boundary growth method can simulate the digital surface model of the current time phase based on the average elevation value of the grid, adapting to the landslide characteristics under different terrain conditions. By comparing the digital surface models of the previous time phase and the current time phase grid by grid, the elevation change difference can be accurately calculated, providing a quantitative basis for landslide identification.

[0154] A method for emergency monitoring of reservoir area landslides based on an unmanned aerial vehicle side-looking camera provided by the present invention, before simulating the digital surface model of the previous time phase of the candidate landslide area based on the simulated elevation values of each row of grids in the grid landslide area, the above method further includes:

[0155] Based on the target digital surface model of the later time phase, obtain the upper grid elevation, lower grid elevation, first grid elevation, and last grid elevation of each row of grids in the grid landslide area;

[0156] Take the average value between the upper grid elevation, lower grid elevation, first grid elevation, and last grid elevation as the simulated elevation value of each row of grids in the grid landslide area.

[0157] In the embodiment of the present invention, when simulating the digital surface model of the previous time phase of the suspected landslide area (candidate landslide area), from top to bottom, the elevation of each row of grids takes the average value of the upper, lower, first, and last grid elevations of this row as the elevation simulation value of this row of grids. Here, the upper, lower, first, and last grids refer to the existing values that can be directly read on the existing DSM product, not the simulated values. The grid values at the boundary positions are used as the starting initial values of the model.

[0158] The simulation process is carried out row by row from top to bottom to ensure the continuity of the elevation values. The elevation value of each row of grids is calculated based on the existing elevation values of its adjacent grids. For each row of grids, take the average value of the existing elevation values of the upper, lower, first, and last grids of this row as the elevation simulation value of this row of grids. Among them, the existing elevation values of the upper, lower, first, and last grids are directly read from the existing DSM product to ensure the reliability of the simulation value.

[0159] The grid values at the boundary positions are used as the starting initial values of the model to ensure a clear starting point for the simulation process. The existing elevation values of the boundary grids are directly read from the existing DSM product as the basis for the simulation.

[0160] Through the embodiment of the present invention, through the top-down simulation method, based on the existing elevation values of the upper, lower, first, and last grids, the DSM of the previous time phase of the suspected landslide area can be generated efficiently and accurately, ensuring the continuity and reliability of the elevation simulation.

[0161] A method for emergency monitoring of reservoir area landslides based on an unmanned aerial vehicle side-looking camera provided by the present invention, before simulating the digital surface model of the current time phase of the candidate landslide area based on the average elevation value of each grid in the grid landslide area according to the boundary growth method, the above method further includes:

[0162] Take each grid in the grid landslide area as the grid to be estimated, and perform the following steps to obtain the average elevation value of the grid to be estimated:

[0163] Based on the target post-temporal digital surface model, obtain the grid elevation values of the four adjacent areas of the grid to be estimated;

[0164] Based on the grid elevation values of the four adjacent areas of the grid to be estimated, determine the average elevation value of the grid to be estimated:

[0165]

[0166] Among them, represents the average elevation value of the grid to be estimated, represents the elevation value of the directly right grid of the grid to be estimated, represents the elevation value of the directly left grid of the grid to be estimated, represents the elevation value of the directly upper grid of the grid to be estimated, represents the elevation value of the directly lower grid of the grid to be estimated, represents the number of grids participating in the calculation.

[0167] In the embodiments of the present invention, the boundary growth method is used to estimate the average elevation in each grid within the suspected landslide area from the outside to the inside of the boundary, and construct the current temporal DSM.

[0168] For the grids within the suspected landslide area, using the elevation value at the boundary as the initial value, from top to bottom and from left to right, use the average value of the elevation values of the grids in the four adjacent areas (or eight adjacent areas) of the grid to be estimated, namely the directly upper, directly lower, directly left, and directly right grids of the grid to be estimated, as the elevation value of this grid.

[0169] In the above formula, represents the elevation value of the grid to be estimated in the i th row and j th column, n represents the number of grids participating in the estimation. In this example, four adjacent areas are used for simulation, that is, the directly upper, directly lower, directly left, and directly right grids of this grid. When there are grids that have not been estimated among these four grids, then this grid is removed and does not participate in the estimation.

[0170] Through the above embodiments of the present invention, for the need of monitoring the embankment landslide in the reservoir area of the hydropower station, a five - panel camera system for drones capable of side - view has been developed, which is suitable for fixed - wing drones and can efficiently complete the acquisition of optical image data for the production of three - dimensional products of the embankment in the reservoir area, especially for the large - drop terrain. The developed fixed - wing drone side - view five - panel camera system, without increasing the flight workload (the course overlap and other flight parameters remain unchanged), can, during a single - route flight, obtain a large number of homologous image points from the image obtained by a single exposure point to meet the data requirements for the production of the DSM product in the image coverage area, greatly improving the data acquisition efficiency.

[0171] Based on the data obtained by the side-looking five-stitch camera system, there are many homologous points in the images of the same ground object. By using only the data of single-exposure points or the data of the front, middle, and rear three-exposure points, the production of (high-quality) DSM products can be completed. The input data for DSM production is small, which reduces the processing pressure of the system and greatly improves the data production efficiency. Combining with the hardware device, the adopted technical solution of hierarchical processing reduces the amount of data input and processed each time, avoids the data processing pressure and processing cost brought by the whole, improves the data processing efficiency, and through the hierarchical processing and step-by-step refinement mode, reduces the workload of ineffective processing, reduces the omission rate of recognition, and achieves a good recognition effect. It can complete the technical method for estimating suspected landslides when there is no previous-phase DSM product.

[0172] The following describes the reservoir area landslide emergency monitoring device based on the UAV side-looking camera provided by the present invention. The reservoir area landslide emergency monitoring device based on the UAV side-looking camera described below can be mutually corresponding and referred to the reservoir area landslide emergency monitoring method described above.

[0173] Reference Figure 8 , Figure 8 is a schematic structural diagram of the reservoir area landslide emergency monitoring device based on the UAV side-looking camera provided by the present invention.

[0174] An acquisition module 801 is configured to acquire at least one exposure image captured by a side-looking camera group in a fixed-wing UAV, wherein the side-looking camera group includes a plurality of stitching cameras with zero inclination angles;

[0175] A first recognition module 802 is configured to perform first-level landslide recognition on the central camera image of at least one exposure image based on the established texture features through a plurality of preset models, to obtain a candidate landslide image group and a candidate exposure position list, wherein the central camera image is captured by the camera at the central position of the side-looking camera group;

[0176] A construction module 803 is configured to construct a post-phase digital surface model for each candidate landslide image in the candidate landslide image group based on at least one exposure image;

[0177] The above-mentioned acquisition module 801 is further configured to acquire a pre-constructed pre-phase digital surface model corresponding to the candidate exposure position list;

[0178] A second recognition module 804 is configured to perform second-level landslide recognition based on the post-phase digital surface model and the pre-phase digital surface model to obtain a target landslide area, wherein the elevation difference value between the post-phase digital surface model and the pre-phase digital surface model of the target landslide area is greater than the elevation difference threshold;

[0179] A determination module 805 is configured to determine the three-dimensional spatial volume change of the ground object in the target landslide area as the landslide volume of the target landslide area.

[0180] Specifically, the above-mentioned reservoir area landslide emergency monitoring device based on the UAV side-view camera provided by the present invention can implement all the method steps realized by the above-mentioned method embodiment of the reservoir area landslide emergency monitoring method based on the UAV side-view camera, and can achieve the same technical effects. The same parts and beneficial effects as those in the method embodiment will not be specifically described herein again.

[0181] Figure 9 is a schematic physical structure diagram of the electronic device provided by the present invention. As Figure 9 shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930, and a communication bus 940. Among them, the processor 910, the communication interface 920, and the memory 930 complete communication with each other through the communication bus 940. The processor 910 can call the logical instructions in the memory 930 to execute the reservoir area landslide emergency monitoring method based on the UAV side-view camera. The method includes: obtaining at least one exposure image captured by a side-view camera group in a fixed-wing UAV, where the side-view camera group includes multiple stitching cameras with zero inclination; based on the established texture features, performing first-level landslide identification on the central camera image of at least one exposure image through multiple preset models to obtain a candidate landslide image group and a candidate exposure position list, where the central camera image is captured by the central position camera of the side-view camera group; based on at least one exposure image, constructing a post-temporal digital surface model for each candidate landslide image in the candidate landslide image group; obtaining a pre-constructed pre-temporal digital surface model corresponding to the candidate exposure position list; performing second-level landslide identification based on the post-temporal digital surface model and the pre-temporal digital surface model to obtain a target landslide area, where the elevation difference value between the post-temporal digital surface model and the pre-temporal digital surface model of the target landslide area is greater than the elevation difference threshold; determining the three-dimensional spatial volume change of the ground objects in the target landslide area as the landslide volume of the target landslide area.

[0182] In addition, when the logical instructions in the above-mentioned memory 930 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0183] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the method for emergency monitoring of reservoir area landslides based on the side-looking camera of an unmanned aerial vehicle provided by the above-mentioned various methods. The method includes: acquiring at least one exposure image captured by a side-looking camera group in a fixed-wing unmanned aerial vehicle, where the side-looking camera group includes multiple stitching cameras with zero inclination angles; based on the established texture features, performing first-level landslide recognition on the central camera image of at least one exposure image through multiple preset models to obtain a candidate landslide image group and a candidate exposure position list, where the central camera image is captured by the central position camera of the side-looking camera group; based on at least one exposure image, constructing a post-temporal digital surface model for each candidate landslide image in the candidate landslide image group; acquiring a pre-constructed pre-temporal digital surface model corresponding to the candidate exposure position list; performing second-level landslide recognition based on the post-temporal digital surface model and the pre-temporal digital surface model to obtain a target landslide area, where the elevation difference value between the post-temporal digital surface model and the pre-temporal digital surface model of the target landslide area is greater than the elevation difference threshold; determining the three-dimensional spatial volume change of the ground objects in the target landslide area as the landslide volume of the target landslide area.

[0184] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the method for emergency monitoring of reservoir area landslides based on a drone side-looking camera, and the method includes: obtaining at least one exposure image captured by a side-looking camera group in a fixed-wing drone, wherein the side-looking camera group includes a plurality of stitching cameras with zero inclination angles; according to the established texture features, performing first-level landslide recognition on the central camera image of at least one exposure image through a plurality of preset models to obtain a candidate landslide image group and a candidate exposure position list, wherein the central camera image is captured by the central position camera of the side-looking camera group; based on at least one exposure image, constructing a post-temporal digital surface model for each candidate landslide image in the candidate landslide image group; obtaining a pre-constructed pre-temporal digital surface model corresponding to the candidate exposure position list; performing second-level landslide recognition based on the post-temporal digital surface model and the pre-temporal digital surface model to obtain a target landslide area, wherein the elevation difference value between the post-temporal digital surface model and the pre-temporal digital surface model of the target landslide area is greater than the elevation difference threshold; determining the three-dimensional spatial volume change of the ground objects in the target landslide area as the landslide volume of the target landslide area.

[0185] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0186] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solutions, in essence, or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0187] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An emergency monitoring method for reservoir area landslides based on a drone side-looking camera, characterized in that, Including: Obtaining at least one exposure image captured by a side-view camera group in a fixed-wing unmanned aerial vehicle, wherein the side-view camera group includes a plurality of stitching cameras with zero inclination angles; Based on the established texture features, performing first-level landslide recognition on the central camera image of the at least one exposure image through a plurality of preset models to obtain a candidate landslide image group and a candidate exposure position list, wherein the central camera image is captured by a camera at the central position of the side-view camera group; Based on the at least one exposure image, constructing a post-temporal digital surface model for each candidate landslide image in the candidate landslide image group; Obtaining a pre-constructed pre-temporal digital surface model corresponding to the candidate exposure position list; Performing second-level landslide recognition based on the post-temporal digital surface model and the pre-temporal digital surface model to obtain a target landslide area, wherein the elevation difference value between the post-temporal digital surface model and the pre-temporal digital surface model of the target landslide area is greater than an elevation difference threshold; Determining the three-dimensional spatial volume change of the ground objects in the target landslide area as the landslide volume of the target landslide area.

2. The emergency monitoring method for reservoir area landslide based on the side-view camera of the unmanned aerial vehicle according to claim 1, wherein, The obtaining at least one exposure image captured by a side-view camera group in a fixed-wing unmanned aerial vehicle includes: Invoking the fixed-wing unmanned aerial vehicle to fly on a single route, and using the side-view camera group located on the side of the fuselage of the fixed-wing unmanned aerial vehicle to collect ground object images of a target location when the fixed-wing unmanned aerial vehicle is at a preset target exposure point to obtain at least one exposure image; Wherein the side-view camera group includes a target number of fixed-position cameras, and the lens spacing between the target number of fixed-position cameras is less than a preset lens spacing threshold.

3. The method for emergency monitoring of reservoir area landslides based on a drone side-looking camera according to claim 2, wherein The target exposure points include a front exposure point, a middle exposure point, and a rear exposure point. Based on the at least one exposure image, constructing a post-temporal digital surface model for each candidate landslide image in the candidate landslide image group includes: Determining a first exposure image when the fixed-wing unmanned aerial vehicle is at the front exposure point, a second exposure image when the fixed-wing unmanned aerial vehicle is at the middle exposure point, and a third exposure image when the fixed-wing unmanned aerial vehicle is at the rear exposure point; Determining the pixels of the target location in the first exposure image, the second exposure image, and the third exposure image as the repeated coverage area; Based on the repeated coverage area, constructing a post-temporal digital surface model for each candidate landslide image in the candidate landslide image group.

4. The method for emergency monitoring of reservoir area landslides based on a drone side-looking camera according to claim 2, wherein, After constructing the post-temporal digital surface model for each candidate landslide image in the candidate landslide image group based on the at least one exposure image, the method further includes: When there is no corresponding pre-constructed pre-temporal digital surface model in the candidate exposure position list, determining the candidate landslide area boundary of the candidate landslide image based on the candidate exposure position list to obtain a candidate landslide area; Obtaining a target post-temporal digital surface model corresponding to the candidate landslide area; Based on the grid size of the target post-temporal digital surface model, gridifying the candidate landslide area to obtain a grid landslide area; Based on the simulated elevation values of each row of grids in the grid landslide area, simulate the digital surface model of the previous time phase of the candidate landslide area; According to the boundary growth method, based on the average elevation value of each grid in the grid landslide area, simulate the digital surface model of the current time phase of the candidate landslide area; Based on the comparison and analysis of the digital surface model of the previous time phase and the digital surface model of the current time phase, obtain the elevation change difference of the candidate landslide area; When the elevation change difference of the candidate landslide area is greater than the elevation change threshold, use the candidate landslide area as the target landslide area.

5. The emergency monitoring method for reservoir area landslide based on the side-view camera of the unmanned aerial vehicle according to claim 4, characterized in that Before simulating the digital surface model of the previous time phase of the candidate landslide area based on the simulated elevation values of each row of grids in the grid landslide area, the method further includes: Based on the digital surface model of the target later time phase, obtain the elevation of the upper grid, the elevation of the lower grid, the elevation of the first grid, and the elevation of the last grid of each row of grids in the grid landslide area; Take the average value among the elevation of the upper grid, the elevation of the lower grid, the elevation of the first grid, and the elevation of the last grid as the simulated elevation value of each row of grids in the grid landslide area.

6. The emergency monitoring method for reservoir area landslide based on the side-view camera of the unmanned aerial vehicle according to claim 4, wherein Before simulating the digital surface model of the current time phase of the candidate landslide area according to the boundary growth method based on the average elevation value of each grid in the grid landslide area, the method further includes: Take each grid in the grid landslide area as a grid to be estimated, and perform the following steps to obtain the average elevation value of the grid to be estimated: Based on the digital surface model of the target later time phase, obtain the grid elevation values of the four adjacent areas of the grid to be estimated; Based on the grid elevation values of the four adjacent areas of the grid to be estimated, determine the average elevation value of the grid to be estimated: ； Among them, represents the average elevation value of the grid to be estimated, represents the elevation value of the directly right grid of the grid to be estimated, represents the elevation value of the directly left grid of the grid to be estimated, represents the elevation value of the directly upper grid of the grid to be estimated, represents the elevation value of the directly lower grid of the grid to be estimated, represents the number of grids participating in the calculation.

7. An emergency monitoring device for reservoir area landslides based on a drone side-looking camera, characterized in that, Include: An acquisition module, configured to acquire at least one exposure image captured by a side-view camera group in a fixed-wing unmanned aerial vehicle, where the side-view camera group includes a plurality of stitching cameras with zero inclination angles; A first recognition module, configured to perform first-level landslide recognition on the central camera image of the at least one exposure image based on a plurality of preset models according to the established texture features, to obtain a candidate landslide image group and a candidate exposure position list, where the central camera image is captured by a camera at the central position of the side-view camera group; A construction module, configured to construct a digital surface model of the later time phase of each candidate landslide image in the candidate landslide image group based on the at least one exposure image; The acquisition module is further configured to acquire a pre-constructed digital surface model of the previous time phase corresponding to the candidate exposure position list; A second recognition module, configured to perform second-level landslide recognition based on the digital surface model of the later time phase and the digital surface model of the previous time phase, to obtain a target landslide area, where the elevation difference value between the digital surface model of the later time phase and the digital surface model of the previous time phase of the target landslide area is greater than the elevation difference threshold; A determination module, configured to determine the three-dimensional spatial volume change of the ground objects in the target landslide area as the landslide volume of the target landslide area.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method for emergency monitoring of reservoir area landslides based on an unmanned aerial vehicle side-looking camera according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method for emergency monitoring of reservoir area landslides based on an unmanned aerial vehicle side-looking camera according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for emergency monitoring of reservoir area landslides based on an unmanned aerial vehicle side-looking camera according to any one of claims 1 to 6.

Images