Reservoir area landslide emergency monitoring method and device based on unmanned aerial vehicle side view camera

Abstract

The invention provides a reservoir area landslide emergency monitoring method and device based on a side view camera of an unmanned aerial vehicle, and is applied to the technical field of remote sensing, and the method comprises the steps: obtaining at least one exposure image shot by a side view camera group in a fixed-wing unmanned aerial vehicle; performing first-level landslide identification based on the central camera image of the at least one exposure image through a plurality of preset models to obtain a candidate landslide image group and a candidate exposure position list; based on the at least one exposure image, constructing a post-phase digital surface model of each candidate landslide image in the candidate landslide image group; obtaining a pre-constructed front-time-phase digital surface model corresponding to the candidate exposure position list; performing second-stage landslide identification based on the rear-time-phase digital surface model and the front-time-phase digital surface model to obtain a target landslide area; determining the ground object three-dimensional space volume change of the target landslide area as the landslide volume of the target landslide area; according to the invention, the data acquisition efficiency and the landslide identification efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing technology, and in particular to a reservoir area landslide emergency monitoring method and device based on an unmanned aerial vehicle side-view camera. Background Art

[0002] In the field of remote sensing, high-precision landslide identification and estimation of landslide volume require the use of three-dimensional data products from two time phases. Data acquisition is a key prerequisite and top priority, and the production of three-dimensional data products has relatively high requirements for data acquisition.

[0003] The satellite SAR (Synthetic Aperture Radar) data used should be able to form interference, the optical data should be able to form stereo relativity, and the spatial resolution of satellite lidar data needs to be greatly improved. It is easy to obtain coherent data or stereo image pairs through manned aerial remote sensing, but the cost of data acquisition is high.

[0004] Therefore, whether it is satellite remote sensing or manned aerial remote sensing, it is relatively difficult to construct corresponding three-dimensional data products to achieve rapid identification of landslides, there are uncontrollable risks in the implementation cycle, and the cost is relatively high. Summary of the invention

[0005] The present invention provides a reservoir area landslide emergency monitoring method and device based on an unmanned aerial vehicle side-view camera, which is used to solve the defect of the landslide identification method in the prior art that it is difficult to obtain three-dimensional data, and can improve the data acquisition efficiency and landslide identification efficiency.

[0006] The present invention provides a reservoir area landslide emergency monitoring method based on an unmanned aerial vehicle side-view camera, comprising the following steps.

[0007] Acquire at least one exposure image captured by a side-view camera group in a fixed-wing unmanned aerial vehicle, wherein the side-view camera group includes a plurality of zero-tilt stitched cameras; perform first-level landslide identification based on a central camera image of the at least one exposure image through a plurality of preset models according to established texture features, and obtain a candidate landslide image group and a candidate exposure position list, wherein the central camera image is captured by a central position camera of the side-view camera group; construct a post-phase digital surface model of each candidate landslide image in the candidate landslide image group based on the at least one exposure image; obtain a pre-constructed front-phase digital surface model corresponding to the candidate exposure position list; perform second-level landslide identification based on the post-phase digital surface model and the front-phase digital surface model, and obtain a target landslide area, wherein the elevation difference value between the post-phase digital surface model and the front-phase digital surface model of the target landslide area is greater than an elevation difference threshold; determine the three-dimensional spatial volume change of the object in the target landslide area as the landslide volume of the target landslide area.

[0008] According to a reservoir area landslide emergency monitoring method based on a UAV side-view camera provided by the present invention, the acquisition of at least one exposure image captured by a side-view camera group in a fixed-wing UAV comprises: calling the fixed-wing UAV to fly a single route, and through the side-view camera group located on the fuselage side of the fixed-wing UAV, when the fixed-wing UAV is at a preset target exposure point, collecting ground image of the target location to obtain at least one exposure image; wherein the side-view camera group includes a target number of fixed-position cameras, and the lens spacing between the target number of fixed-position cameras is less than a preset lens spacing threshold.

[0009] According to a reservoir area landslide emergency monitoring method based on an unmanned aerial vehicle side-view camera provided by the present invention, the target exposure point includes a front exposure point, a middle exposure point and a rear exposure point, and the post-phase digital surface model of each candidate landslide image in the candidate landslide image group is constructed based on the at least one exposure image, including: determining a first exposure image of the fixed-wing unmanned aerial vehicle at the front exposure point, a second exposure image of the fixed-wing unmanned aerial vehicle at the middle exposure point, and a third exposure image of the fixed-wing unmanned aerial vehicle at the rear exposure point; determining pixels for the target location in the first exposure image, the second exposure image and the third exposure image as repeated coverage areas; and constructing a post-phase digital surface model of each candidate landslide image in the candidate landslide image group based on the repeated coverage areas.

[0010] According to a reservoir area landslide emergency monitoring method based on a side-view camera of an unmanned aerial vehicle provided by the present invention, after constructing a post-phase digital surface model of each candidate landslide image in the candidate landslide image group based on the at least one exposure image, the method further comprises: when there is no pre-constructed pre-phase digital surface model in the candidate exposure position list, determining the candidate landslide area boundary of the candidate landslide image based on the candidate exposure position list to obtain a candidate landslide area; acquiring a target post-phase digital surface model corresponding to the candidate landslide area; and based on the grid size of the target post-phase digital surface model, A landslide area is selected for gridding to obtain a grid landslide area; a previous phase digital surface model of the candidate landslide area is simulated based on the simulated elevation value of each row of grids in the grid landslide area; a current phase digital surface model of the candidate landslide area is simulated based on the average elevation value of each grid in the grid landslide area according to a boundary growth method; a comparison analysis is performed based on the previous phase digital surface model and the current phase digital surface model to obtain an elevation change difference of the candidate landslide area; when the elevation change difference of the candidate landslide area is greater than an elevation change threshold, the candidate landslide area is used as a target landslide area.

[0011] According to a reservoir area landslide emergency monitoring method based on an unmanned aerial vehicle side-view camera provided by the present invention, before simulating the previous phase digital surface model of the candidate landslide area based on the simulated elevation value of each row of grids in the grid landslide area, the method also includes: obtaining the upper grid elevation, the lower grid elevation, the first grid elevation and the last grid elevation of each row of grids in the grid landslide area based on the target post-phase digital surface model; and taking the average value of the upper grid elevation, the lower grid elevation, the first grid elevation and the last grid elevation as the simulated elevation value of each row of grids in the grid landslide area.

[0012] According to a reservoir area landslide emergency monitoring method based on a side-view camera of an unmanned aerial vehicle provided by the present invention, before simulating the current phase digital surface model of the candidate landslide area based on the average elevation value of each grid in the grid landslide area according to the boundary growth method, the method further comprises: taking each grid in the grid landslide area as a grid to be estimated, and performing the following steps to obtain the average elevation value of the grid to be estimated: obtaining the grid elevation values ​​of the four neighboring regions of the grid to be estimated based on the target post-phase digital surface model; determining the average elevation value of the grid to be estimated based on the grid elevation values ​​of the four neighboring regions of the grid to be estimated:

[0013] in, represents the average elevation value of the grid to be estimated, represents the elevation value of the grid to the right of the grid to be estimated, represents the elevation value of the grid immediately to the left of the grid to be estimated, represents the elevation value of the grid directly above the grid to be estimated, represents the elevation value of the grid directly below the grid to be estimated, Indicates the number of grids involved in the calculation.

[0014] The present invention also provides a reservoir area landslide emergency monitoring device based on a side-view camera of an unmanned aerial vehicle, comprising the following modules: an acquisition module, used to acquire at least one exposure image taken by a side-view camera group in a fixed-wing unmanned aerial vehicle, wherein the side-view camera group comprises a plurality of zero-tilt stitching cameras; a first recognition module, used to perform a first-level landslide recognition based on the central camera image of the at least one exposure image according to established texture features through a plurality of preset models, and obtain a candidate landslide image group and a candidate exposure position list, wherein the central camera image is taken by a camera at the central position of the side-view camera group; a construction module, used to perform a first-level landslide recognition based on the at least one exposure image according to established texture features and a plurality of preset models, and obtain a candidate landslide image group and a candidate exposure position list, wherein the central camera image is taken by a camera at the central position of the side-view camera group; image, constructing a post-phase digital surface model of each candidate landslide image in the candidate landslide image group; the acquisition module is also used to acquire a pre-constructed pre-phase digital surface model corresponding to the candidate exposure position list; a second identification module is used to perform a second-level landslide identification based on the post-phase digital surface model and the pre-phase digital surface model to obtain a target landslide area, wherein the elevation difference value between the post-phase digital surface model and the pre-phase digital surface model of the target landslide area is greater than an elevation difference threshold; a determination module is used to determine the three-dimensional spatial volume change of the object in the target landslide area as the landslide volume of the target landslide area.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for emergency monitoring of reservoir landslides based on a side-view camera of an unmanned aerial vehicle as described above is implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for emergency monitoring of reservoir landslides based on a side-view camera of an unmanned aerial vehicle as described above is implemented.

[0017] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for emergency monitoring of reservoir landslides based on a side-view camera of an unmanned aerial vehicle.

[0018] The method and device for emergency monitoring of reservoir landslides based on side-view cameras of unmanned aerial vehicles provided by the present invention obtain at least one exposure image through a side-view camera group carried by a fixed-wing unmanned aerial vehicle. The multiple zero-tilt splicing cameras in the side-view camera group can increase the number of points with the same name of the same feature that can be obtained by a single exposure point without increasing the flight workload, thereby improving the point cloud density of the subsequent digital surface model and enhancing the ability to depict the terrain; using the established texture features and multiple preset models, the first-level landslide recognition is performed only on the images obtained by the camera at the center position, which greatly reduces the single and total input data volume of the data to be detected, and multiple models are combined The proposed method can improve the recognition accuracy and reduce the recognition omission rate. Based on at least one exposure image, the post-phase digital surface model of each candidate landslide image in the candidate landslide image group is constructed, which can provide detailed terrain information of the suspected landslide area. The second-level landslide identification is carried out based on the post-phase digital surface model and the pre-phase digital surface model, and the target landslide area with elevation difference value greater than the threshold is screened out, which can exclude the misidentified area and improve the accuracy and reliability of landslide identification. By calculating the three-dimensional spatial volume change of the target landslide area and determining the landslide volume, the scale of the landslide can be quantified, realizing efficient and accurate landslide identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of the reservoir area landslide emergency monitoring method based on the UAV side-view camera provided by the present invention.

[0021] Figure 2 It is a schematic diagram of the overall process of the reservoir area landslide emergency monitoring method based on the UAV side-view camera provided by the present invention.

[0022] Figure 3 The present invention provides a schematic diagram of the installation of the five-camera UAV and a diagram of the overlapping effect of the acquired data.

[0023] Figure 4 It is a schematic diagram of the same-name point area of ​​three adjacent exposure points provided by the present invention.

[0024] Figure 5 It is a schematic diagram of the installation of a right-view fixed-wing UAV and a five-piece camera provided by the present invention.

[0025] Figure 6 It is a schematic diagram of data collection of the side-view system of the fixed-wing UAV provided by the present invention.

[0026] Figure 7 It is a grid schematic diagram of the candidate landslide area provided by the present invention.

[0027] Figure 8 It is a structural schematic diagram of a reservoir area landslide emergency monitoring device based on an unmanned aerial vehicle side-view camera provided by the present invention.

[0028] Fig. 9 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] Landslide disasters are one of the natural disasters that cause major economic losses, second only to earthquakes. The rise and fall of water levels in the reservoir area of ​​a hydropower station due to water storage and flood discharge will directly affect the stability of the upstream and downstream banks of the reservoir area, thereby posing safety hazards to cruise ships, dam bodies and power generation facilities in the reservoir area. Therefore, it is an important part of emergency monitoring of geological disasters in the reservoir area to identify disaster bodies and monitor key landslide bodies within a certain range upstream and downstream of the hydropower station. In recent years, many detection and monitoring technologies have been widely used in the field of landslide research, covering landslide area investigation, landslide displacement monitoring, landslide hazard assessment, and landslide hazard assessment. Among them, landslide investigation and monitoring is the basic link of the entire research work.

[0031] At present, the emergency monitoring methods for geological disasters such as landslides in reservoir areas include ground monitoring methods based on the Global Navigation Satellite System (GNSS) and remote sensing monitoring methods.

[0032] The GNSS automated deformation observation method is to set up GNSS deformation monitoring stations at key locations such as reservoir mountains, and set up a base station outside the landslide body. The monitoring station uses wireless transmission or remote control through a computer in the Internet management center to monitor the real-time deformation of reservoir mountains and other places, so as to monitor the landslide body 24 hours a day. This method has high monitoring time resolution, good effect and strong reliability. However, the disadvantages are also obvious, that is, due to the cost constraints of deformation station construction, it is impossible to deploy a large number of large areas, and the size of the monitoring grid is limited.

[0033] Remote sensing is currently one of the important technical means for rapid landslide monitoring due to its large monitoring range, few restrictions on ground conditions, all-day, all-weather, high precision, and high resolution. The basic principle of remote sensing landslide identification is to obtain changes (deformations) in the three-dimensional space of the same area through the two phases of data, as the result of suspected landslide identification, mainly including methods based on SAR (Synthetic Aperture Radar), methods based on optical remote sensing, methods based on lidar, and methods combining multiple remote sensing methods.

[0034] The SAR-based method mainly calculates (acquires) SAR interference quantities. A phase SAR can be used for differential interference processing based on external digital elevation model (DEM) data, or the interferometer data can be directly used to obtain differential interference results (deformation maps, intensity maps, coherence maps, and overlapping shadow maps). The interference results are used as the basis for identifying and judging landslides.

[0035] The main idea of ​​the optical remote sensing-based method is to obtain high-overlapping images of the target area and generate a regional three-dimensional numerical model (for example, digital surface model products (DSM, Digital Surface Model) and digital elevation model) through photogrammetry, compare the three-dimensional model products before and after, and identify landslides in the changed area.

[0036] The LiDAR-based method is different from optical remote sensing, which produces a regional 3D numerical model through photogrammetry. Instead, it directly obtains 3D point cloud data of the target area through sensors to generate a 3D numerical model product. By comparing this product with the previous 3D model product, landslides can be identified in the changed area.

[0037] Remote sensing monitoring methods that combine multiple means mainly integrate the advantages of different technical methods and make up for their shortcomings to improve the efficiency and accuracy of monitoring. For example, laser radar, optical remote sensing and drones are combined, and drones carrying LiDAR (Light Detection and Ranging) systems and optical lenses are used to combine PPP-RTK technology (a combination of precision single-point positioning and real-time dynamic positioning), and oblique photography post-processing methods are used to obtain high-precision three-dimensional terrain data, and compared with other phase data to identify suspected landslide areas.

[0038] With the development of technology, in addition to relying on three-dimensional information to identify suspected landslide areas, a method has been developed to identify landslides by directly using two-dimensional remote sensing images without constructing three-dimensional data product information, integrating multiple feature information such as spectrum, shape and texture, and using spatial statistics, analysis technology, deep learning technology, etc.

[0039] From the above analysis, it can be seen that one of the keys to using remote sensing methods to carry out landslide monitoring is to obtain specific types of remote sensing data in the target area in a timely manner. This type of data can be obtained through satellite remote sensing, manned aircraft and unmanned aerial remote sensing. Satellite remote sensing data has stable quality and a large coverage area, but it is greatly affected by local weather conditions when it is obtained, especially optical images. The aerial remote sensing method is relatively flexible compared to satellite remote sensing, especially unmanned aerial vehicles. Due to their flexibility and low cost, they have obvious advantages in obtaining remote sensing data in a small area, and are suitable for landslide monitoring in a certain area in front of and behind the dam of a hydropower station reservoir.

[0040] In order to improve the efficiency of obtaining aerial remote sensing data, one feasible method is to use multiple cameras and obtain a virtual stitched image through external field of view stitching technology, thereby expanding the ground coverage corresponding to the image obtained in a single exposure, thereby improving the efficiency of aerial photography.

[0041] In the field of remote sensing, data acquisition is a key prerequisite and top priority for high-precision landslide identification and landslide volume estimation. The production of such products has relatively high requirements for data acquisition. The satellite SAR data used should be able to form interference (InSAR), and the optical data should be able to form stereo relativity, while the spatial resolution of satellite lidar data needs to be greatly improved. It is easy to obtain coherent data or stereo image pairs through manned aerial remote sensing, but the cost of data acquisition is high. Therefore, whether it is satellite remote sensing or manned aerial remote sensing, especially to form a three-dimensional data product corresponding to the previous and next two times to realize the rapid identification and efficient early warning of reservoir landslides, its implementation is relatively difficult, there are uncontrollable risks in the implementation cycle, and the cost is relatively high.

[0042] Compared with satellite remote sensing and manned aerial remote sensing, the use of drones to monitor small-scale regional landslides is a low-cost and feasible technical solution. However, due to the limited image overlap in conventional drone flight modes (heading overlap and lateral overlap), the point cloud density of the digital surface model produced is not enough to accurately depict the surface topography. It is necessary to increase the image overlap by encrypting the routes and increase the number of pairs of images with the same name to increase the point cloud density of the digital surface model, but such a design will greatly reduce the efficiency of data acquisition. The ability to directly obtain three-dimensional data by carrying laser radars and other equipment increases the requirements for the performance and reliability of the drone itself, and the flight risks and costs are greatly increased. Moreover, drones suitable for this type of equipment (mostly multi-rotors) are often unable to carry out relatively large-scale and long-distance data acquisition tasks in one sortie compared to fixed-wing drones.

[0043] In addition, landslides may cause great harm to hydropower station dams. In applications, we should not only focus on improving the efficiency and ability of landslide detection, but more importantly, reduce the omission rate of landslide identification.

[0044] In the embodiment of the present invention, in view of these problems and the demand for emergency monitoring of landslide geological disasters in the reservoir area of ​​a hydropower station, a fixed-wing UAV five-camera side-view system is proposed to take into account efficiency and effect, so as to solve the problem of data acquisition. The present invention realizes the acquisition of multiple pairs of images with the same name for the same target in one flight of the UAV without adding additional flights, and satisfies the production of high-point cloud density digital surface models under the same acquisition conditions; uses a technical solution of hierarchical data processing to compress the data processing volume, and realizes the identification of suspected landslide areas under multiple methods in parallel based on two-dimensional images, thereby improving the identification effect and reducing the identification omission rate; completes the production of digital surface models for the suspected landslide area (image group) separately, reduces the cycle of producing digital surface models of large amounts of data, and improves the time efficiency of early warning; uses the boundary addition method to solve the problem of landslide volume estimation in the absence of a previous phase digital surface model; and finally realizes the rapid identification and landslide volume estimation of landslides along the coast of the reservoir area in a single flight, and issues early warnings based on the identification results.

[0045] The present invention solves the problems of three-dimensional data acquisition and hierarchical rapid identification required for identification through specific hardware equipment and hierarchical ideas, reduces the cost of emergency monitoring of landslides in the reservoir area of ​​a hydropower station, improves identification efficiency, reduces the identification omission rate, and enhances the ability to quickly warn of landslides.

[0046] Optionally, the reservoir area landslide emergency monitoring method based on the side-view camera of an unmanned aerial vehicle in the embodiment of the present application can be executed by a server, or by a terminal device, or jointly by a server and a terminal device, taking the execution of the reservoir area landslide emergency monitoring method based on the side-view camera of an unmanned aerial vehicle in the embodiment by a terminal device as an example.

[0047] Figure 1 is a flow chart of a reservoir area landslide emergency monitoring method based on a UAV side-view camera provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 101: Acquire at least one exposure image captured by a side-view camera group in a fixed-wing drone.

[0048] Wherein, the side-view camera group includes a plurality of zero-tilt stitched cameras; The present invention uses a different idea from the existing aerial remote sensing multi-camera to increase the coverage area of ​​the single exposure point image. Instead, it adopts multiple zero-tilt stitching cameras with large overlap. Under the premise of not increasing the UAV flight workload, the number of points with the same name on the same feature that can be obtained at a single exposure point is increased, thereby improving the point cloud density of subsequent digital surface model products and enhancing the ability to depict terrain.

[0049] Fixed-wing UAVs have the characteristics of fast flight speed and wide coverage, and are suitable for image collection tasks in large areas. Fixed-wing UAVs are equipped with side-view camera groups, which can shoot the target area at a side-view angle during flight and obtain multi-perspective image data.

[0050] The side-view camera group is composed of multiple zero-tilt stitched cameras. Zero tilt means that the optical axis of the camera is parallel to the horizontal plane, ensuring that the captured image is consistent in the vertical direction. Multiple cameras are arranged in a stitching manner to simultaneously acquire multi-angle images of the target area, increasing the image coverage and resolution.

[0051] It should be noted that, through the zero-tilt, high-overlap multi-camera mode, it is possible to achieve that the image acquired with a single exposure can meet the requirements for producing DSM data with high point cloud density; The camera side-view observation method can improve the ability to collect texture information on the side of the target to meet the requirements for obtaining geological disaster information on the side of the embankment, mountain and dam in the reservoir area; Step 102 : performing first-level landslide identification based on the central camera image of at least one exposure image by using multiple preset models according to the established texture features, and obtaining a candidate landslide image group and a candidate exposure position list.

[0052] The central camera image is captured by the central position camera of the side view camera group.

[0053] In the embodiment of the present invention, landslide recognition adopts a hierarchical mode (including first-level landslide recognition and second-level landslide recognition), uses image recognition or target recognition algorithm, and only performs landslide recognition on the images obtained by the camera at the center position, which greatly reduces the single and total input data volume of the data to be detected. Multiple models run in parallel, which improves the recognition accuracy and reduces the recognition omission rate.

[0054] The main method of using drone remote sensing to carry out high-precision landslide identification is to use the comparative analysis of the three-dimensional data (DSM / DEM) before and after. However, directly producing DSM products based on the acquired drone images is time-consuming and labor-intensive, and most of the time it is unnecessary.

[0055] Therefore, in the embodiment of the present invention, the first-level landslide identification directly targets the texture features of at least one exposure image (landslide image) collected, and uses methods such as deep learning and maximum likelihood model to extract suspected landslides.

[0056] First, input data preparation, only call the camera image at the center position, no color and tone processing is done on the image, and no geometric correction, distortion correction, or mosaicking is performed. The image is directly input into at least two preset models according to the scene (picture), and suspected landslides are identified based on the established texture features. The candidate landslide images and candidate exposure positions (list) corresponding to the suspected landslide area are extracted and recorded.

[0057] Secondly, the suspected landslide areas identified by various methods were merged to establish a list of candidate landslide image groups and candidate exposure locations where the suspected landslide areas were located, which served as the data screening basis for secondary landslide identification.

[0058] The suspected landslides identified by the above method have very low accuracy and there are a lot of misjudgments. However, when using this method, only a single image is input each time, the data volume is small, the data analysis pressure is small, and the time saved can be used to use the multi-model parallel repeated detection method to identify suspected landslides at the same time, thereby greatly reducing the recognition omission rate.

[0059] In some embodiments, the texture features of the pre-established landslide area are obtained as the basis for subsequent identification. The texture features are an important basis for landslide identification, and usually include information such as color, shape, edge, roughness, etc. in the image.

[0060] The images taken by the central camera in the side-view camera group are used as the core data source because of its central viewing angle, which can cover the main part of the target area and has relatively stable image quality. The resolution and coverage of the central camera image are moderate, making it suitable as the basic data for the first-level landslide identification.

[0061] Based on the established texture features, multiple preset models (such as convolutional neural networks, support vector machines, random forests, etc.) analyze the central camera image pixel by pixel or region by region to identify areas where landslides may occur. The identification results include candidate landslide image groups (i.e., image segments of suspected landslides) and their corresponding candidate exposure location lists (i.e., geographical locations of suspected landslides).

[0062] Through the embodiments of the present invention, the first-level landslide identification quickly screens out areas where landslides may occur through multiple preset models, greatly reducing the amount of data to be processed subsequently and improving the overall identification efficiency.

[0063] Step 103: constructing a post-temporal digital surface model of each candidate landslide image in the candidate landslide image group based on at least one exposure image.

[0064] In the embodiment of the present invention, at least one exposure image provides image data of the target area at different times or under different lighting conditions, which can more comprehensively reflect the characteristics of the surface.

[0065] For the candidate landslide image group obtained by the first-level landslide identification, each candidate landslide image is processed one by one, and the three-dimensional point cloud data of the ground object is extracted from at least one exposure image using photogrammetry technology (such as stereo matching, dense matching, etc.).

[0066] Based on the extracted 3D point cloud data, a post-temporal digital surface model (DSM) is generated through interpolation or fitting algorithms (such as triangulation interpolation, Kriging interpolation, etc.). The post-temporal digital surface model reflects the surface elevation and terrain characteristics after the target time (for example, after the landslide occurs).

[0067] Through the embodiment of the present invention, a high-precision post-phase digital surface model can be constructed by at least one exposure image to accurately reflect the surface changes after the landslide occurs. Only the images in the candidate landslide image group are processed, which reduces the amount of calculation and improves the efficiency of model construction.

[0068] Step 104: Obtain a pre-constructed previous phase digital surface model corresponding to the candidate exposure position list.

[0069] In the embodiment of the present invention, the candidate exposure position list is obtained by first-level landslide identification, and records the geographical location information of the suspected landslide area. According to the coordinate range of the candidate exposure position list, the corresponding previous phase digital surface model is retrieved and extracted from the existing digital surface model database.

[0070] The pre-temporal digital surface model is pre-built using historical image data (such as images taken by drones, satellites or other remote sensing equipment) and reflects the surface elevation and terrain characteristics at a specific time (before the landslide occurred).

[0071] Step 105 , performing second-level landslide identification based on the later-phase digital surface model and the earlier-phase digital surface model to obtain a target landslide area.

[0072] Among them, the elevation difference value between the later phase digital surface model and the earlier phase digital surface model of the target landslide area is greater than the elevation difference threshold.

[0073] In the embodiment of the present invention, firstly, according to the first-level suspected landslide identification result, an image group containing suspected landslides (each candidate landslide image in the candidate landslide image group) is extracted, and a post-phase digital surface model is produced based on the photogrammetry method only for the images in the image group.

[0074] Secondly, quantitative comparison and analysis is carried out with the previous phase DSM product. If any abnormal area is found, it is identified as a suspected landslide area, and the three-dimensional spatial volume change of the suspected landslide area is calculated as the suspected landslide volume.

[0075] Finally, a set (list) of suspected landslide results to be verified is established. Using this method, the number of images to be processed each time is greatly reduced, the system overhead is small, and the processing is easy and fast.

[0076] In some embodiments, the post-phase digital surface model (reflecting the surface state after the landslide) is compared with the pre-phase digital surface model (reflecting the surface state before the landslide) on a pixel-by-pixel or region-by-region basis. The comparison content includes changes in elevation values, deformation of terrain, etc.

[0077] The elevation difference between the digital surface model of the later phase and the digital surface model of the previous phase is calculated, that is, the elevation value of the later phase minus the elevation value of the previous phase. The elevation difference value reflects the change of the surface before and after the landslide occurs.

[0078] According to the characteristics of the landslide and the geological conditions of the region, a reasonable elevation difference threshold is set. The elevation difference threshold is used to distinguish normal terrain changes from terrain changes caused by landslides. The area with an elevation difference value greater than the elevation difference threshold is marked as the target landslide area. The target landslide area is the core area where the landslide occurs and has significant surface change characteristics.

[0079] Through the embodiments of the present invention, the second-level landslide identification is performed based on the later-phase digital surface model and the earlier-phase digital surface model, and the target landslide area can be accurately identified through the calculation of elevation difference values ​​and threshold screening.

[0080] Step 106, determining the three-dimensional spatial volume change of the ground object in the target landslide area as the landslide volume of the target landslide area.

[0081] In an embodiment of the present invention, three-dimensional spatial comparison data of the target landslide area is extracted based on the pre-phase digital surface model and post-phase digital surface model of the target landslide area, wherein the pre-phase DSM reflects the surface elevation before the landslide occurs, and the post-phase DSM reflects the surface elevation after the landslide occurs.

[0082] Based on the three-dimensional spatial comparison data, the elevation difference of each pixel or area is calculated by comparing the elevation values ​​of the previous phase DSM and the later phase DSM pixel by pixel; the elevation difference value is multiplied by the area of ​​the pixel or area to obtain the volume change of each pixel or area.

[0083] The volume changes of all pixels or regions in the target landslide area are accumulated to obtain the three-dimensional volume change of the entire target landslide area; the volume change is the landslide volume of the target landslide area.

[0084] refer to Figure 2 , Figure 2It is a schematic diagram of the overall process of the reservoir area landslide emergency monitoring method based on the side-view camera of the unmanned aerial vehicle provided by the present invention. It includes: task initiation; unmanned aerial vehicle reservoir area landslide monitoring (cruise mode); the side-view system of the unmanned aerial vehicle equipped with a five-piece camera acquires data; the data is imported into the system after the flight; the suspected landslide texture is quickly identified based on the data of the central camera; the exposure point data of the suspected landslide texture is retrieved; the DSM product production based on the photogrammetry method; judging whether there is a previous phase DSM; if not, estimating the DSM by the boundary growth method; if so, conducting quantitative comparative analysis with the previous phase DSM product; secondary landslide determination; if not, excluding the landslide risk; if so, calculating the landslide volume; uploading the results to the system, starting or terminating the warning according to the regulations; the task is completed.

[0085] First of all, the present invention uses a different idea from the existing aerial remote sensing multi-camera to increase the coverage area of ​​the single exposure point image. By adopting a camera five-shot mode with a large overlap, the number of points with the same name that can be obtained by a single exposure point is increased without increasing the flight workload, thereby improving the point cloud density of the DSM product and enhancing the ability to depict the terrain.

[0086] Secondly, due to the change in the camera stitching method, in view of the characteristics of the reservoir shore data demand, the multi-route flight was changed to a single-route flight, that is, one flight was used to obtain data that meets the production of DSM three-dimensional products. Combined with the route overlap rate (usually the flight setting is 60%), dozens of groups of image pairs with the same name can be obtained for the same target.

[0087] Third, the fixed-wing UAV’s downward-looking image acquisition mode (the observation port is located on the belly of the aircraft) is changed to an observation port set up on the side of the fuselage, and the five-camera system acquires images of coastal features along the reservoir in a side-view manner.

[0088] Fourth, landslide identification adopts a hierarchical mode, using image recognition or target recognition algorithms, and only performs landslide identification on images obtained by the central camera, which greatly reduces the single and total input data volume of the data to be detected. Multiple models run in parallel, which improves the recognition accuracy and reduces the recognition omission rate. For the suspected results of the identification, the image group of the exposure point is retrieved, and the photogrammetry method is used to produce the DSM product of the point, and compared with the previous DSM products for quantitative analysis, further determine the landslide area, and obtain the volume of the landslide; because the images within a small range of the landslide location are used as input, the number of inputs to the DSM production model is small, the workload of completing the DSM production is small, and the time is fast, which improves the time efficiency of the entire identification process.

[0089] Fifth, when the system is idle, it completes the production of DSM products based on the data obtained by the drone, which serves as the benchmark data for the next phase of detection.

[0090] Sixth, when there is no DSM product for the previous phase, all identification results are submitted to manual interpretation. The DSM product production is completed for the data of the exposure points where the suspected landslide area is located, and the DSM of the previous phase is estimated according to the boundary growth method to estimate the landslide volume.

[0091] In this invention, DSM (digital surface model) is used. On the one hand, DSM is a three-dimensional product without removing ground cover, which can more comprehensively and meticulously reflect the surface conditions than DEM digital elevation model. On the other hand, DSM production of DEM requires additional manpower and time costs. Obviously, in terms of application scenarios, the use of DSM is more appropriate in terms of both analysis effect and cost.

[0092] Through the above steps of the embodiment of the present invention, at least one exposure image is obtained by the side-view camera group carried by the fixed-wing UAV. The multiple zero-tilt stitching cameras in the side-view camera group can increase the number of points with the same name of the same feature that can be obtained by a single exposure point without increasing the flight workload, thereby improving the point cloud density of the subsequent digital surface model and enhancing the ability to depict the terrain; using the established texture features and multiple preset models, the first-level landslide recognition is performed only on the image obtained by the camera at the center position, which greatly reduces the single and total input data volume of the data to be detected, and multiple models run in parallel, which improves The accuracy of recognition is improved, and the omission rate of recognition is reduced; based on at least one exposure image, a post-phase digital surface model of each candidate landslide image in the candidate landslide image group is constructed, which can provide detailed terrain information of the suspected landslide area; the second-level landslide identification is performed based on the post-phase digital surface model and the pre-phase digital surface model, and the target landslide area with elevation difference value greater than the threshold is screened out, which can exclude the misidentified area and improve the accuracy and reliability of landslide identification; by calculating the three-dimensional spatial volume change of the target landslide area, the landslide volume is determined, the scale of the landslide can be quantified, and efficient and accurate landslide identification is achieved.

[0093] According to a reservoir area landslide emergency monitoring method based on a UAV side-view camera provided by the present invention, at least one exposure image captured by a side-view camera group in a fixed-wing UAV is obtained, comprising: The fixed-wing UAV is called to fly a single route, and a side-view camera group located on the side of the fuselage of the fixed-wing UAV is used to collect ground image of the target location when the fixed-wing UAV is at a preset target exposure point, so as to obtain at least one exposure image; The side view camera group includes a target number of fixed position cameras, and the lens spacing between the target number of fixed position cameras is less than a preset lens spacing threshold.

[0094] As mentioned above, when general aerial remote sensing produces digital orthophotos, optical cameras are used to design flight routes according to the specifications for setting the overlap of heading (flight direction) and lateral (vertical route direction), such as 60% heading and 30% lateral. This can not only meet the accuracy requirements of digital orthophoto products, but also achieve a balance between product accuracy and quality and data acquisition efficiency.

[0095] However, when producing high-quality digital surface models, a larger proportion of image overlap is required to increase the number of image pairs with the same name (to build stereo image pairs). The usual practice is to increase the number of flight routes, which brings about the problem of a significant increase in data acquisition time and data volume.

[0096] In some embodiments, the landslide occurs on the mountain, and in the reservoir area of ​​the hydropower station, the height of the mountains (banks) on both sides of the reservoir bank is generally tens to hundreds of meters. Therefore, the data obtained by the drone only needs to reach this coverage capability, that is, the image width is tens to hundreds of meters.

[0097] In order to solve this problem, the embodiment of the present invention combines the characteristics of landslide monitoring in reservoir areas, that is, only focusing on the landslide situation within a certain range of the reservoir bank, and redesigns the drone camera. While meeting the width requirement of a single image, the original camera splicing method to increase the image width is converted into a zero-tilt camera splicing method, thereby increasing the coverage area of ​​the overlapping area during a single exposure, thereby increasing the number of image pairs with the same name and achieving the effect of enhancing the point cloud density of the digital surface model.

[0098] refer to Figure 3 , Figure 3 The present invention provides a schematic diagram of the installation of the five-camera UAV and a diagram of the overlapping effect of the acquired data.

[0099] like Figure 3 As shown, the blue frame is the camera installation transition plate, which is made of carbon steel and is lightweight and has sufficient mechanical strength. An image window is opened in the center of the plate, through which the camera lens obtains external images. Boxes marked with numbers 1-5 and different colors represent spliced ​​cameras. Considering the performance of the drone and the relatively low flight altitude during the mission, high-pixel, lightweight card-type digital cameras are selected as component cameras. Each camera is rigidly fixed on the transition plate with zero inclination to ensure that the geometric position between the cameras is stable and unchanged. Under the premise that the camera size and the installation conditions of the transition plate allow, the lenses of each camera are installed as close as possible (less than the preset lens spacing threshold) to increase the overlapping area of ​​the images obtained by each sub-camera and increase the number of effective same-life image points.

[0100] Through the embodiments of the present invention, the lens spacing between fixed position cameras in the side view camera group is less than a preset lens spacing threshold, ensuring that the overlap rate between images meets subsequent processing requirements. The control of the lens spacing can improve the accuracy of image stitching and three-dimensional reconstruction.

[0101] According to a reservoir area landslide emergency monitoring method based on a UAV side-view camera provided by the present invention, the target exposure points include front exposure points, middle exposure points and rear exposure points. Based on at least one exposure image, a rear-phase digital surface model of each candidate landslide image in a candidate landslide image group is constructed, including: Determine a first exposure image of the fixed-wing drone at a front exposure point, a second exposure image of the fixed-wing drone at a middle exposure point, and a third exposure image of the fixed-wing drone at a rear exposure point; Determine pixels corresponding to the target location in the first exposure image, the second exposure image, and the third exposure image as repeated coverage areas; Based on the repeated coverage area, a post-temporal digital surface model of each candidate landslide image in the candidate landslide image group is constructed.

[0102] refer to Figure 4 , Figure 4 It is a schematic diagram of the same-name point area of ​​three adjacent exposure points provided by the present invention.

[0103] Through the high-overlap five-camera mosaic, one exposure can generate 10 pairs of image points with the same name for one target object viewpoint in the data overlap area.

[0104] The flight path is designed according to the 60% heading overlap. The data overlap area ( Figure 4 (As shown in the figure, a target object viewpoint can generate more image points with the same name. The blue well frame indicates the overlapping coverage area of ​​the front and back exposure images, and the red diagonal frame indicates the overlapping coverage area of ​​the middle and back exposure images. In the overlapping coverage area, the same object has image points in 10 scenes, while the area covered by the front, middle and back exposure point images at the same time has 15 scenes. In this way, the data demand for producing high-quality DSM products with drone optical cameras is solved.

[0105] Based on the multi-view images of the repeated coverage area, the three-dimensional point cloud data of the target location is extracted through photogrammetry techniques such as stereo matching and dense matching. Using the three-dimensional point cloud data, through interpolation or fitting algorithms (such as triangulation interpolation, Kriging interpolation, etc.), a post-temporal digital surface model (DSM) of each candidate landslide image in the candidate landslide image group is constructed.

[0106] Here, the repeated coverage area may be an overlapping coverage area of ​​an image acquired by a single exposure, or an overlapping coverage area of ​​an image acquired by three exposure points.

[0107] It should be noted that for the front exposure point, the middle exposure point and the rear exposure point, due to the zero-tilt, high-overlap multi-camera mode, the image group obtained by only a single exposure point also has a sufficient number of repeated coverage areas.

[0108] Through the embodiments of the present invention, comprehensive three-dimensional information of the target location can be obtained through multi-view images of the front, middle and rear exposure points, thereby improving the accuracy of the digital surface model; based on the multi-view images of the repeated coverage area, the post-phase digital surface model constructed by photogrammetry technology has high accuracy and can accurately reflect the surface state after the landslide occurs.

[0109] In general UAV remote sensing tasks, such as DOM production, oblique photogrammetry, etc., the lens module is kept vertically downward. In order to meet such requirements, a gimbal is often installed to maintain the stability of the lens module. Therefore, the UAV lens window is usually located on the belly of the aircraft.

[0110] In the landslide monitoring of the reservoir area of ​​a hydropower station, in order to obtain the topography of the embankment, it is necessary to fly high. At this time, the spatial resolution of the image is reduced, and the test texture of the ground object (the potential landslide area) is compressed on the image, and the effect becomes worse. By looking sideways, reducing the flight altitude can solve this problem of data acquisition.

[0111] The camera side view can be achieved by controlling the aircraft attitude, that is, controlling the fuselage tilt. However, the disadvantage of this method is the problem of aircraft attitude control accuracy, which leads to instability in the tilt, causing image coverage deviation and increasing the difficulty of later data processing.

[0112] You can also add a gimbal and adjust the camera posture to achieve side view by controlling the gimbal. For fixed-wing aircraft, due to its high flight speed and large wind resistance, the fuselage is hung on the gimbal for fast and long flight time. This method affects the stability of the fuselage, and has weaker safety and lower reliability.

[0113] Therefore, the present invention adopts the fixed-wing UAV fuselage modification method to achieve it.

[0114] First, the drone lens window is relocated from the belly to the side of the fuselage. A lens window is opened on the right side of the fuselage where there is no wing, connecting rod, or support rod blocking the view, with the size slightly larger than the image hole of the transition plate.

[0115] Secondly, a connecting rod is installed at the belly of the fuselage, one end of the connecting rod is vertically rigidly connected to the belly of the fuselage, and the other end is a mounting plate with an angle adjustment device, and the mounting plate is connected to the five-piece camera transition plate.

[0116] The connecting rod is fixed to the camera transition plate by bolts, and then the connecting rod is fixed to the belly bottom plate by bolts.

[0117] refer to Figure 5 , Figure 5 It is a schematic diagram of the installation of a right-side-view fixed-wing UAV and a five-piece camera provided by the present invention, which includes: a belly bottom plate, a connecting rod mounting plate, a mounting connecting rod, an angle adjuster, a transition plate mounting plate, a five-piece camera transition plate and a camera side view angle.

[0118] Before the mission, the terrain of the target monitoring area of ​​the hydropower station reservoir is first investigated and analyzed. After the side view angle is determined, the camera transition plate tilt angle is set and fixed through the angle adjuster. The angle does not change during the same flight. The angle adjustment range is limited by the cabin size. Figure 5 In the figure, the dotted line represents the camera position state after adjustment.

[0119] refer to Figure 6 , Figure 6 It is a schematic diagram of data acquisition of the side-view system of the fixed-wing UAV provided by the present invention, which includes: water surface, relative altitude, fuselage (including five-camera system), image coverage, maximum horizontal safety distance corresponding to the flight altitude in the flight area, embankment target and average distance between the aircraft and the ground.

[0120] During the mission, a round-trip single-route flight mode is adopted, and the flight altitude (relative altitude) is set according to the image width. It is usually determined by the maximum horizontal safety distance corresponding to the ground (water surface) coverage (long side) of the maximum image width (i.e. image coverage range) and the flight altitude in the flight area, such as Figure 6 The maximum horizontal safety distance is defined as: the minimum horizontal distance between the aircraft and the ground object (embankment target) at this flight altitude (or the average distance between the aircraft and the ground object). When this distance is less than the minimum horizontal safety distance of the drone during flight, the minimum safe flight distance of the drone is used instead.

[0121] When the flight zone space cannot meet the maximum horizontal safety distance, adjust the camera side view angle, change the flight altitude, and solve the image coverage problem (such as Figure 6 After adjusting the side view angle and altitude, if the image acquired by one route cannot completely cover one side of the embankment (vertical direction), the outward route is used again, and the image data of the other half of the embankment on the same side is acquired after adjusting the flight altitude. At this time, the altitude can be set based on the last altitude and the situation of the remaining embankment.

[0122] Since the system is right-side-view, the outbound route obtains the right bank data, and the return route obtains the left bank data.

[0123] According to a reservoir area landslide emergency monitoring method based on a UAV side-view camera provided by the present invention, after constructing a post-phase digital surface model of each candidate landslide image in a candidate landslide image group based on at least one exposure image, the method further comprises: When there is no pre-constructed previous phase digital surface model in the candidate exposure position list, determining the candidate landslide area boundary of the candidate landslide image based on the candidate exposure position list to obtain the candidate landslide area; Obtaining a target post-temporal digital surface model corresponding to the candidate landslide area; Based on the grid size of the target post-phase digital surface model, the candidate landslide area is gridded to obtain the grid landslide area; Based on the simulated elevation value of each row of grids in the grid landslide area, a digital surface model of the previous phase of the candidate landslide area is simulated; According to the boundary growth method, based on the average elevation value of each grid in the grid landslide area, the current phase digital surface model of the candidate landslide area is simulated; Based on the comparison and analysis between the previous phase digital surface model and the current phase digital surface model, the elevation change difference of the candidate landslide area is obtained; When the elevation change difference of the candidate landslide area is greater than the elevation change threshold, the candidate landslide area is taken as the target landslide area.

[0124] In the embodiment of the present invention, when there is no previous phase digital surface model, it is necessary to use a landslide volume estimation algorithm without a reference digital surface model to complete the discrimination and volume estimation. Here, it is considered that the surface space is continuous, so the elevation information of the upper, lower, left and right points can be used to simulate the ground elevation information of its center point.

[0125] When there is no pre-constructed previous phase digital surface model in the candidate exposure position list, the candidate landslide area boundary of the candidate landslide image is determined based on the candidate exposure position list through image analysis or machine learning model to obtain the candidate landslide area.

[0126] The target post-temporal digital surface model (DSM) corresponding to the candidate landslide area is obtained, which reflects the surface elevation after the landslide occurs.

[0127] refer to Figure 7 , Figure 7 It is a grid schematic diagram of the candidate landslide area provided by the present invention.

[0128] Based on the grid size of the target post-phase digital surface model, the candidate landslide area is gridded to obtain the grid landslide area. Each grid represents an area of ​​fixed size.

[0129] Based on the simulated elevation values ​​of each row of grids in the grid landslide area, the digital surface model of the previous phase of the candidate landslide area is simulated by interpolation or fitting algorithm. This model assumes the surface state before the landslide occurred.

[0130] Based on the boundary growth method, the average elevation value of each grid in the grid landslide area is used to simulate the current phase digital surface model of the candidate landslide area. This model reflects the surface state after the landslide occurs.

[0131] The simulated digital surface model of the previous phase is compared with the digital surface model of the current phase grid by grid, and the elevation change difference of each grid is calculated. When the elevation change difference of the candidate landslide area is greater than the preset elevation change threshold, the area is marked as the target landslide area.

[0132] In some embodiments, the DSM of the suspected landslide area before and after the quantitative analysis is compared, and the difference in change is calculated and included in the list to be excluded. Based on the list to be excluded, the suspected landslide area is visually interpreted by human-computer interaction to eliminate identification errors, and the identifiable or suspected landslide area and its related information (such as coordinates, suspected landslide volume, etc.) are added to the set of results to be verified (list).

[0133] In some embodiments, the identification results of all suspected landslide areas are automatically merged to generate a set (list) of results to be verified, and the system issues an early warning to remind the staff to organize higher-level experts to interpret or conduct on-site verification.

[0134] In some embodiments, during idle time periods, flight data is automatically extracted to complete the generation of DSM products for the entire flight area.

[0135] When using this method to carry out estimation, the problem of insufficient data can be solved. Combined with human-computer interaction interpretation, it can play an early warning role to a certain extent. It is especially applicable when the target monitoring area is carried out for the first time and there is no DSM product. After the first monitoring is completed, the system can use idle time to produce DSM products for the entire area as a reference value for subsequent monitoring.

[0136] Through the embodiment of the present invention, in the absence of a pre-constructed digital surface model of the previous phase, a digital surface model of the previous phase is generated by a simulation method, and the gridding process divides the candidate landslide area into grids of fixed size, which simplifies the calculation process of the elevation change difference and improves the processing efficiency; the boundary growth method can simulate the digital surface model of the current phase based on the average elevation value of the grid, and adapt to the landslide characteristics under different terrain conditions. By comparing the digital surface models of the previous phase and the current phase grid by grid, the elevation change difference can be accurately calculated, providing a quantitative basis for landslide identification.

[0137] According to a reservoir area landslide emergency monitoring method based on a UAV side-view camera provided by the present invention, before simulating the previous phase digital surface model of the candidate landslide area based on the simulated elevation value of each row of grids in the grid landslide area, the method further includes: Based on the target post-phase digital surface model, the upper grid elevation, lower grid elevation, first grid elevation and last grid elevation of each row of grids in the grid landslide area are obtained; The average value among the upper grid elevation, the lower grid elevation, the first grid elevation and the last grid elevation is taken as the simulated elevation value of each row of grids in the grid landslide area.

[0138] In the embodiment of the present invention, the digital surface model of the previous phase of the suspected landslide area (candidate landslide area) is simulated. From top to bottom, the elevation of each row of grids is taken as the average value of the elevation of the upper, lower, first and last grids of the row as the simulated value of the elevation of the row of grids. Here, the upper, lower, first and last grids refer to the existing values ​​that can be directly read on the existing DSM products, not the simulated values. The grid value at the boundary position is used as the initial value of the model.

[0139] The simulation process is carried out row by row from top to bottom to ensure the continuity of the elevation values. The elevation value of each row of grids is calculated based on the existing elevation values ​​of its adjacent grids. For each row of grids, the average of the existing elevation values ​​of the upper, lower, first and last grids in the row is taken as the elevation simulation value of the row of grids. Among them, the existing elevation values ​​of the upper, lower, first and last grids are directly read from the existing DSM products to ensure the reliability of the simulation values.

[0140] The grid values ​​at the boundary locations are used as the initial values ​​of the model to ensure a clear starting point for the simulation process. The existing elevation values ​​of the boundary grid are directly read from the existing DSM products as the basis for the simulation.

[0141] Through the embodiments of the present invention, a DSM of the upper phase of the suspected landslide area can be efficiently and accurately generated based on the existing elevation values ​​of the upper, lower, first and last grids through a top-down simulation method, thereby ensuring the continuity and reliability of the elevation simulation.

[0142] According to a reservoir area landslide emergency monitoring method based on a UAV side-view camera provided by the present invention, before simulating the current phase digital surface model of the candidate landslide area based on the average elevation value of each grid in the grid landslide area according to the boundary growth method, the method further includes: Take each grid in the grid landslide area as the grid to be estimated, and perform the following steps to obtain the average elevation value of the grid to be estimated: Based on the target post-phase digital surface model, the grid elevation values ​​of the four neighboring areas of the grid to be estimated are obtained; Based on the grid elevation values ​​of the four neighboring regions of the grid to be estimated, determine the average elevation value of the grid to be estimated:

[0143] in, Represents the average elevation value of the grid to be estimated, Indicates the elevation value of the grid to the right of the grid to be estimated. Indicates the elevation value of the grid to the left of the grid to be estimated. Indicates the elevation value of the grid just above the grid to be estimated. Indicates the elevation value of the grid directly below the grid to be estimated. Indicates the number of grids involved in the calculation.

[0144] In the embodiment of the present invention, the boundary growth method is used to estimate the average elevation of each grid in the suspected landslide area from the outside to the inside of the boundary, and construct the DSM of the current phase.

[0145] For the grids in the suspected landslide area, the elevation value at the boundary is used as the initial value, and from top to bottom and from left to right, the average elevation value of the grids in the upper, lower, left and right four neighboring domains or eight neighboring domains of the grid to be estimated is used as the elevation value of the grid.

[0146] In the above formula, Indicates i Line j List the elevation values ​​of the grid to be estimated. n Indicates the number of grids involved in the estimation. In this example, four neighboring regions are used for simulation, namely the grids directly above, directly below, directly to the left, and directly to the right of the grid. If there is a grid that has not been estimated among these four grids, the grid will be removed and will not be involved in the estimation.

[0147] Through the above-mentioned embodiments of the present invention, a UAV five-piece camera system capable of side-viewing has been developed to meet the needs of landslide monitoring on the embankment of a hydropower station reservoir area. The system is suitable for fixed-wing UAVs and can efficiently complete the acquisition of optical image data for the production of three-dimensional products on the embankment of the reservoir area, especially on terrain with large drop heights. The developed fixed-wing UAV side-view five-piece camera system can fly on a single route without increasing the flight workload (with overlapping headings and other flight parameters unchanged). The image acquired at a single exposure point can obtain a large number of image points with the same name to meet the data required for the production of DSM products in the image coverage area, greatly improving the data acquisition efficiency.

[0148] Relying on the data obtained by the side-view five-camera system, there are many points with the same name in the same ground feature image. Only single exposure point data, or front, middle and back three exposure point data, can complete the (high-quality) DSM product production. The DSM production process requires less input data, which reduces the system processing pressure and greatly improves data production efficiency. Combined with hardware devices, the hierarchical processing technical solution is adopted to reduce the amount of data input and processing at a single time, avoid the data processing pressure and processing cost brought by the overall addition, improve data processing efficiency, and reduce the invalid processing workload and recognition omission rate through hierarchical processing and step-by-step refinement mode, so as to achieve better recognition effect. A technical method that can complete the estimation of suspected landslides when there is no previous phase DSM product.

[0149] The reservoir area landslide emergency monitoring device based on the UAV side-view camera provided by the present invention is described below. The reservoir area landslide emergency monitoring device based on the UAV side-view camera described below and the reservoir area landslide emergency monitoring method based on the UAV side-view camera described above can be referenced to each other.

[0150] refer to Figure 8 , Figure 8 It is a structural schematic diagram of a reservoir area landslide emergency monitoring device based on an unmanned aerial vehicle side-view camera provided by the present invention.

[0151] An acquisition module 801 is used to acquire at least one exposure image captured by a side-view camera group in a fixed-wing UAV, wherein the side-view camera group includes a plurality of zero-tilt stitching cameras; A first recognition module 802 is used to perform first-level landslide recognition based on the central camera image of at least one exposure image through multiple preset models according to the established texture features, and obtain a candidate landslide image group and a candidate exposure position list, wherein the central camera image is obtained by taking a picture of the central position of the side view camera group; A construction module 803 is used to construct a post-temporal digital surface model of each candidate landslide image in the candidate landslide image group based on at least one exposure image; The acquisition module 801 is further used to acquire a pre-constructed previous phase digital surface model corresponding to the candidate exposure position list; The second identification module 804 is used to perform second-level landslide identification based on the digital surface model of the later phase and the digital surface model of the previous phase to obtain a target landslide area, wherein the elevation difference value between the digital surface model of the later phase and the digital surface model of the previous phase of the target landslide area is greater than the elevation difference threshold; The determination module 805 is used to determine the three-dimensional spatial volume change of the ground object in the target landslide area as the landslide volume of the target landslide area.

[0152] Specifically, the above-mentioned reservoir area landslide emergency monitoring device based on the side-view camera of an unmanned aerial vehicle provided by the present invention can implement all the method steps implemented in the above-mentioned reservoir area landslide emergency monitoring method embodiment based on the side-view camera of an unmanned aerial vehicle, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those of the method embodiment will not be described in detail here.

[0153] Fig. 9 is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as Fig. 9As shown, the electronic device may include: a processor (processor) 910 , a communication interface (Communications Interface) 920 , a memory (memory) 930 and a communication bus 940 , wherein the processor 910 , the communication interface 920 , and the memory 930 communicate with each other through the communication bus 940 . The processor 910 can call the logic instructions in the memory 930 to execute the reservoir area landslide emergency monitoring method based on the side-view camera of the unmanned aerial vehicle, the method comprising: obtaining at least one exposure image taken by the side-view camera group in the fixed-wing unmanned aerial vehicle, wherein the side-view camera group includes multiple zero-tilt stitching cameras; according to the established texture features, a first-level landslide identification is performed based on the central camera image of the at least one exposure image through multiple preset models to obtain a candidate landslide image group and a candidate exposure position list, wherein the central camera image is obtained by taking the central position camera of the side-view camera group; based on the at least one exposure image, a post-phase digital surface model of each candidate landslide image in the candidate landslide image group is constructed; a pre-constructed front-phase digital surface model corresponding to the candidate exposure position list is obtained; a second-level landslide identification is performed based on the post-phase digital surface model and the front-phase digital surface model to obtain a target landslide area, wherein the elevation difference value between the post-phase digital surface model and the front-phase digital surface model of the target landslide area is greater than the elevation difference threshold; and a three-dimensional spatial volume change of the ground object in the target landslide area is determined as the landslide volume of the target landslide area.

[0154] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0155] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the reservoir area landslide emergency monitoring method based on the side-view camera of an unmanned aerial vehicle provided by the above-mentioned methods. The method includes: obtaining at least one exposure image taken by a side-view camera group in a fixed-wing unmanned aerial vehicle, wherein the side-view camera group includes a plurality of zero-tilt stitched cameras; according to the established texture features, a first-level landslide recognition is performed based on the central camera image of at least one exposure image through a plurality of preset models to obtain a candidate landslide image; An image group and a list of candidate exposure positions, wherein the central camera image is taken by the central position camera of the side view camera group; based on at least one exposure image, a back-phase digital surface model of each candidate landslide image in the candidate landslide image group is constructed; a pre-constructed front-phase digital surface model corresponding to the candidate exposure position list is obtained; a second-level landslide recognition is performed based on the back-phase digital surface model and the front-phase digital surface model to obtain a target landslide area, wherein the elevation difference value between the back-phase digital surface model and the front-phase digital surface model of the target landslide area is greater than an elevation difference threshold; and a three-dimensional spatial volume change of the object in the target landslide area is determined as the landslide volume of the target landslide area.

[0156] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the processor executes the method for emergency monitoring of landslides in a reservoir area based on a side-view camera of an unmanned aerial vehicle provided by the above methods, the method comprising: obtaining at least one exposure image taken by a side-view camera group in a fixed-wing unmanned aerial vehicle, wherein the side-view camera group comprises a plurality of stitched cameras with zero inclination; performing first-level landslide identification based on a central camera image of at least one exposure image through a plurality of preset models according to established texture features, and obtaining a candidate landslide image group and a list of candidate exposure positions, wherein, The central camera image is obtained by taking pictures by the central position camera of the side-view camera group; based on at least one exposure image, a back-phase digital surface model of each candidate landslide image in the candidate landslide image group is constructed; the pre-constructed front-phase digital surface model corresponding to the candidate exposure position list is obtained; the second-level landslide recognition is performed based on the back-phase digital surface model and the front-phase digital surface model to obtain the target landslide area, wherein the elevation difference value between the back-phase digital surface model and the front-phase digital surface model of the target landslide area is greater than the elevation difference threshold; the three-dimensional spatial volume change of the object in the target landslide area is determined as the landslide volume of the target landslide area.

[0157] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0158] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A reservoir area landslide emergency monitoring method based on a UAV side-view camera, characterized in that: include: Acquire at least one exposure image captured by a side-view camera group in a fixed-wing drone, wherein the side-view camera group includes a plurality of zero-tilt stitched cameras; According to the established texture features, a first-level landslide identification is performed based on the central camera image of the at least one exposure image through multiple preset models to obtain a candidate landslide image group and a candidate exposure position list, wherein the central camera image is taken by a camera at the central position of the side view camera group; Based on the at least one exposure image, constructing a post-temporal digital surface model of each candidate landslide image in the candidate landslide image group; Acquire a pre-constructed previous phase digital surface model corresponding to the candidate exposure position list; Performing second-level landslide identification based on the latter-phase digital surface model and the former-phase digital surface model to obtain a target landslide area, wherein an elevation difference value between the latter-phase digital surface model and the former-phase digital surface model of the target landslide area is greater than an elevation difference threshold; The three-dimensional spatial volume change of the ground object in the target landslide area is determined as the landslide volume of the target landslide area.

2. The reservoir area landslide emergency monitoring method based on the side-view camera of an unmanned aerial vehicle according to claim 1 is characterized in that: The step of acquiring at least one exposure image captured by a side-view camera group in the fixed-wing UAV comprises: Invoking a fixed-wing UAV to fly a single route, and using a side-view camera group located on the side of the fuselage of the fixed-wing UAV, when the fixed-wing UAV is at a preset target exposure point, collecting ground image of the target location to obtain at least one exposure image; The side-view camera group includes a target number of fixed-position cameras, and the lens spacing between the target number of fixed-position cameras is less than a preset lens spacing threshold.

3. The reservoir area landslide emergency monitoring method based on the side-view camera of an unmanned aerial vehicle according to claim 2 is characterized in that: The target exposure points include a front exposure point, a middle exposure point and a rear exposure point, and the step of constructing a rear-phase digital surface model of each candidate landslide image in the candidate landslide image group based on the at least one exposure image includes: Determine a first exposure image of the fixed-wing drone at the front exposure point, a second exposure image of the fixed-wing drone at the middle exposure point, and a third exposure image of the fixed-wing drone at the rear exposure point; Determine pixels corresponding to the target location in the first exposure image, the second exposure image, and the third exposure image as a repeated coverage area; Based on the repeated coverage area, a post-temporal digital surface model of each candidate landslide image in the candidate landslide image group is constructed.

4. The reservoir area landslide emergency monitoring method based on the side-view camera of an unmanned aerial vehicle according to claim 2 is characterized in that: After constructing the post-temporal digital surface model of each candidate landslide image in the candidate landslide image group based on the at least one exposure image, the method further includes: When there is no pre-constructed previous phase digital surface model in the candidate exposure position list, determining a candidate landslide area boundary of the candidate landslide image based on the candidate exposure position list to obtain a candidate landslide area; Acquire a target post-phase digital surface model corresponding to the candidate landslide area; Based on the grid size of the target post-phase digital surface model, gridding the candidate landslide area to obtain a grid landslide area; Simulating a previous phase digital surface model of the candidate landslide area based on the simulated elevation value of each row of grids in the grid landslide area; Simulating a current phase digital surface model of the candidate landslide area based on the average elevation value of each grid in the grid landslide area according to a boundary growth method; Based on the comparison and analysis between the previous phase digital surface model and the current phase digital surface model, the elevation change difference of the candidate landslide area is obtained; When the elevation change difference of the candidate landslide area is greater than the elevation change threshold, the candidate landslide area is taken as the target landslide area.

5. The reservoir area landslide emergency monitoring method based on the side-view camera of an unmanned aerial vehicle according to claim 4 is characterized in that: Before simulating the previous phase digital surface model of the candidate landslide area based on the simulated elevation values ​​of each row of grids in the grid landslide area, the method further includes: Based on the target post-phase digital surface model, the upper grid elevation, the lower grid elevation, the first grid elevation and the last grid elevation of each row of grids in the grid landslide area are obtained; The average value among the upper grid elevation, the lower grid elevation, the first grid elevation and the last grid elevation is used as the simulated elevation value of each row of grids in the grid landslide area.

6. The reservoir area landslide emergency monitoring method based on a UAV side-view camera according to claim 4 is characterized in that: Before simulating the current phase digital surface model of the candidate landslide area based on the average elevation value of each grid in the grid landslide area according to the boundary growth method, the method further includes: Each grid in the grid landslide area is used as a grid to be estimated, and the following steps are performed to obtain the average elevation value of the grid to be estimated: Based on the target post-phase digital surface model, obtaining grid elevation values ​​of four neighboring regions of the grid to be estimated; Based on the grid elevation values ​​of the four neighboring regions of the grid to be estimated, determine the average elevation value of the grid to be estimated: in, represents the average elevation value of the grid to be estimated, represents the elevation value of the grid to the right of the grid to be estimated, represents the elevation value of the grid immediately to the left of the grid to be estimated, represents the elevation value of the grid directly above the grid to be estimated, represents the elevation value of the grid directly below the grid to be estimated, Indicates the number of grids involved in the calculation.

7. A reservoir area landslide emergency monitoring device based on a UAV side-view camera, characterized in that: include: An acquisition module, used to acquire at least one exposure image captured by a side-view camera group in a fixed-wing UAV, wherein the side-view camera group includes a plurality of zero-tilt stitching cameras; A first recognition module is used to perform first-level landslide recognition based on the central camera image of the at least one exposure image through multiple preset models according to the established texture features, and obtain a candidate landslide image group and a candidate exposure position list, wherein the central camera image is obtained by taking a picture of a central position camera of the side view camera group; A construction module, used for constructing a post-temporal digital surface model of each candidate landslide image in the candidate landslide image group based on the at least one exposure image; The acquisition module is further used to acquire a pre-constructed front-phase digital surface model corresponding to the candidate exposure position list; a second identification module, configured to perform second-level landslide identification based on the latter-phase digital surface model and the former-phase digital surface model to obtain a target landslide area, wherein an elevation difference value between the latter-phase digital surface model and the former-phase digital surface model of the target landslide area is greater than an elevation difference threshold; The determination module is used to determine the three-dimensional spatial volume change of the ground object in the target landslide area as the landslide volume of the target landslide area.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the reservoir area landslide emergency monitoring method based on the unmanned aerial vehicle side-view camera as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the method for emergency monitoring of reservoir landslides based on a side-view camera of an unmanned aerial vehicle as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by the processor, the method for emergency monitoring of reservoir landslides based on a side-view camera of an unmanned aerial vehicle as described in any one of claims 1 to 6 is implemented.

Images