Method for detecting changes in urban buildings based on drone videos and three-dimensional models

Abstract

The application discloses a kind of urban building change detection methods based on unmanned aerial vehicle video and three-dimensional model, comprising: obtaining the multi-period unmanned aerial vehicle aerial video data of target area to the urban building base three-dimensional model of target area as benchmark, to the multi-period unmanned aerial vehicle aerial video data is carried out uniform frame extraction processing, generates multi-period urban building three-dimensional model;The urban building base three-dimensional model is based on the unmanned aerial vehicle oblique photography data of target area and constructs;Based on the multi-period urban building three-dimensional model to generate the dense point cloud of multi-period urban building;To the dense point cloud of the multi-period urban building is carried out change detection, extracts urban building change map spot.The present application solves the problem that the existing unmanned aerial vehicle oblique photography reconstruction three-dimensional model is high in cost, slow in updating, cannot carry out fast urban building change detection.

Description

Technical Field

[0001] This invention relates to the field of remote sensing for detecting changes in urban buildings, and specifically to a method for detecting changes in urban buildings based on UAV video and 3D models. Background Technology

[0002] Urban architectural changes are a significant reflection of urban development and are crucial for urban planning and refined management. To improve the level of refined urban management, dynamic monitoring of urban architectural changes is necessary. By dynamically monitoring these changes, we can accurately grasp the spatial location, structural form, and external characteristics of buildings, and promptly identify irrational phenomena in the use of urban resources, such as improper land use, excessive building density, and illegal construction. This provides data support for optimizing resource allocation and adjusting urban development strategies, offering managers multi-faceted, comprehensive, three-dimensional, multi-temporal, and highly accurate intuitive architectural information, and providing decision support for improving urban safety and optimizing urban resource allocation.

[0003] Traditional urban building change detection mostly utilizes satellite or aerial high-resolution remote sensing imagery, which can identify the spatial distribution of buildings and some detailed information. Urban building change detection based on high-resolution remote sensing imagery has the advantages of low image acquisition cost and high frequency, allowing for quarterly or monthly monitoring. However, limitations due to image tilt and building obstruction mean that only changes at the top floor can be identified, while many suspected changes at the ground level and near the ground cannot be detected. Furthermore, high-resolution remote sensing imagery can only roughly pinpoint the location of suspected changes, not distinguish the type of change.

[0004] The method of reconstructing 3D models based on UAV oblique photogrammetry has the most comprehensive and numerous detections of suspected changes. The model is less affected by occlusion when screening for suspected changes, and it is clearer and more intuitive than satellite imagery. It can also identify the layer number and material of suspected changes. However, the cost of reconstructing 3D models based on UAV oblique photogrammetry is high, making it difficult to conduct high-frequency routine monitoring. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for detecting urban building changes based on UAV video and 3D models. This method aims to solve the problems of high cost and slow update of UAV oblique photogrammetry reconstruction of 3D models, which makes it impossible to quickly detect urban building changes.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] Urban building change detection methods based on UAV video and 3D models include:

[0008] Acquire multiple drone aerial video data of the target area

[0009] Using the 3D model of the urban building base in the target area as a reference, the multi-period UAV aerial video data is subjected to uniform frame extraction processing to generate a multi-period 3D model of the urban building base; the 3D model of the urban building base is constructed based on the UAV oblique photogrammetry data of the target area.

[0010] Based on the aforementioned multi-phase urban building 3D model, a dense point cloud of multi-phase urban buildings is generated;

[0011] Change detection is performed on the dense point cloud of urban buildings in the multiple phases, and urban building change patches are extracted.

[0012] Optionally, a semi-global matching algorithm is used to detect changes in the dense point cloud of urban buildings in the multi-phase period. The dense point cloud is used as the data source, and the elevation difference or Euclidean distance between the two time phase data is used as the change measure. Then, threshold segmentation is used to extract the changed area.

[0013] Optionally, clustering can be used to process the changed regions.

[0014] Optionally, the semi-global matching algorithm includes a matching cost calculation to measure the similarity of each pair of pixels in two viewpoint images, using the squared difference cost as the cost metric, calculated as follows:

[0015] C(d)=(I L (x,y)-IR(xd,y)) 2

[0016] Where C(d) is the degree of difference between the left and right images under disparity d; I L (x,y) and IR(xd,y) represent the element values ​​of the left and right images at position (x,y), respectively;

[0017] The formula for cost aggregation is as follows:

[0018]

[0019] Where: C(p,d) is the matching cost of point p under disparity d, N(x,y) is the neighborhood of point (x,y), D(p,x,y) is the disparity difference measure between point p and point (x,y), and λ is the weight parameter that controls the influence of the smoothing term.

[0020] Optionally, after cost aggregation, the disparity value of each pixel in the disparity map is determined by selecting the disparity with the minimum cost. The disparity calculation formula is as follows:

[0021] d(x,y)=argmin d C agg (x,y,d)

[0022] Among them, C agg(x, y, d) is the cost function after cost aggregation, and the disparity value d(x, y) corresponds to the disparity with the minimum cost.

[0023] Optionally, before performing uniform frame extraction on the multi-period drone aerial video data, the process further includes: using Python and OpenCV to perform uniform frame division and save the multi-period drone aerial video data, determining the number of frames to be extracted, and correcting the distortion of the extracted images according to camera distortion parameters, so as to preprocess the multi-period drone aerial video data.

[0024] Optionally, before performing uniform frame subtraction on the multi-phase UAV aerial video data, the process further includes: further processing the preprocessed multi-phase UAV aerial video data, including: removing images that do not meet the repetition requirements; filtering frame images and removing frames that do not meet the requirements; and checking the occlusion of buildings by temporary obstacles in the frame photos.

[0025] Optionally, the three-dimensional model of the urban building foundation of the target area is constructed in the following manner:

[0026] Image pair selection: Acquire UAV oblique photography data of the target area, select several image pairs, perform feature matching, establish spatial relationships between images, obtain matched point cloud image pairs, and generate preliminary 3D point cloud data.

[0027] Point cloud matching and densification: Extract 3D coordinate points from the matched point cloud image pairs, and use photometric consistency and multi-view geometric constraints to achieve point cloud densification based on the sparse point cloud obtained by stereo matching.

[0028] Triangulation construction and optimization: Generating 3D meshes based on point cloud data to provide a structural foundation for 3D models;

[0029] Texture mapping: Mapping actual acquired image data onto a 3D mesh to generate a 3D model of the city building base with a realistic appearance.

[0030] Optionally, the step of using a 3D model of the urban building base in the target area as a reference to perform uniform frame-sampling processing on the multi-phase drone video data to generate a multi-phase urban building 3D model includes:

[0031] By fusing multi-angle inspection video frame-by-frame photos from multiple phases of unmanned aerial vehicle (UAV) video data with GPS / IMU data from photos of 3D models of urban building foundations, geographic alignment of the images can be achieved.

[0032] Using data from the 3D model of the urban building foundation, feature mapping is assisted in the reconstruction of the 3D model of the uniformly divided frame of the inspection video.

[0033] Based on the alignment results of the 3D model data of urban building foundations, the camera intrinsic and extrinsic parameters of the drone video data are optimized;

[0034] A multi-view stereo matching algorithm is used to convert multi-angle inspection video frame photos into new point clouds; the three-dimensional model of the urban building base is used as a constraint to ensure that the new point cloud data is consistent with the geometry of the three-dimensional model of the urban building base.

[0035] Optionally, before performing change detection on the dense point cloud of the multi-phase urban buildings, the method further includes:

[0036] A voxel meshing method is used to map point cloud data onto a uniform 3D mesh for downsampling and noise reduction of dense point clouds.

[0037] Compared with the prior art, the advantages of this invention are as follows:

[0038] This invention first utilizes regular drone patrols at airports to acquire multi-period drone video data. Then, using an existing refined 3D model of urban building foundations as a benchmark, it performs uniform frame extraction on the multi-period drone video data to rapidly build 3D models based on the video images, generating multi-period 3D building models. Finally, it uses the 3D models to generate dense point clouds of urban buildings, and employs a semi-global matching algorithm (SGM) to detect changes in the dense point clouds from different periods, extracting urban building change patches, providing data support for improving urban planning, management, and supervision. Compared to previous 3D models generated using oblique photogrammetry, although the accuracy of 3D models generated from aerial video is slightly reduced, the cost is lower, the update speed is faster, and the main 3D features of urban buildings are preserved, meeting the requirements for urban building change detection. Rapidly reconstructing 3D models from drone video and extracting urban building change patches enables regular and effective change detection of urban buildings, better meeting the needs of actual operational production. Attached Figure Description

[0039] Figure 1 A flowchart of a method for detecting urban building changes based on UAV video and 3D models provided in an embodiment of this application;

[0040] Figure 2 Flowchart for voxelization of building point clouds and assignment of voxel mesh values;

[0041] Figure 3 This is a map showing the data collection area in an application scenario example.

[0042] Figure 4 This is a schematic diagram of an aerial oblique photography flight of a drone in an application scenario example;

[0043] Figure 5Frame-by-frame imagery from drone aerial video in an application scenario example;

[0044] Figure 6 The result image of a 3D reconstruction model of the urban building foundation in an application scenario example;

[0045] Figure 7 To quickly reconstruct 3D models from drone videos in application scenarios;

[0046] Figure 8 This refers to non-permanent building changes in application scenarios;

[0047] Figure 9 This refers to permanent building changes in application scenarios. Detailed Implementation

[0048] Example:

[0049] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0050] See Figure 1 As shown, the urban building change detection method based on UAV video and 3D model provided in this embodiment mainly includes the following steps:

[0051] Step 110: Acquire multiple phases of drone video data of the target area;

[0052] In practice, drones are used to conduct regular patrols of urban buildings within the monitoring area. These drones are equipped with multi-lens digital aerial cameras to capture multi-view videos of the buildings. The drone aerial video capture process mainly involves: dividing the monitoring area into regular phases, determining the takeoff point for regular patrols, designing flight paths within the monitoring area, setting instrument parameters for the patrol mission, checking the data acquisition equipment, acquiring the video data, and organizing and backing up the video data. To ensure the accuracy of the 3D model being inspected and to achieve rapid inspection, the heading and lateral overlap requirements of the acquired image data must be fully met. Furthermore, due to the limitations of drone endurance, an appropriate flight altitude must be selected. Finally, the data acquisition mission must be conducted in optimal weather conditions—overcast, fog-free, and without strong winds—to ensure uniform lighting in the urban building videos, eliminate color differences caused by overexposure, and achieve good image quality.

[0053] After acquiring multi-phase drone video data of the target area, Python and OpenCV were used to evenly divide the video into frames and save them, determining the extraction interval while ensuring image quality. Finally, distortion correction was performed on the extracted frames based on camera distortion parameters, completing the preprocessing of the inspection data and providing a data foundation for the subsequent construction of multi-phase urban building 3D models.

[0054] Step 120: Using the 3D model of the urban building base in the target area as a reference, perform uniform frame extraction on the multi-phase drone video data to generate a multi-phase urban building 3D model; the urban building base 3D model is constructed based on the drone oblique photography data of the target area;

[0055] In this step, rapid 3D modeling is performed based on multi-phase drone video data to generate multi-phase urban building 3D models. Compared with the previous 3D models generated using oblique photogrammetry, although the accuracy of 3D models generated from aerial video is reduced, the cost is lower and the update speed is faster. At the same time, since the urban building base 3D model is constructed based on the drone oblique photogrammetry data of the target area, the main 3D features of urban buildings are preserved by using the existing refined urban building base 3D model as a benchmark, which can meet the requirements of urban building change detection.

[0056] In the specific implementation, the pre-processed multi-period drone video data is further processed to remove photos with excessive repetition and reduce redundant image frames in order to shorten the modeling time; framed images are screened to remove frames with unsatisfactory imaging in order to improve modeling efficiency and accuracy; and the occlusion of buildings by temporary obstacles in the framed photos is checked to avoid the impact of temporary obstacles on the imaging of the 3D model.

[0057] After further processing of multiple drone video data, a 3D model of the city's building foundation and its original photographs were used as the basis for reconstructing the 3D model from the evenly divided frame photographs of the inspection video, including:

[0058] ① The GPS / IMU data of multi-angle inspection video frame photos from multiple phases of unmanned aerial vehicle video data are fused with the 3D model photos of urban building foundations to achieve geographic alignment of the images;

[0059] ② Using the 3D model data of urban building foundations, SIFT, SURF and other methods are used to assist the feature mapping work in the reconstruction process of the 3D model of the uniformly divided video frame, so as to improve the accuracy of the 3D model reconstruction of the video frame uniformly divided frame.

[0060] ③ Optimize camera pose and intrinsic parameters: Based on the alignment results of the existing 3D model of urban building base, further optimize the camera intrinsic parameters (focal length, principal point position, distortion coefficient, etc.) and extrinsic parameters (camera position, pose, etc.) of the UAV video data. Utilize the geometric characteristics of the existing 3D model of urban building base to perform camera bundle adjustment, reduce model reprojection error, and improve the accuracy of the rapidly reconstructed 3D model.

[0061] ④ Point cloud density optimization: The multi-view stereo matching (MVS) algorithm is used to convert the evenly divided frame photos of the multi-angle inspection video into a new, denser point cloud. The existing refined 3D model of the urban building base can be used as an optimization constraint to ensure that the new point cloud data matches its geometry.

[0062] In this way, useful information can be extracted from the dynamic video data of drones in a short time through the above operations, and then a 3D model can be generated, which is especially suitable for scenarios of real-time monitoring, rapid evaluation and large-scale data collection.

[0063] Specifically, the three-dimensional model of the city's building foundation is constructed in the following directions:

[0064] Acquiring UAV oblique photogrammetry data: UAVs equipped with high-definition lenses are used for aerial surveying to capture high-resolution oblique photograms, which are then used to construct a 3D model based on the UAV oblique photogrammetry, serving as a reference benchmark. The main steps of UAV aerial surveying include: determining the monitoring area, applying for airspace in the target area, dividing the aerial survey area, pre-planning the flight route, selecting the takeoff point, making secondary adjustments to the flight route, checking flight equipment, conducting aerial surveys, preprocessing aerial survey data, and organizing and backing up the oblique photogrammetry images.

[0065] The acquired oblique UAV photographs and aerial triangulation results were used to create a 3D model of the urban building foundation in the target area. This process included image pair selection, point cloud matching, triangulation construction and optimization, and texture mapping, ultimately generating a refined 3D model of the urban building foundation. The specific implementation process is as follows:

[0066] (1) Image pair selection: Select suitable image pairs from multiple images, use feature extraction algorithms in computer vision (such as SIFT, SURF or ORB) and perform feature matching to establish the spatial relationship between images, thereby generating preliminary three-dimensional point cloud data.

[0067] (2) Point cloud matching and densification: Extract three-dimensional coordinate points from the matched point cloud image pairs, and use photometric consistency and multi-view geometric constraints to achieve point cloud densification based on the sparse point cloud obtained by stereo matching.

[0068] (3) Triangulation construction and optimization: Triangular meshes are generated based on point cloud data to provide a structural foundation for the 3D model. First, the point cloud data is processed using the Delaunay triangulation algorithm to generate non-intersecting triangular patches; then, the Poisson reconstruction method is used to fill any holes in the point cloud and optimize the surface quality of the point cloud; finally, the surface quality of the 3D model is optimized through mesh refinement and surface smoothing techniques.

[0069] (4) Texture mapping: The actual acquired image data is mapped onto a 3D mesh to generate a 3D model with a realistic appearance. By calculating the UV coordinates of each triangular facet, the image texture is mapped onto the corresponding 3D model surface; in order to avoid the influence of the image's lighting and color differences on the texture, lighting and color consistency processing is required to ensure a natural transition of the texture.

[0070] In this way, a detailed three-dimensional model of the urban building foundation can be generated through the above operations.

[0071] Step 130: Generate a dense point cloud of urban buildings from the multi-phase urban building 3D model.

[0072] Since dense point clouds typically contain a large number of points with uneven density and noise, this affects the extraction efficiency and quality of key points. Therefore, this method also employs a voxel meshing approach to map the point cloud data onto a uniform 3D mesh, thereby eliminating the effects of uneven density, reducing redundant data, and improving processing speed.

[0073] Step 140: Perform change detection on the dense point cloud of the multi-phase urban buildings and extract the urban building change patches.

[0074] In the specific implementation, after downsampling and denoising the dense point cloud, a semi-global matching algorithm (SGM) is used to perform stereo matching based on a single geometric information. The dense point cloud is used as the data source, and the elevation difference or Euclidean distance between two temporal data is used as the change measure. Then, threshold segmentation is used to extract the change region. In order to further improve the detection accuracy of the change region, this method uses clustering to refine the change region.

[0075] The SGM algorithm mainly consists of four steps: matching cost calculation, cost aggregation, disparity calculation, and disparity optimization. The matching cost measures the similarity between each pair of pixels in two viewpoint images. This method uses the squared difference cost (SSD) as the cost metric, and the calculation method is as follows:

[0076] C(d)=(I L (x,y)-IR(xd,y)) 2

[0077] Where C(d) represents the degree of difference between the left and right images under disparity d; the smaller the value, the more similar the match. L (x,y) and IR(xd,y) represent the element values ​​of the left and right images at position (x,y), respectively;

[0078] The goal of cost aggregation is to smooth the matching cost by incorporating neighborhood information, thereby enhancing the algorithm's robustness and reducing the impact of noise on disparity estimation. SGM employs a semi-global approach to cost aggregation, meaning that for each pixel, it performs cost aggregation along multiple directions (such as horizontal, vertical, and diagonal directions). The formula for cost aggregation is as follows:

[0079]

[0080] Where: C(p,d) is the matching cost of point p under disparity d, N(x,y) is the neighborhood of point (x,y), D(p,x,y) is the disparity difference measure between point p and point (x,y), and λ is the weight parameter that controls the influence of the smoothing term.

[0081] After cost aggregation, the disparity value of each pixel in the disparity map is determined by selecting the disparity with the minimum cost. Specifically, the formula for disparity calculation is:

[0082] d(x,y)=argmin d C agg (x,y,d)

[0083] Among them, C agg (x, y, d) is the cost function after cost aggregation, and the disparity value d(x, y) corresponds to the disparity with the minimum cost.

[0084] The goal of disparity optimization is to reduce noise in the disparity map, resulting in a smoother and more accurate map. To achieve this, SGM employs an optimization strategy based on local smoothing terms and global consistency. After these steps, pixel-level disparity is obtained. Then, quadratic curve interpolation is performed to obtain sub-pixel-level disparity: a quadratic curve is fitted to the optimal disparity and the cost values ​​of the two disparities before and after the optimal disparity; the disparity corresponding to the extreme point of the curve is the sub-pixel-level disparity.

[0085] Finally, the detected areas of building land change were overlaid with the images of the 3D reconstruction model of the urban building foundation to extract urban building change patches and establish a ledger. The timeline of the application materials for each change area was compared with the change time detected in the aerial imagery. The time attributes of the application materials were combined with the time judgment of the change area results to obtain a table of the actual change phases. The table includes attributes such as serial number, residential area name, application time, change time, and description of the change area.

[0086] In summary, this method first acquires multiple phases of aerial video data of urban buildings. Then, using a refined 3D model of the urban building base as a benchmark, it performs uniform frame extraction on the multiple phases of video data to quickly build 3D models based on UAV aerial video images, generating multiple phases of 3D building models. Finally, it uses the multiple phases of 3D building models to generate dense point clouds of urban buildings, detects changes in the dense point clouds at different times, and extracts urban building change patches, providing data support for improving the level of urban planning, management, and supervision. This solves the problems of high cost, slow update speed, and inability to quickly detect changes in urban buildings in existing UAV oblique photogrammetry reconstruction 3D models.

[0087] The following application scenario example will be used to further illustrate this method:

[0088] The monitoring target is a certain area in Danzao Town, Nanhai District, Foshan City.

[0089] Data acquisition and preprocessing steps include:

[0090] UAV oblique photography: In accordance with the technical requirements of UAV oblique photography, image control points are set up in the mission area, and then oblique aerial photography and image control measurement are carried out respectively.

[0091] Drone aerial video capture: The takeoff point was determined, a multi-angle building photo acquisition flight path was planned, the inspection flight route was established, and instruments were checked. A drone equipped with a high-resolution optical camera was used to collect quarterly community inspection videos. Data was saved and backed up. Python and OpenCV were used to evenly divide the video into frames and save them, ensuring image quality while determining the extraction interval. Finally, distortion correction was performed on the extracted frames based on camera distortion parameters, completing the inspection data preprocessing and providing a data foundation for subsequently constructing a 3D model of the city's periodic building inspections.

[0092] Steps for constructing a 3D model of urban building foundations based on UAV oblique photogrammetry:

[0093] The oblique imagery data from the UAV is preprocessed, and the image control data is simultaneously solved and organized. Utilizing the oblique aerial photography results and image control measurement results, a 3D model is automatically generated through processes such as aerial triangulation, dense matching, automatic modeling, and texture mapping. This model serves as the data base for benchmark correction.

[0094] (1) Aerial triangulation

[0095] The system incorporates oblique images and ground control point data into the automated modeling software system. The control points and images are then manually correlated. The automated modeling software then uses bundle adjustment for the overall regional network adjustment. Each image is used as a light ray as an adjustment unit, and the collinearity equation of the central projection is used as the basic equation of the adjustment unit. By rotating and translating each light ray in space, the common light rays between the models are optimally intersected, and the entire region is optimally added to the control point coordinate system, thereby restoring the spatial positional relationship between ground features.

[0096] Meanwhile, by using the coordinate files of connection points and control points, combined with GNSS / IMU information, the self-calibration area network adjustment iterative calculation is realized. Through repeated joint calculations, systematic and random errors in the external orientation are eliminated and reduced, providing a mathematical accuracy basis for the production of 3D models.

[0097] (2) Puncture point

[0098] To improve the accuracy of aerial triangulation, control points can be manually selected from the imagery to be marked. These points are then compared with control points obtained from external field measurements to determine their precise locations within the imagery. After adding the control points, the aerial triangulation is run again to check for success. If it fails, check for errors in the marking or POS data until encryption is successful.

[0099] (3) Dense matching and 3D model construction

[0100] Aerial triangulation is used to obtain the exterior orientation elements of the images, determine their position in three-dimensional space, and construct stereo pairs. Multi-view image dense matching technology is employed to identify and match feature points from multiple images, generating dense point cloud data. Based on this matched point cloud data, a triangular mesh model is generated.

[0101] (4) Texture bonding

[0102] The collected UAV images are cut into corresponding textured triangles according to the triangles in the 3D mesh model, and a correspondence is established with the 3D mesh model. The 2D planar textured triangles are mapped onto the 3D mesh model to complete the texture mapping, automatically generate the regional building 3D model, and check the model accuracy.

[0103] The steps for quickly generating multi-stage 3D models of urban buildings include:

[0104] (1) Further process the video photos collected by the preprocessed drone aerial video, remove photos with excessive repetition, filter frame images, and check the occlusion of buildings by temporary obstacles in the frame photos.

[0105] (2) Based on the 3D model of the urban building base and its original photos, the 3D model is reconstructed by the uniformly divided frame photos of the inspection video: ① GPS / IMU data of the uniformly divided frame photos of the inspection video and the 3D model photos of the urban building base are fused to achieve geographic alignment of the images; ② SIFT, SURF and other methods are used to assist the feature mapping work in the reconstruction process of the 3D model of the uniformly divided frame photos of the inspection video with the help of the 3D model data of the urban building base to improve the accuracy of the 3D model reconstructed by the uniformly divided frame photos of the video; ③ Optimize camera attitude and intrinsic parameters: Based on the alignment results of the 3D model of the urban building base, the intrinsic parameters (focal length, principal point position, distortion coefficient, etc.) and extrinsic parameters (camera position, attitude, etc.) of the UAV video data are further optimized. The camera bundle adjustment is performed by utilizing the geometric characteristics of the 3D model of the urban building base to reduce the reprojection error of the model and improve the accuracy of the rapidly reconstructed 3D model; ④ Point cloud density optimization: The multi-view stereo matching (MVS) algorithm is used to convert the uniformly divided frame photos of the inspection video from multiple angles into a denser point cloud. Existing detailed 3D models of urban building foundations can serve as constraints for optimization, ensuring that new point cloud data matches their geometry.

[0106] The above operations can be used to quickly generate multi-stage 3D models of urban buildings.

[0107] The steps for 3D model change detection based on dense point clouds include:

[0108] Dense point clouds are generated using the produced multi-phase urban building 3D models. A voxel grid filter algorithm is used to denoise the dense point cloud data, removing unnecessary noise. Based on the dense point cloud data of the urban building base 3D model and the 3D model reconstructed from multiple phases of inspection video by uniform frame division, a semi-global matching (SGM) algorithm is used to detect changes in the dense point cloud at different times. After steps such as matching cost calculation, cost aggregation, disparity calculation, and disparity optimization, pixel-level disparity is obtained. Then, a quadratic curve fitting is performed on the optimal disparity and the cost of the two disparities before and after the optimal disparity to obtain sub-pixel-level disparity, thereby quickly and automatically extracting changed areas in urban buildings.

[0109] The residential area investigation ledger mainly involves creating map data for suspected problematic residences and adding the ledger information to the map attribute data. Using the residential community as the basic unit, statistics are compiled on the ledger's building type, building material, and type of illegal construction, and the changes in suspected buildings in each community are analyzed.

[0110] The above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the content of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting urban building changes based on UAV video and 3D models, characterized in that, include: Acquire multiple drone aerial video data of the target area; Using the three-dimensional model of the urban building base in the target area as a reference, the multi-phase UAV aerial video data is subjected to uniform frame extraction processing to generate multi-phase urban building three-dimensional models. The three-dimensional model of the urban building foundation is constructed based on UAV oblique photogrammetry data of the target area; Based on the aforementioned multi-phase urban building 3D model, a dense point cloud of multi-phase urban buildings is generated; Change detection is performed on the dense point cloud of urban buildings in the multiple phases, and urban building change patches are extracted; A semi-global matching algorithm is used to detect changes in the dense point cloud of urban buildings in multiple phases. The dense point cloud is used as the data source, and the elevation difference or Euclidean distance between two time phases is used as the change measure. Then, threshold segmentation is used to extract the changed area. The process involves using a 3D model of the urban building base in the target area as a reference, and performing uniform frame-sampling processing on the multi-phase drone video data to generate a multi-phase 3D urban building model, including: By fusing multi-angle inspection video frame-by-frame photos from multiple phases of unmanned aerial vehicle (UAV) video data with GPS / IMU data from photos of 3D models of urban building foundations, geographic alignment of the images can be achieved. Using data from the 3D model of the urban building foundation, feature mapping is assisted in the reconstruction of the 3D model of the uniformly divided frame of the inspection video. Based on the alignment results of the 3D model data of urban building foundations, the camera intrinsic and extrinsic parameters of the drone video data are optimized; A multi-view stereo matching algorithm is used to convert multi-angle inspection video frame photos into new point clouds; the three-dimensional model of the urban building base is used as a constraint to ensure that the new point cloud data is consistent with the geometry of the three-dimensional model of the urban building base. Clustering is used to process the changed regions; The semi-global matching algorithm includes a matching cost calculation to measure the similarity of each pair of pixels in two viewpoint images. The squared difference cost is used as the cost metric, and the calculation method is as follows: in, It represents the degree of difference between the left and right images under the parallax d; and These represent the element values ​​at position (x, y) in the left and right images, respectively. The formula for cost aggregation is as follows: in: It is a point In parallax The matching cost below, It is a point The neighborhood, It is a point With point The measure of disparity These are weighting parameters that control the influence of the smoothing term; Before performing uniform frame extraction on the multi-phase drone aerial video data, the process further includes: using Python and OpenCV to perform uniform frame division and save the multi-phase drone aerial video data, determining the number of frames to be extracted, and correcting the distortion of the extracted images according to the camera distortion parameters, so as to preprocess the multi-phase drone aerial video data. Before performing uniform frame-drop processing on the multi-phase drone aerial video data, the process further includes: further processing the preprocessed multi-phase drone aerial video data, including: removing images that do not meet the repetition requirements; filtering frame images and removing frames that do not meet the requirements; and checking the occlusion of buildings by temporary obstacles in the frame photos. The 3D model of the urban building foundations in the target area is constructed in the following manner: Image pair selection: Acquire UAV oblique photography data of the target area, select several image pairs, perform feature matching, establish spatial relationships between images, obtain matched point cloud image pairs, and generate preliminary 3D point cloud data. Point cloud matching and densification: Extract 3D coordinate points from the matched point cloud image pairs, and use photometric consistency and multi-view geometric constraints to achieve point cloud densification based on the sparse point cloud obtained by stereo matching. Triangulation construction and optimization: Generating 3D meshes based on point cloud data to provide a structural foundation for 3D models of urban building foundations; Texture mapping: Mapping actual acquired image data onto a 3D mesh to generate a 3D model of the city building base with a realistic appearance.

2. The urban building change detection method based on UAV video and 3D model as described in claim 1, characterized in that, After cost aggregation, the disparity value of each pixel in the disparity map is determined by selecting the disparity with the minimum cost. The formula for disparity calculation is as follows: in, The cost function after cost aggregation, disparity value The parallax that corresponds to the minimum cost.

3. The urban building change detection method based on UAV video and 3D model as described in claim 1, characterized in that, Before performing change detection on the dense point cloud of the multi-phase urban buildings, the following steps are also included: A voxel meshing method is used to map point cloud data onto a uniform 3D mesh for downsampling and noise reduction of dense point clouds.

Images