Urban building change detection method based on unmanned aerial vehicle video and three-dimensional model

Abstract

The invention discloses a city building change detection method based on unmanned aerial vehicle videos and a three-dimensional model, and the method comprises the steps: obtaining multi-stage unmanned aerial vehicle aerial video data of a target region, taking a city building base three-dimensional model of the target region as a reference, and carrying out the uniform frame extraction of the multi-stage unmanned aerial vehicle aerial video data, generating a multi-stage urban building three-dimensional model; the urban building base three-dimensional model is constructed based on unmanned aerial vehicle oblique photography data of a target area; generating dense point clouds of the multi-period urban buildings based on the multi-period urban building three-dimensional model; and carrying out change detection on the dense point clouds of the multi-period urban buildings, and extracting urban building change pattern spots. The method solves the problems that the existing unmanned aerial vehicle oblique photography reconstruction of the three-dimensional model is high in cost and slow in updating, and rapid urban building change detection cannot be carried out.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing for urban building change detection, and particularly to a method for urban building change detection based on unmanned aerial vehicle (UAV) video and three-dimensional model. Background Art

[0002] Urban building change is an important reflection of urban development and is crucial for urban planning and refined management. To improve the level of urban refined management, it is necessary to dynamically monitor urban building changes. By dynamically monitoring urban building changes, accurately grasping the spatial location, structural form, and external characteristics of buildings, and timely identifying unreasonable phenomena in the use of urban resources, such as improper land use, excessive building density, illegal construction, etc., it can provide data support for optimizing resource allocation and adjusting urban development strategies, provide multi-angle, all-round, three-dimensional multi-temporal and high-precision intuitive building information for managers, and provide decision-making support for improving urban safety levels and optimizing urban resource allocation.

[0003] Most traditional urban building change detections use satellite or aerial high-resolution remote sensing images, which can identify the spatial distribution and some detailed information of buildings. Urban building change detection based on high-resolution remote sensing images has the advantages of low image acquisition cost and high frequency, and can achieve quarterly or monthly monitoring. However, due to the problem of image inclination and some building occlusions, only the top-layer changes of buildings can be identified, and many suspected changes in the ground and near-ground parts cannot be detected. Moreover, high-resolution remote sensing images can only roughly find the locations of suspected changes and cannot distinguish the types of changes.

[0004] Based on the method of reconstructing a three-dimensional model by UAV oblique photography, the most and most complete suspected changes are found. The model is less affected by occlusion when screening suspected changes, is clearer and more intuitive than satellite images, and can identify the floors and materials where the suspected changes are located. However, the cost of reconstructing a three-dimensional model by UAV oblique photography is relatively high, and it is difficult to conduct high-frequency and regular monitoring. Summary of the Invention

[0005] The purpose of the present invention is to overcome the above-mentioned deficiencies of the prior art and provide a method for urban building change detection based on UAV video and three-dimensional model, aiming to solve the problems of high cost and slow update of reconstructing a three-dimensional model by UAV oblique photography and the inability to perform rapid urban building change detection.

[0006] To achieve the above purpose, the technical solution of the present invention is:

[0007] A method for urban building change detection based on UAV video and three-dimensional model, comprising:

[0008] Obtaining multi-period UAV aerial video data of the target area

[0009] Based on the three-dimensional model of the urban building base in the target area, perform uniform frame extraction on the multi-phase UAV aerial video data to generate multi-phase three-dimensional models of urban buildings; the three-dimensional model of the urban building base is constructed based on the UAV oblique photography data of the target area;

[0010] Generate multi-phase dense point clouds of urban buildings based on the multi-phase three-dimensional models of urban buildings;

[0011] Perform change detection on the multi-phase dense point clouds of urban buildings to extract urban building change patches.

[0012] Optionally, use the semi-global matching algorithm to perform change detection on the multi-phase dense point clouds of urban buildings. Taking the dense point cloud as the data source, use the elevation difference or Euclidean distance between two temporal data as the change measure, and then use threshold segmentation to extract the change area.

[0013] Optionally, use clustering to process the change area.

[0014] Optionally, the semi-global matching algorithm includes matching cost calculation, which is used to measure the similarity of each pair of pixel points in two viewpoint images. Use the sum of squared differences cost as the cost metric, and the calculation method is as follows:

[0015] C(d) = (I L (x,y) - IR(x - d,y)) 2

[0016] where C(d) is the degree of difference between the left and right images at disparity d; I L (x,y) and IR(x - d,y) respectively represent the element values at position (x, y) in the left and right images;

[0017] The formula for cost aggregation is as follows:

[0018]

[0019] where: C(p,d) is the matching cost of point p at disparity d, N(x, y) is the neighborhood of point (x, y), D(p, x, y) is the disparity difference measure between point p and point (x, y), and λ is the weight parameter that controls the influence of the smoothing term.

[0020] Optionally, after cost aggregation, the disparity value of each pixel in the disparity map is determined by selecting the disparity with the minimum cost. The formula for disparity calculation is:

[0021] d(x,y) = argmin d C agg (x,y,d)

[0022] where C agg(x, y, d) is the cost function after cost aggregation, and the disparity value d(x, y) corresponds to the disparity with the minimum cost.

[0023] Optionally, before performing uniform frame extraction on the multi-period UAV aerial video data, the following steps are also included: using Python and OpenCV to evenly divide and save the frames of the multi-period UAV aerial video data, determining the extraction interval frames, and correcting the distortion of the extracted frames according to the camera distortion parameters to preprocess the multi-period UAV aerial video data.

[0024] Optionally, before performing uniform frame extraction on the multi-period UAV aerial video data, the following steps are also included: further processing the preprocessed multi-period UAV aerial video data, including: removing images with non-compliant repetition degrees; screening the frame-divided images and removing the non-compliant frame-divided images; checking the occlusion of the temporary obstacles in the frame-divided photos on the buildings.

[0025] Optionally, the three-dimensional model of the urban building base in the target area is constructed in the following way:

[0026] Image pair selection: Obtain the UAV oblique photography data of the target area, select several image pairs from it, perform feature matching, establish the spatial relationship between the images, obtain the matching point cloud image pairs, and generate the preliminary three-dimensional point cloud data;

[0027] Point cloud matching and densification: Extract the three-dimensional coordinate points from the matching point cloud image pairs, and use photometric consistency and multi-view geometric constraints to achieve point cloud densification based on the sparse point cloud obtained by stereo matching;

[0028] Triangulation network construction and optimization: Generate a three-dimensional grid based on the point cloud data to provide a structural basis for the three-dimensional model;

[0029] Texture mapping: Map the actually collected image data onto the three-dimensional grid to generate a three-dimensional model of the urban building base with a real appearance.

[0030] Optionally, based on the three-dimensional model of the urban building base in the target area, performing uniform frame extraction on the multi-period UAV aerial video data to generate multi-period three-dimensional models of urban buildings includes:

[0031] Fusing the GPS / IMU data of the evenly divided frame photos of the multi-angle inspection videos of the multi-period UAV aerial video data with the photos of the three-dimensional model of the urban building base to achieve geographical alignment of the images;

[0032] With the help of the data of the three-dimensional model of the urban building base, assisting the feature mapping work in the reconstruction process of the evenly divided frame three-dimensional model of the inspection video;

[0033] Optimize the internal and external parameters of the camera of the unmanned aerial vehicle video data based on the alignment result of the three-dimensional model data of the urban building base;

[0034] Convert the evenly divided frame photos of the multi-angle inspection video into a new point cloud through the multi-view stereo matching algorithm; use the three-dimensional model of the urban building base as a constraint condition to ensure that the new point cloud data is consistent with the geometric shape of the three-dimensional model of the urban building base.

[0035] Optionally, before performing change detection on the dense point cloud of the multi-phase urban buildings, it further includes:

[0036] Adopt a voxel grid processing method to map the point cloud data into a uniform three-dimensional grid for downsampling and denoising processing of the dense point cloud.

[0037] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0038] The present invention first uses the regular cruise of the unmanned aerial vehicle airport to obtain multi-phase unmanned aerial vehicle video data; then uses the existing refined three-dimensional model of the urban building base as a reference, performs even frame extraction processing on the multi-phase unmanned aerial vehicle video data, and performs rapid three-dimensional model modeling based on the video image to generate multi-phase building three-dimensional models; finally, uses the three-dimensional model to generate the dense point cloud of the urban building, and uses the semi-global matching algorithm (SGM) to perform change detection on the dense point clouds of different periods, and extracts the urban building change patches, providing data support for improving the level of urban planning management and supervision. Compared with the three-dimensional model generated by using oblique photogrammetry in the past, although the accuracy of the three-dimensional model generated based on the aerial video is reduced, the required cost is lower, the update speed is faster, and the main three-dimensional features of the urban building are retained, which can meet the requirements of urban building change detection. By quickly reconstructing the three-dimensional model through the unmanned aerial vehicle video and extracting the urban building change patches, it is possible to regularly and effectively perform change detection on urban buildings, which better meets the needs of actual operation and production. Description of the Drawings

[0039] Figure 1 It is a flowchart of the urban building change detection method based on the unmanned aerial vehicle video and the three-dimensional model provided by the embodiment of the present application;

[0040] Figure 2 It is a flowchart of the voxelization of the building point cloud and the assignment of voxel grids;

[0041] Figure 3 It is a range map of the data collection area in the application scenario example;

[0042] Figure 4 It is a schematic diagram of the unmanned aerial vehicle oblique photography flight in the application scenario example;

[0043] Figure 5It is the frame-extracted image of the UAV aerial video in the application scenario example;

[0044] Figure 6 It is the result map of the 3D reconstruction model of the urban building base in the application scenario example;

[0045] Figure 7 It is the 3D model quickly reconstructed from the UAV video in the application scenario example;

[0046] Figure 8 It is the change of non-permanent buildings in the application scenario example;

[0047] Figure 9 It is the change of permanent buildings in the application scenario example. Detailed implementation manners

[0048] Example:

[0049] The technical solution of the present invention will be further described below in conjunction with the accompanying drawings and examples.

[0050] Refer to Figure 1 As shown, the urban building change detection method based on UAV video and 3D model provided in this embodiment mainly includes the following steps:

[0051] Step 110, obtain multi-period UAV aerial video data of the target area;

[0052] In specific implementation, use a UAV airport to regularly cruise the urban buildings in the monitoring area, use a UAV to carry a multi-lens digital aerial camera to collect multi-view videos of urban buildings. The UAV aerial video mainly includes the division of the regular monitoring area, the determination of the regular cruise take-off point, the design of the monitoring area flight route, the setting of the instrument parameters for the cruise task, the inspection of the acquisition equipment for the cruise task, the collection of the regular cruise video, and the collation and backup of the cruise video data. To ensure that the accuracy of the inspection 3D model meets the standard and achieve the purpose of rapid inspection, it is necessary to fully meet the requirements of the heading and side overlap of the collected image data. And due to the limitation of the UAV endurance condition, it is necessary to select an appropriate flight altitude. Finally, it is necessary to carry out the acquisition task in the best weather of cloudy, fog-free and windless to ensure that the light of the urban building video is uniform, there is no color difference caused by overexposure, and the imaging quality is good.

[0053] After obtaining the multi-period UAV aerial video data of the target area, use Python and OpenCV to evenly divide the video into frames and save them, and determine the extraction interval frames on the premise of ensuring the image effect. Finally, correct the distortion of the frame-extracted image according to the camera distortion parameters to complete the preprocessing work of the inspection data and provide a data basis for the subsequent construction of the multi-period urban building 3D model.

[0054] Step 120: Based on the 3D model of the urban building base in the target area, perform uniform frame extraction on the multi-phase unmanned aerial vehicle video data to generate multi-phase 3D models of urban buildings. The 3D model of the urban building base is constructed based on the UAV oblique photography data of the target area.

[0055] In this step, rapid 3D model modeling is performed based on multi-phase unmanned aerial vehicle video data to generate multi-phase 3D models of urban buildings. Compared with the 3D models generated by traditional oblique photogrammetry, although the accuracy of the 3D model generated based on aerial video is reduced, the required cost is lower and the update speed is faster. At the same time, since the 3D model of the urban building base is constructed based on the UAV oblique photography data of the target area, by using the existing refined 3D model of the urban building base as a reference, the main 3D features of urban buildings are retained, which can meet the requirements of urban building change detection.

[0056] In specific implementation, the preprocessed multi-phase unmanned aerial vehicle video data is further processed to eliminate photos with too high redundancy, reduce redundant image frame division to shorten the modeling time; screen the framed images and remove the framed images with unsatisfactory imaging to improve the modeling efficiency and accuracy; check the occlusion of buildings by temporary obstacles in the framed photos to avoid the influence of temporary obstacle occlusion on 3D model imaging.

[0057] After further processing the multi-phase unmanned aerial vehicle video data, based on the 3D model of the urban building base and its original photos, assist in reconstructing the 3D model with evenly divided framed photos of the inspection video, including:

[0058] ①Fuse the GPS / IMU data of the evenly divided framed photos of the multi-angle inspection video of the multi-phase unmanned aerial vehicle video data with the photos of the 3D model of the urban building base, so as to achieve the geographical alignment of the images.

[0059] ②With the help of the 3D model data of the urban building base, use methods such as SIFT and SURF to assist the feature mapping work in the process of reconstructing the evenly divided framed 3D model of the inspection video, so as to improve the accuracy of the evenly divided framed video reconstruction of the 3D model.

[0060] ③Optimize the camera pose and internal parameters: Based on the alignment result of the existing 3D model of the urban building base, further optimize the camera internal parameters (focal length, principal point position, distortion coefficient, etc.) and external parameters (camera position, attitude, etc.) of the UAV video data, and use the geometric characteristics of the existing 3D model of the urban building base to perform bundle adjustment of the camera to reduce the reprojection error of the model and improve the accuracy of the rapidly reconstructed 3D model.

[0061] ④ Point cloud density optimization: The evenly divided frame photos of multi-angle inspection videos are converted into a denser new point cloud through the multi-view stereo matching (MVS) algorithm. The existing refined 3D model of urban building bases can be used as a constraint condition for optimization to ensure the matching of the new point cloud data with its geometric shape.

[0062] In this way, useful information can be extracted from the UAV dynamic video data in a relatively short time through the above operations, and then a 3D model can be generated, which is especially suitable for scenarios of real-time monitoring, rapid assessment, and large-scale data collection.

[0063] Specifically, the 3D model of urban building bases is constructed in the following directions:

[0064] Obtain UAV oblique photography data: Use a UAV equipped with a high-definition camera for UAV aerial survey to take high-resolution oblique photography photos for constructing a 3D model based on UAV oblique photography as a reference benchmark. The main steps of UAV aerial survey include: determining the monitoring area, applying for airspace in the target area, dividing the aerial survey area, pre-planning the flight route, selecting the take-off point, making secondary adjustments to the flight route, checking the flight equipment, conducting aerial survey, preprocessing the aerial survey data, and sorting and backing up the oblique photography photos.

[0065] With the obtained UAV oblique photography photos and the results of aerial triangulation encryption, the 3D model of urban building bases in the target area is completed. This process includes image pair selection, point cloud matching, triangulation network construction and optimization, and texture mapping, and finally generates a refined 3D model of urban building bases. The specific implementation process is as follows:

[0066] (1) Image pair selection: Select appropriate image pairs from multiple images, use feature extraction algorithms in computer vision (such as SIFT, SURF, or ORB) and perform feature matching to establish the spatial relationship between images, thereby generating preliminary 3D point cloud data.

[0067] (2) Point cloud matching and densification: Extract 3D coordinate points from the matched point cloud image pairs, and use photometric consistency and multi-view geometric constraints to achieve point cloud densification based on the sparse point cloud obtained from stereo matching.

[0068] (3) Triangulation network construction and optimization: Generate a triangular mesh based on the point cloud data to provide a structural basis for the 3D model. First, use the Delaunay triangulation algorithm to process the point cloud data to generate non-crossing triangular patches; then use the Poisson reconstruction method to fill the possible holes in the point cloud and optimize the surface quality of the point cloud; finally, optimize the surface quality of the 3D model through mesh refinement and surface smoothing techniques.

[0069] (4) Texture mapping: Map the actually collected image data onto a three-dimensional grid to generate a three-dimensional model with a realistic appearance. By calculating the UV coordinates of each triangular patch, map the image texture onto the corresponding surface of the three-dimensional model; to avoid the influence of illumination and color differences in the image on the texture, illumination and color consistency processing is required to ensure the natural transition of the texture.

[0070] In this way, a refined three-dimensional model of the urban building base can be generated through the above operations.

[0071] Step 130: Generate a dense point cloud of multi-phase urban buildings based on the multi-phase three-dimensional model of urban buildings.

[0072] Since dense point clouds usually contain a large number of points, with uneven density and noise, this affects the extraction efficiency and quality of key points in the point cloud. For this reason, this method also adopts a voxel grid processing method to map the point cloud data into a uniform three-dimensional grid, thereby eliminating the influence of uneven density, reducing redundant data, and improving the processing speed.

[0073] Step 140: Perform change detection on the dense point cloud of multi-phase urban buildings to extract urban building change patches.

[0074] In specific implementation, after downsampling and denoising the dense point cloud, use the Semi-Global Matching (SGM) algorithm to perform stereo matching based on single geometric information. Using the dense point cloud as the data source, use the elevation difference or Euclidean distance between two temporal data as the change measure, and then use threshold segmentation to extract the change area. To further improve the detection accuracy of the change area, this method uses clustering to refine the change area.

[0075] The SGM algorithm mainly includes four steps: matching cost calculation, cost aggregation, disparity calculation, and disparity optimization. Among them, the matching cost is used to measure the similarity of each pair of pixel points in two viewpoint images. This method uses the Sum of Squared Differences (SSD) as the cost metric, and the calculation method is as follows:

[0076] C(d) = (I L (x, y) - IR(x - d, y)) 2

[0077] where C(d) is the degree of difference between the left and right images at disparity d. The smaller the value, the more similar the match; I L (x, y) and IR(x - d, y) respectively represent the element values at position (x, y) in the left and right images;

[0078] The goal of cost aggregation is to smooth the matching cost by introducing neighborhood information, thereby enhancing the robustness of the algorithm and reducing the impact of noise on disparity estimation. SGM adopts a semi-global approach for cost aggregation. That is, for each pixel, it performs cost aggregation along multiple directions (such as horizontal, vertical, and diagonal directions). The formula for cost aggregation is as follows:

[0079]

[0080] where: C(p, d) is the matching cost of point p at disparity d, N(x, y) is the neighborhood of point (x, y), D(p, x, y) is the disparity difference measure between point p and point (x, y), and λ is the weight parameter that controls the influence of the smoothing term.

[0081] After cost aggregation, the disparity value of each pixel in the disparity map is determined by selecting the disparity with the minimum cost. Specifically, the formula for disparity calculation is:

[0082] d(x,y) = argmin d C agg (x,y,d)

[0083] where, C agg (x, y, d) is the cost function after cost aggregation, and the disparity value d(x, y) corresponds to the disparity with the minimum cost.

[0084] The purpose of disparity optimization is to reduce the noise in the disparity map and obtain a smoother and more accurate disparity map. To achieve this goal, SGM will adopt an optimization strategy based on local smoothing terms and global consistency. After these steps, pixel-level disparity is obtained, and then the disparity is further interpolated by a quadratic curve to obtain sub-pixel-level disparity: perform a quadratic curve fitting on the optimal disparity and the cost values of the two disparities before and after the optimal disparity, and the disparity corresponding to the extreme point of the curve is the sub-pixel-level disparity.

[0085] Finally, the detected building land change areas are overlaid with the images of the 3D reconstruction model of the urban building base to extract the urban building change patches, establish ledger information, compare the timeline of the construction application materials for each change area with the detected change time in the aerial images, and combine the time attribute of the construction application materials with the time judgment of the change area results to obtain a table of the actual change time phase. The attributes included in the table are serial number, residential area name, construction application time, change time, description of the change area, etc.

[0086] In summary, the method first obtains multi - period aerial video data of urban buildings, then takes the refined three - dimensional model of the urban building base as a benchmark, performs uniform frame extraction on the multi - period video data, conducts rapid three - dimensional model modeling based on the UAV aerial video images to generate multi - period building three - dimensional models; finally, uses the multi - period building three - dimensional models to generate dense point clouds of urban buildings, performs change detection on the dense point clouds of different periods, extracts urban building change patches, provides data support for improving the level of urban planning management and supervision, and thus solves the problems of high cost and slow update of the existing UAV oblique photography for three - dimensional model reconstruction and the inability to perform rapid urban building change detection.

[0087] The following further illustrates this method with an application scenario example:

[0088] The monitoring object is a certain area in Danzao Town, Nanhai District, Foshan City.

[0089] The data acquisition and pre - processing steps include:

[0090] UAV oblique photography: According to the technical requirements of UAV oblique photography, set up photo control points in the task area, and then carry out oblique aerial photography and photo control survey work respectively;

[0091] UAV aerial video shooting: Determine the take - off point, plan the multi - angle building photo acquisition route, determine the inspection flight route, check the instruments, and use the UAV equipped with a high - resolution optical camera to collect quarterly community inspection videos. Save and back up the data. Use Python and OpenCV to evenly divide the video frames and save them. Determine the extraction interval frames on the premise of ensuring the image effect. Finally, correct the distortion of the extracted frames according to the camera distortion parameters to complete the pre - processing work of the inspection data and provide a data basis for the subsequent construction of the three - dimensional model for regular inspection of urban buildings.

[0092] Steps for constructing the three - dimensional model of the urban building base based on UAV oblique photography:

[0093] Pre - process the UAV oblique image data, and at the same time solve and organize the photo control data. Utilize the results of oblique aerial photography and photo control survey, and through processes such as aerial triangulation, dense matching, automatic modeling, and texture mapping, automatically generate three - dimensional model results as the data base plate for reference correction.

[0094] (1) Aerial triangulation

[0095] The oblique images and image control point data are introduced into the automatic modeling software system, and the control points and images are manually associated. Then, the automatic modeling software is used to adopt the bundle method for overall adjustment of the regional network. A beam of light composed of a photograph is used as an adjustment unit, and the collinear equation of the central projection is used as the basic equation of the adjustment unit. Through the rotation and translation of each light beam in space, the common light between the models can achieve the best intersection, and the overall area is optimally added to the control point coordinate system, thereby restoring the spatial position relationship between the objects.

[0096] At the same time, by connecting the point and control point coordinate files and combining GNSS / IMU information, the iterative calculation of the image self-calibration regional network adjustment is realized. Through repeated joint solutions, the systematic errors and random errors in the external orientation are eliminated and weakened, providing a mathematical accuracy foundation for the production of three-dimensional models.

[0097] (2) Puncture point

[0098] To improve the accuracy of aerial triangulation, you can manually select the image where the control points are located for puncturing. Then compare these points with the control points measured by external operations to determine their exact location in the image. After adding the control points, run the aerial triangulation again to check if it is successful. If it fails, check whether there are errors in the puncture points or POS data until encryption is successful.

[0099] (3) Dense matching and 3D model construction

[0100] Use aerial triangulation technology to obtain the exterior orientation elements of the image, determine the position of the image in three-dimensional space, and form a stereo pair. Use multi-view image dense matching technology to identify and match feature points in multiple images to generate dense point cloud data. Based on these matched point cloud data, a triangular mesh model is generated.

[0101] (4) Texture lamination

[0102] The collected drone images are cut into corresponding texture triangles according to the triangles in the three-dimensional mesh model, and a corresponding relationship is established with the three-dimensional mesh model. The two-dimensional plane texture triangles are mapped to the three-dimensional mesh model to complete the texture mapping, automatically generate the three-dimensional model of the regional buildings, and check the model accuracy.

[0103] The steps for quickly generating a multi-phase urban building 3D model include:

[0104] (1) Further process the video photos collected by the pre-processed drone aerial video, remove photos with too high repetition, filter the framed images, and check the temporary obstacles in the framed photos to block the buildings.

[0105] (2) Based on the 3D model of the urban building base and its original photos, assist in reconstructing the 3D model from evenly divided frame photos of the inspection video: ① Fuse the GPS / IMU data of the evenly divided frame photos of the inspection video and the photos of the urban building base 3D model to achieve the geographical alignment of the images; ② Use methods such as SIFT and SURF with the help of the urban building base 3D model data to assist the feature mapping work in the process of reconstructing the 3D model from evenly divided frames of the inspection video, so as to improve the accuracy of reconstructing the 3D model from evenly divided frames of the video; ③ Optimize the camera pose and internal parameters: Based on the alignment result of the urban building base 3D model, further optimize the internal parameters (focal length, principal point position, distortion coefficient, etc.) and external parameters (camera position, pose, etc.) of the UAV video data, and use the geometric characteristics of the urban building base 3D model to perform Bundle Adjustment of the camera to reduce the reprojection error of the model and improve the accuracy of the quickly reconstructed 3D model; ④ Optimize the point cloud density: Convert the evenly divided frame photos of the multi-angle inspection video into a denser point cloud through the Multi-View Stereo Matching (MVS) algorithm. The existing refined 3D model of the urban building base can be used as an optimization constraint condition to ensure the matching of the new point cloud data with its geometric shape.

[0106] Through the above operations, multi-phase 3D models of urban buildings can be quickly generated.

[0107] The steps of 3D model change detection based on dense point clouds include

[0108] Using the generated multi-phase 3D models of urban buildings, generate dense point clouds. Use the Voxel Grid Filter algorithm to perform voxel filtering and denoising on the dense point cloud data to remove redundant noise. According to the 3D model of the urban building base and the dense point cloud data of the multi-phase evenly divided frame reconstruction of the inspection video, use the Semi-Global Matching (SGM) algorithm to perform change detection on the dense point clouds of different periods. After steps such as matching cost calculation, cost aggregation, disparity calculation, and disparity optimization, obtain the pixel-level disparity, and then perform quadratic curve fitting on the optimal disparity and the cost values of the two disparities before and after the optimal disparity to obtain the sub-pixel-level disparity, so as to quickly and automatically extract the changed areas in urban buildings.

[0109] The main purpose of the residential area inspection ledger is to draw patch data for the suspected problem houses and add ledger information to the patch attributes. Taking the community as the basic unit, count the content such as the ledger building type, building material, and illegal construction type in the community, and analyze the suspected building changes in each community.

[0110] The above embodiments are only for illustrating the technical concept and features of the present invention, and the purpose is to enable those of ordinary skill in the art to understand the content of the present invention and implement it accordingly. It is not intended to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the essence of the content of the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for detecting urban building changes based on drone videos and 3D models, characterized in that, Including: Obtaining multi - period UAV aerial video data of the target area; Based on the 3D model of the urban building base in the target area, performing uniform frame extraction on the multi - period UAV aerial video data to generate multi - period 3D models of urban buildings; The 3D model of the urban building base is constructed based on the UAV oblique photography data of the target area; Generating multi - period dense point clouds of urban buildings based on the multi - period 3D models of urban buildings; Performing change detection on the dense point clouds of the multi - period urban buildings and extracting urban building change patches.

2. The method for detecting changes in urban buildings based on drone videos and 3D models according to claim 1, wherein, Using the semi - global matching algorithm to perform change detection on the dense point clouds of the multi - period urban buildings. Taking the dense point clouds as the data source, using the elevation difference or Euclidean distance between two - phase data as the change measure, and then using threshold segmentation to extract the change area.

3. The method for detecting urban building changes based on drone video and 3D model according to claim 2, wherein Using clustering to process the change area.

4. The method for detecting urban building changes based on UAV video and 3D model according to claim 2, wherein, The semi - global matching algorithm includes matching cost calculation, which is used to measure the similarity of each pair of pixel points in two view - point images. Using the squared - difference cost as the cost metric, and the calculation method is as follows: C(d) = (I L (x,y) - IR(x - d,y)) 2 Among them, C(d) is the degree of difference between the left and right images at disparity d; I L (x, y) and IR(x - d, y) respectively represent the element values of the left and right images at position (x, y); The formula for cost aggregation is as follows: Where: C(p, d) is the matching cost of point p at disparity d, N(x, y) is the neighborhood of point (x, y), D(p, x, y) is the disparity difference measure between point p and point (x, y), and λ is the weight parameter that controls the influence of the smoothing term.

5. The method for detecting urban building changes based on drone videos and 3D models according to claim 4, wherein, After cost aggregation, the disparity value of each pixel in the disparity map is determined by selecting the disparity with the minimum cost. The formula for disparity calculation is: d(x,y) = argmin d C agg (x,y,d) Among them, C agg (x, y, d) is the cost function after cost aggregation, and the disparity value d(x, y) corresponds to the disparity with the minimum cost.

6. The urban building change detection method based on UAV video and 3D model according to claim 1, characterized in that, Before performing uniform frame extraction on the multi - period UAV aerial video data, it also includes: Using Python and OpenCV to perform video uniform frame division and saving on the multi - period UAV aerial video data, determining the extraction interval number of frames, and correcting the distortion of the extracted frame images according to the camera distortion parameters to pre - process the multi - period UAV aerial video data.

7. The method for detecting urban building changes based on UAV videos and 3D models according to claim 6, wherein Before performing uniform frame extraction on the multi - period UAV aerial video data, it also includes: Further processing the pre - processed multi - period UAV aerial video data, including: Removing images with non - compliant repetition degrees; Screening the frame - divided images and removing the non - compliant frame - divided ones; Checking the occlusion of buildings by temporary obstacles in the frame - divided photos.

8. The method for detecting urban building changes based on UAV video and 3D model according to claim 1, characterized in that, The 3D model of the urban building base in the target area is constructed in the following way: Image pair selection: Obtaining UAV oblique photography data of the target area, selecting several image pairs from it, performing feature matching, establishing the spatial relationship between images, obtaining the matching point cloud image pairs, and generating preliminary 3D point cloud data; Point cloud matching and densification: Extracting three - dimensional coordinate points from the matching point cloud image pairs, and based on the sparse point cloud obtained by stereo matching, using photometric consistency and multi - view geometric constraints to achieve point cloud densification; Triangulation network construction and optimization: Generating a 3D mesh based on the point cloud data to provide a structural basis for the 3D model of the urban building base; Texture mapping: Mapping the actually collected image data onto the 3D mesh to generate a 3D model of the urban building base with a real appearance.

9. The method for detecting urban building changes based on UAV video and 3D model according to claim 1 or 8, characterized in that Based on the 3D model of the urban building base in the target area, performing uniform frame extraction on the multi - period UAV aerial video data to generate multi - period 3D models of urban buildings, including: Fuse the GPS / IMU data of the evenly divided frame photos of the multi-angle inspection videos of multi-period unmanned aerial vehicle video data and the photos of the 3D model of the urban building base to achieve the geographical alignment of the images; With the help of the data of the 3D model of the urban building base, assist the feature mapping work in the reconstruction process of the evenly divided frame 3D model of the inspection video; Based on the alignment result of the 3D model data of the urban building base, optimize the internal and external parameters of the camera of the unmanned aerial vehicle video data; Convert the evenly divided frame photos of the multi-angle inspection video into a new point cloud through the multi-view stereo matching algorithm; use the 3D model of the urban building base as a constraint condition to ensure that the new point cloud data is consistent with the geometric shape of the 3D model of the urban building base.

10. The method for detecting urban building changes based on UAV video and 3D model according to claim 1, wherein Before performing change detection on the dense point cloud of the multi-period urban buildings, it also includes: Adopt a voxel gridding processing method to map the point cloud data into a uniform three-dimensional grid for downsampling and denoising of the dense point cloud.

Images