A method for quickly updating white models of buildings in large-scale regional scenes

Abstract

This invention discloses a method for rapid updating of white models of buildings in large-scale regional scenes, comprising the following steps: S1, acquiring the image to be updated and calculating the range of the image region to be updated in the low-altitude data base image; S2, calculating affine transformation parameters; S3, performing polynomial geometric correction on the image to be updated; S4, extracting building outlines from the low-altitude data base image and the image to be updated; S5, regularizing the building outlines to generate regularized polygons; S6, determining the white model region of the building to be updated; S7, calculating the height data of the white model region of the building to be updated; S8, fusing the updated height data with the low-altitude data base white model data to generate updated building white model data; S9, repeating steps S1-S8 to continuously update the low-altitude data base white model data. This invention effectively and quickly completes the rapid updating of low-altitude urban bases, providing safe flight path guarantees for low-altitude unmanned aerial vehicles (UAVs) for obstacle avoidance and path planning.

Description

Technical Field

[0001] This invention relates to the field of remote sensing data processing and application technology, and in particular to a method for rapid updating of white models of buildings in large-scale regional scenes. Background Technology

[0002] In recent years, the low-altitude economy has represented the development direction of new-type productivity, integrating with economic and social activities and continuously exploring new application scenarios and business models to significantly improve the efficiency of economic and social activities and user experience. Around multiple low-altitude scenarios, the country will comprehensively deploy services in areas such as urban air traffic, air rescue, low-altitude cultural tourism, low-altitude logistics, unmanned aerial vehicle training, and low-altitude flight support, creating a rich, convenient, and efficient multi-dimensional living space to truly realize the ideal urban lifestyle of "work is life, and life is vacation."

[0003] The spatiotemporal digital foundation, as a realistic, three-dimensional, and time-series reflection of human production, living, and ecological spaces, is a crucial new type of national infrastructure, the spatiotemporal basis for the construction of Digital China, and an important strategic data resource and production factor for digital government and the digital economy. Through real-world 3D modeling technology, scenes, buildings, and terrains can be transformed into 3D models. White models of buildings form the data foundation for developing the low-altitude data foundation.

[0004] A building white model, also known as a simplified building model, is a simplified 3D model of a building that includes basic information such as its location, shape, area, and height. Traditional surveying data, aerial photogrammetry imagery, and airborne LiDAR data can all be used to construct building white models. Stereo satellite imagery, due to its unrestricted observation range, wider image swath, lower acquisition cost, and higher update frequency, has become a more suitable data source for constructing large-scale building white models.

[0005] With the rapid changes in the scale of urban development, current technologies mainly focus on quickly producing white model data for large-scale building scenes. For example, they use deep learning-based monocular images to extract building height data, generate digital surface models based on satellite stereo image pairs to construct building white models, and construct building white model height data based on sparse matching and forward intersection of satellite stereo image pairs. With the gradual acceleration of urbanization and the booming development of the low-altitude economy, there is currently a lack of rapid and effective methods for updating large-scale building white models in urban areas. How to quickly and effectively update building white model data for large-scale regional scenes is a necessary approach for the construction of a low-altitude data base. Summary of the Invention

[0006] To address the existing problems, this invention provides a method for rapidly updating white models of buildings in large-scale regional scenes, the specific solution of which is as follows:

[0007] A method for rapidly updating white models of buildings in a large-scale regional scene includes the following steps:

[0008] S1. Obtain the image to be updated. Based on the RPC parameter file of the image to be updated, preliminarily calculate the range of the area to be updated in the low-altitude data base image.

[0009] S2, obtain the corresponding points of the low-altitude data base image and the image to be updated through an automated matching algorithm, and calculate the affine transformation parameters based on the correspondence of the corresponding points;

[0010] S3, using affine transformation parameters to perform polynomial geometric correction on the image to be updated, so that it is aligned with the low-altitude data base image in spatial position.

[0011] S4, extract building outlines from the low-altitude base image and the image to be updated;

[0012] S5, regularizes the building outline to generate regular polygons;

[0013] S6, perform difference calculation on the building outline binary map of the low-altitude data base image and the image to be updated to determine the white model area of ​​the building to be updated;

[0014] S7, calculate the height data of the white model area of ​​the building to be updated;

[0015] S8, merges the updated height data with the low-altitude base white model data to generate updated building white model data;

[0016] S9. For newly added images to be updated, repeat steps S1-S8 in sequence to continuously update the white model data of the low-altitude data base.

[0017] Preferably, the image to be updated in step S1 is a pair of stereo satellite images in the same orbit or a monocular orthophoto.

[0018] Preferably, step S4 uses HRnet-18 as the skeleton for building feature extraction and combines spatial pyramid pooling to fuse features of different depths to extract building outlines from the low-altitude base image and the image to be updated.

[0019] Preferably, the regularization adjustment of the building outline in step S5 includes long side direction classification, short side direction correction, and line segment merging. Specifically, different edge thresholds are set according to the building area, the edges are divided into long sides and short sides, the direction of the longest side is added to the main direction list, the directions of other long sides are compared with the main directions in the list, if the angle is within the threshold range, it is added to the list, the long side is rotated to be near the main direction or its perpendicular direction, the short side is adjusted to be parallel or perpendicular to the main direction, parallel lines with a distance less than the threshold are merged or connected, and all adjusted lines are connected to form the final polygon outline.

[0020] Preferably, in step S7, if the image to be updated is a pair of stereo satellite images on the same orbit, the height data of the area to be updated is calculated based on the RPC parameters and the forward intersection algorithm; if the image to be updated is a monocular orthophoto, the height data of the area to be updated is retrieved based on a pre-trained deep learning model.

[0021] Preferably, in step S7, the coordinates of corresponding image points in the forward and backward views of the same-track stereo satellite image pair are used to construct a rational function model in combination with RPC parameters, and the ground point height of the area to be updated is calculated by least squares forward intersection. In addition, the deep learning model adopts the DINV2 open-source framework, and the building height inversion model is pre-trained based on the low-altitude data base. At the same time, the model is fine-tuned for the height data calculated for the area with stereo image pairs. Finally, the height data of the white model area of ​​the building to be updated is calculated based on the deep learning model.

[0022] Preferably, the fusion process in step S8 includes spatially overlaying and topological consistency correction of the newly added or changed building height data with the original white model data in the low-altitude data base.

[0023] Preferably, the low-altitude data base image includes at least one of a real-scene 3D model, aerial photogrammetry image, or airborne LiDAR data.

[0024] The present invention also discloses a computer-readable storage medium storing a computer program, which, when executed, performs the method described in any of the above-mentioned embodiments.

[0025] The present invention also discloses a computer system including a processor, a storage medium storing a computer program, and the processor reading from the storage medium and running the computer program to perform the method described in any of the preceding claims.

[0026] The beneficial effects of this invention are as follows:

[0027] This invention provides a method for rapidly updating white models of buildings in large-scale regional scenes, enabling quick updates of white model data for buildings in such scenes. With the rapid development of the low-altitude economy, the timeliness and update frequency of building white models are increasingly important. The technical solution of this patent can effectively and quickly update the base of low-altitude urban structures, providing safe flight path guarantees for low-altitude unmanned aerial vehicles (UAVs) in obstacle avoidance and path planning. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A schematic diagram of a stereo image pair with RPC parameters on the same track;

[0030] Figure 2 A diagram illustrating the matching effect between low-altitude data base imagery and the imagery to be updated;

[0031] Figure 3 Image showing the effect of correcting and registering the low-altitude data base image and the image to be updated;

[0032] Figure 4 This is an example of the low-altitude data base building extraction effect provided by the present invention;

[0033] Figure 5 This invention provides an image of the extracted buildings from the image to be updated.

[0034] Figure 6 The updated building outline range diagram provided by this invention;

[0035] Figure 7 Workflow diagram for updating low-altitude data based on stereo image pairs;

[0036] Figure 8 Workflow diagram for updating low-altitude data base based on monocular orthophotos. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] The purpose of this invention is to provide a method for rapid updating of white models of buildings in large-scale regional scenes, and to realize its application in the rapid updating scenario of low-altitude data base, so as to assist in the route planning and obstacle avoidance of low-altitude unmanned vehicles.

[0039] This invention employs an automated matching algorithm. First, it obtains corresponding points between the low-altitude data base image and the image to be updated using a matching algorithm. Then, by utilizing the correspondence between the corresponding points, it calculates the affine transformation relationship between the low-altitude data base image and the image to be updated. Finally, it uses the affine transformation parameters to perform polynomial geometric correction on the image to be updated, thereby aligning the image to be updated with the low-altitude data base image in spatial position. This ensures that the same features in the image to be updated and the low-altitude data base image are located in the same position as much as possible.

[0040] When the image to be updated is a stereo satellite image pair, this invention uses the back-view image of the stereo image pair as the image to be updated and performs difference calculation with the low-altitude data base image to obtain the outline region of the building to be updated. Then, based on the forward intersection technology of the stereo image pair, the height of the outline region of the building to be updated is calculated.

[0041] When the image to be updated is a monocular orthophoto, the outline region of the building to be updated is obtained by combining the image to be updated with the low-altitude data base image and performing difference calculation. Then, the height of the outline region of the building to be updated can be quickly calculated by using a deep learning neural network model.

[0042] Specifically, such as Figure 2-8 A method for rapidly updating white models of buildings in a large-scale regional scene includes the following steps:

[0043] S1. Acquire the image to be updated. Based on the RPC parameter file of the image to be updated, preliminarily calculate the range of the area to be updated in the low-altitude data base image. The image to be updated is either a pair of stereo satellite images in the same orbit or a monocular orthophoto. The low-altitude data base image includes at least one of the following: a real-world 3D model, aerial photogrammetry image, or airborne LiDAR data.

[0044] S2, obtain the corresponding points of the low-altitude data base image and the image to be updated through an automated matching algorithm, and calculate the affine transformation parameters based on the correspondence of the corresponding points;

[0045] S3 uses affine transformation parameters to perform polynomial geometric correction on the image to be updated, so that the image to be updated is spatially aligned with the low-altitude data base image, and the same features in the image to be updated and the low-altitude data base image are in the same position as much as possible.

[0046] S4. Building outlines are extracted from the low-altitude base image and the image to be updated. Specifically, the HRNet network, through its unique multi-resolution parallel structure and feature fusion strategy, effectively captures details and global information while maintaining high-resolution features. This invention uses HRnet-18 as the skeleton for building feature extraction and combines spatial pyramid pooling to fuse features of different depths to extract building outlines from the low-altitude base image and the image to be updated.

[0047] S5. Since the building outline edges extracted by the automated algorithm are not very regular and complete, it is necessary to make regular adjustments to the building outline to generate regularized polygons.

[0048] The building outline is regularized through adjustments including long-side direction classification, short-side direction correction, and line segment merging. Specifically, different edge thresholds are set based on the building area. Edges are divided into long and short sides. The direction of the longest side is added to the primary direction list. The directions of other long sides are compared with the primary directions in the list. If the angle is within the threshold range, it is added to the list. The long sides are rotated to be near the primary direction or its perpendicular direction. The short sides are adjusted to be parallel or perpendicular to the primary direction. Parallel lines with a distance less than the threshold are merged or connected. All adjusted lines are connected to form the final polygon outline.

[0049] S6. Based on step S5, the building outline binary map (0: background, 1: building) of the low-altitude data base image (within the range of the image to be updated) and the building outline binary map (0: background, 1: building) of the stereo image pair (back view image) of the image to be updated can be obtained. The difference operation is performed on the building outline binary map of the low-altitude data base image and the image to be updated to determine the white model area of ​​the building to be updated.

[0050] S7, calculate the height data of the white model area of ​​the building to be updated; if the image to be updated is a pair of stereo satellite images in the same orbit, calculate the height data of the area to be updated based on RPC parameters and forward intersection algorithm; if the image to be updated is a monocular orthophoto, retrieve the height data of the area to be updated based on a pre-trained deep learning model.

[0051] Specifically, such as Figure 1Define the local tangent plane coordinate system O-XYZ as the object space coordinate system, with the origin located near the center point of the ground coverage area of ​​the stereo model; PRP represents the object space reference plane, which is the average elevation plane of the object space with elevation H; points p1 and p2 are any pair of conjugate points (same image points) on the left and right scenes; point P is the ground point corresponding to points p1 and p2; according to the definition of the epipolar line of the linear array image based on the projection trajectory method, Ep1 on the right scene represents the epipolar curve corresponding to p1 on the left scene, and Ep2 on the left scene represents the epipolar curve corresponding to p2 on the right scene; EP is the approximate epipolar surface; ED is the approximate epipolar line; AP represents the rectangular area of ​​the epipolar image on the reference plane PRP, and the horizontal side of the rectangle is parallel to the direction of the epipolar line.

[0052] For the image pairs to be updated, which are stereo image pairs with RPC parameters, the coordinates of corresponding image points p1(c1,r1) and p2(c2,r2) are automatically extracted from the stereo image pair based on the outline area of ​​the building to be updated. Here, c is the column number of the point in the image, and r is the row number of the point. The geometric relationship between the object and image sides of the foreground and background images of the stereo image pair can be described using a rational function model established with the RPC parameters of the stereo image pair. Then, the coordinates of ground points are calculated using the least squares method for stereo intersection, ultimately obtaining the height data of the white model area of ​​the building to be updated.

[0053] For cases where the image to be updated is a monocular orthophoto, the open-source DINV2 model is used. The building height inversion model is pre-trained based on the low-altitude data base. At the same time, the model is fine-tuned for the height data calculated for areas with stereo image pairs. Finally, the height data of the white model area of ​​the building to be updated is calculated based on the model.

[0054] S8. The updated height data is fused with the low-altitude data base white model data to generate updated building white model data. The fusion process includes spatial overlay and topological consistency correction of the newly added or changed building height data with the original white model data in the low-altitude data base.

[0055] S9. For newly added images to be updated, repeat steps S1-S8 in sequence to continuously update the white model data of the low-altitude data base.

[0056] This invention provides a method for rapidly updating white models of buildings in large-scale regional scenes, enabling quick updates of white model data for buildings in such scenes. With the rapid development of the low-altitude economy, the timeliness and update frequency of building white models are increasingly important. The technical solution of this patent can effectively and quickly update the base of low-altitude urban structures, providing safe flight path guarantees for low-altitude unmanned aerial vehicles (UAVs) in obstacle avoidance and path planning.

[0057] The present invention also discloses a computer-readable storage medium and a computer system. The medium stores a computer program, which, upon execution, performs the method described in any of the preceding claims. A computer system includes a processor and a storage medium, the storage medium storing a computer program, and the processor reading from and running the computer program from the storage medium to perform the method described in any of the preceding claims.

[0058] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.

[0059] In the embodiments, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions may be stored or transmitted as one or more instructions or code on or through a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, encompassing any medium that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible to a computer. Any connection is also legitimately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. As used in this article, disk and disc include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0060] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0061] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for rapid updating of white models of buildings in a large-scale regional scene, characterized in that, Includes the following steps: S1. Obtain the image to be updated. Based on the RPC parameter file of the image to be updated, preliminarily calculate the range of the area to be updated in the low-altitude data base image. S2, obtain the corresponding points of the low-altitude data base image and the image to be updated through an automated matching algorithm, and calculate the affine transformation parameters based on the correspondence of the corresponding points; S3, using affine transformation parameters to perform polynomial geometric correction on the image to be updated, so that it is aligned with the low-altitude data base image in spatial position. S4, extract building outlines from the low-altitude base image and the image to be updated; S5, regularizes the building outline to generate regular polygons; S6, perform difference calculation on the building outline binary map of the low-altitude data base image and the image to be updated to determine the white model area of ​​the building to be updated; S7, calculate the height data of the white model area of ​​the building to be updated; S8, merges the updated height data with the low-altitude base white model data to generate updated building white model data; S9. For newly added images to be updated, repeat steps S1-S8 in sequence to continuously update the white model data of the low-altitude data base.

2. The method according to claim 1, characterized in that: The image to be updated in step S1 is a pair of stereo satellite images in the same orbit or a monocular orthophoto.

3. The method according to claim 1, characterized in that: Step S4 uses HRnet-18 as the skeleton for building feature extraction and combines spatial pyramid pooling to fuse features of different depths to extract building outlines from the low-altitude base image and the image to be updated.

4. The method according to claim 1, characterized in that: Step S5 involves regularizing the building outline by classifying the long side direction, correcting the short side direction, and merging line segments. Specifically, different edge thresholds are set according to the building area. The edges are divided into long and short sides. The direction of the longest side is added to the main direction list. The directions of other long sides are compared with the main directions in the list. If the angle is within the threshold range, it is added to the list. The long side is rotated to be near the main direction or its perpendicular direction. The short side is adjusted to be parallel or perpendicular to the main direction. Parallel lines with a distance less than the threshold are merged or connected. All adjusted lines are connected to form the final polygon outline.

5. The method according to claim 2, characterized in that: In step S7, if the image to be updated is a pair of stereo satellite images in the same orbit, the height data of the area to be updated is calculated based on the RPC parameters and the forward intersection algorithm; if the image to be updated is a monocular orthophoto, the height data of the area to be updated is retrieved based on a pre-trained deep learning model.

6. The method according to claim 5, characterized in that: In step S7, the coordinates of corresponding image points in the forward and backward views of the same-track stereo satellite image pair are used to construct a rational function model in conjunction with RPC parameters. The ground point height of the area to be updated is calculated by least-squares forward intersection. In addition, the deep learning model adopts the DINV2 open-source framework and pre-trains the building height inversion model based on the low-altitude data base. At the same time, the model is fine-tuned for the height data calculated for the area with stereo image pairs. Finally, the height data of the white model area of ​​the building to be updated is calculated based on the deep learning model.

7. The method according to claim 1, characterized in that: The fusion process in step S8 includes spatially overlaying and topological consistency correction of the newly added or changed building height data with the original white model data in the low-altitude data base.

8. The method according to claim 1, characterized in that: The low-altitude data base imagery includes at least one of the following: a real-world 3D model, aerial photogrammetry imagery, or airborne LiDAR data.

9. A computer-readable storage medium, characterized in that: The medium contains a computer program, which, when executed, performs the method as described in any one of claims 1 to 8.

10. A computer system, characterized in that: It includes a processor and a storage medium, on which a computer program is stored, and the processor reads from the storage medium and runs the computer program to perform the method as described in any one of claims 1 to 8.

Images