Large-scale area scene building white mold rapid updating method

Abstract

The invention discloses a large-scale area scene building white mold rapid updating method, which comprises the following steps of S1, acquiring an image to be updated, and calculating the range of an image area to be updated in a low-altitude data base image; s2, calculating affine transformation parameters; s3, performing polynomial geometric correction on the to-be-updated image; s4, performing building contour extraction on the low-altitude data base image and the to-be-updated image; s5, performing regularization adjustment on the building contour to generate a regularized polygon; s6, determining a to-be-updated building white mold area; s7, calculating height data of the to-be-updated building white mold area; s8, performing fusion processing on the updated height data and the low-altitude data base white mold data to generate updated building white mold data; and S9, repeating the steps S1 to S8, and continuously updating the low-altitude data base white mold data. According to the method, rapid updating of the low-altitude city base is effectively and rapidly completed, and safe fly line guarantee is provided for fly line obstacle avoidance, path planning and the like of the low-altitude unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing data processing and application technology, and in particular to a method for quickly updating a white model of a building in a large-scale regional scene. Background Art

[0002] In recent years, the low-altitude economy has represented the development direction of new productivity, becoming integrated into economic and social activities and continuously exploring new application scenarios and business models to significantly improve the efficiency of economic and social activities and user experience. Focusing on multiple low-altitude scenarios, the country will comprehensively develop service areas such as urban air transportation, aviation rescue, low-altitude cultural tourism, low-altitude logistics, unmanned driving training, and low-altitude flight support. This will create a rich, convenient, and efficient multi-dimensional living space, truly realizing the ideal urban lifestyle of "work is life, life is vacation."

[0003] The spatiotemporal digital base, which provides a realistic, three-dimensional, and time-sequential reflection of the spatiotemporal information of human production, life, and ecological space, is a key national new infrastructure and the spatiotemporal foundation for the construction of a Digital China. It is also a crucial strategic data resource and production factor for digital government and the digital economy. Using real-world 3D modeling technology, real-world scenes, buildings, and terrain can be transformed into 3D models. Building white models serve as the data foundation for developing low-altitude data bases.

[0004] A building mockup, also known as a simplified three-dimensional model, contains basic information such as its location, shape, area, and height. Traditional surveying and mapping data, aerial photogrammetry imagery, and airborne LiDAR data can all be used to construct building mockups. Stereo satellite imagery, due to its unlimited observation range, larger image width, lower acquisition cost, and higher update frequency, has become a more suitable data source for constructing large-scale building mockups.

[0005] With the rapid changes in the scale of urban development, the current existing technologies mainly focus on quickly producing building white model data for large-scale scenes, such as using monocular images based on deep learning to extract building height data, generating digital surface models based on satellite stereo pairs to construct building white models, and constructing building white model height data based on sparse matching forward intersection of satellite stereo pairs. With the gradual acceleration of urbanization and the vigorous development of the low-altitude economy, there is currently a lack of fast and effective methods for updating large-scale building white models in urban areas. How to quickly and effectively update building white model data for large-scale regional scenes is a necessary way to build a low-altitude data base. Summary of the Invention

[0006] In order to solve the existing problems, the present invention provides a method for quickly updating the white model of a building in a large-scale regional scene. The specific solution is as follows:

[0007] A method for quickly updating a white model of a building in a large-scale regional scene comprises the following steps:

[0008] S1, obtain the image to be updated, and preliminarily calculate the range of the image area to be updated in the low-altitude data base image based on the RPC parameter file of the image to be updated;

[0009] S2, obtain the corresponding homonymous points of the low-altitude data base image and the image to be updated through an automated matching algorithm, and calculate the affine transformation parameters based on the corresponding relationship between the homonymous points;

[0010] S3, using affine transformation parameters to perform polynomial geometric correction on the image to be updated so that it is aligned with the spatial position of the low-altitude data base image;

[0011] S4, extracting building outlines from the low-altitude data base image and the image to be updated;

[0012] S5, regularizing and adjusting the building outline to generate regularized polygons;

[0013] S6, performing a difference operation on the low-altitude data base image and the building outline binary image of the image to be updated to determine the white model area of the building to be updated;

[0014] S7, calculating the height data of the white mold area of the building to be updated;

[0015] S8, fusing the updated height data with the low-altitude base white mold data to generate updated building white mold data;

[0016] S9, repeating steps S1-S8 for the newly added images to be updated, and continuously updating the white model data of the low-altitude data base.

[0017] Preferably, the image to be updated in step S1 is a co-orbital stereo satellite image pair or a monocular orthophoto image.

[0018] Preferably, step S4 uses HRnet-18 as the building feature extraction skeleton, and combines spatial pyramid pooling to fuse different depth features to extract building contours from the low-altitude data base image and the image to be updated.

[0019] Preferably, the regularized adjustment of the building outline in step S5 includes classification of long side directions, correction of short side directions and merging of line segments; specifically, different side thresholds are set according to the building area, the sides are divided into long sides and short sides, the direction of the longest side is added to the main direction list, the directions of other long sides are compared with the main direction in the list, if the angle is within the threshold range, it is added to the list, the long side is rotated to the main direction or near its perpendicular direction, the short side is adjusted to be parallel or perpendicular to the main direction, parallel lines with a distance less than the threshold are merged or connected, and all adjusted lines are connected to form the final polygon outline.

[0020] Preferably, in step S7, if the image to be updated is a co-orbital stereo satellite image pair, the height data of the area to be updated is calculated based on the RPC parameters and the forward intersection algorithm; if the image to be updated is a monocular orthophoto, the height data of the area to be updated is inverted based on a pre-trained deep learning model.

[0021] Preferably, in step S7, the coordinates of the same-name image points in the front and rear view images of the same-track stereo satellite image pair are used, combined with the RPC parameters to construct a rational function model, and the height of the ground points in the area to be updated is calculated by least squares forward intersection; in addition, the deep learning model adopts the DINV2 open source framework, and the building height inversion model is pre-trained based on the low-altitude data base. At the same time, the model is fine-tuned for the height data calculated for the area with stereo image pairs, and finally the height data of the building white model area to be updated is deduced and calculated based on the deep learning model.

[0022] Preferably, the fusion processing in step S8 includes spatially superimposing the newly added or changed building height data with the original white model data in the low-altitude data base and correcting the topological consistency.

[0023] Preferably, the low-altitude data base image includes at least one of a real-life three-dimensional model, an aerial photogrammetry image or airborne LiDAR data.

[0024] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is run, any of the above methods is executed.

[0025] The present invention also discloses a computer system, including a processor and a storage medium, wherein a computer program is stored on the storage medium, and the processor reads and runs the computer program from the storage medium to execute any of the methods described above.

[0026] The beneficial effects of the present invention are:

[0027] This invention rapidly updates building white model data for large-scale regional scenarios by constructing a method for rapidly updating building white models. With the rapid development of the low-altitude economy, the timeliness and update frequency of building white models are required to be higher. The technical solution of this patent can effectively and quickly complete the rapid update of low-altitude urban bases, providing safe flight line protection for low-altitude unmanned aerial vehicles (UAVs) such as obstacle avoidance and path planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 Schematic diagram of a co-track stereo image pair with RPC parameters;

[0030] Figure 2 This is the matching effect diagram of the low-altitude data base image and the image to be updated;

[0031] Figure 3 Correction, registration and overlay effect diagram of the low-altitude data base image and the image to be updated;

[0032] Figure 4 The low-altitude data base building extraction rendering provided by the present invention;

[0033] Figure 5 The image building extraction rendering to be updated provided by the present invention;

[0034] Figure 6 Updated building outline range map provided by the present invention;

[0035] Figure 7 Workflow diagram for updating low-altitude data base based on stereo image pairs;

[0036] Figure 8 Workflow diagram for updating low-altitude data base based on monocular orthophotos. DETAILED DESCRIPTION

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0038] The purpose of the present invention is to provide a method for quickly updating white models of buildings in large-scale regional scenes, and to realize its application in the rapid update scenario of low-altitude data bases, so as to assist low-altitude unmanned vehicle route planning and flight obstacle avoidance.

[0039] The present invention uses an automated matching algorithm, that is, first, the matching algorithm is used to obtain the corresponding same-name points of the low-altitude data base image and the image to be updated, and then the corresponding relationship between the same-name points is used to calculate the affine transformation relationship between the low-altitude data base image and the image to be updated, and the affine transformation parameters are used to perform polynomial geometric correction on the image to be updated, so that the image to be updated and the low-altitude data base image are aligned in spatial position, so that the same features in the image to be updated and the low-altitude data base image are as close as possible to the same position.

[0040] In the case where the image to be updated is a stereo satellite image pair, the present invention uses the rearview image of the stereo pair as the image to be updated and performs a difference operation with the low-altitude data base image to obtain the outline area of the building to be updated, and then calculates the height of the outline area of the building to be updated based on the forward intersection technology of the stereo pair.

[0041] If the image to be updated is a monocular orthophoto, a difference operation is performed between the image to be updated and the low-altitude data base image to obtain the outline area of the building to be updated. Then, the deep learning neural network model is used to quickly calculate the height of the outline area of the building to be updated.

[0042] Specifically, if Figure 2-8 A method for quickly updating a white model of a building in a large-scale regional scene comprises the following steps:

[0043] S1. Obtain an image to be updated and, based on an RPC parameter file for the image to be updated, preliminarily calculate the extent of the image area to be updated within a low-altitude data base image. The image to be updated is a co-orbital stereo satellite image pair or a monocular orthophoto image. The low-altitude data base image includes at least one of a real-world 3D model, an aerial photogrammetric image, or airborne LiDAR data.

[0044] S2, obtain the corresponding homonymous points of the low-altitude data base image and the image to be updated through an automated matching algorithm, and calculate the affine transformation parameters based on the corresponding relationship between the homonymous points;

[0045] S3, using affine transformation parameters to perform polynomial geometric correction on the image to be updated, so that the image to be updated is aligned with the low-altitude data base image in spatial position, so that the same features in the image to be updated and the low-altitude data base image are in the same position as much as possible.

[0046] S4: Extract building outlines from the low-altitude data base image and the image to be updated. Specifically, the HRNet network, through its unique multi-resolution parallel structure and feature fusion strategy, effectively captures both details and global information while maintaining high-resolution features. This paper uses HRNet-18 as the building feature extraction framework and combines spatial pyramid pooling to fuse features at different depths to extract building outlines from the low-altitude data base image and the image to be updated.

[0047] S5, since the building outlines extracted by the automated algorithm are not regular and complete, it is necessary to regularize the building outlines and generate regularized polygons.

[0048] Regularized adjustments are made to the building outline, including long edge direction classification, short edge direction correction, and line segment merging. Specifically, different edge thresholds are set based on the building area, and the edges are divided into long and short edges. The direction of the longest edge is added to the main direction list. The directions of other long edges are compared with the main direction in the list. If the angle is within the threshold, it is added to the list. The long edge is rotated to be close to the main direction or its perpendicular direction. The short edge is adjusted to be parallel or perpendicular to the main direction. Parallel lines with a distance less than the threshold are merged or connected. All adjusted lines are connected to form the final polygon outline.

[0049] S6, based on step S5, a binary image of the building outline (0: background, 1: building) of the low-altitude data base image (within the image to be updated) and a binary image of the building outline (0: background, 1: building) of the stereo image pair to be updated (rearview image) can be obtained, and a difference operation is performed on the binary image of the building outline of the low-altitude data base image and the image to be updated to determine the white mold area of the building to be updated;

[0050] S7, calculate the height data of the white model area of the building to be updated; if the image to be updated is a co-orbital stereo satellite image pair, calculate the height data of the area to be updated based on the RPC parameters and the forward intersection algorithm; if the image to be updated is a monocular orthophoto, invert the height data of the area to be updated based on the pre-trained deep learning model.

[0051] Specifically, if Figure 1, define the local tangent plane coordinate system O-XYZ as the object space coordinate system, and the origin of the coordinate system is located near the center point of the ground coverage range of the stereo model; PRP represents the object reference plane, which is the object space average elevation plane with an elevation of H; point p1 and point p2 are any pair of conjugate points (image points with the same name) on the left and right scenes; point P is the ground point corresponding to point p1 and point p2; according to the definition of the line array image scene epipolar line based on the projection trajectory method, the right scene Ep1 represents the epipolar curve corresponding to p1 on the left scene, and the left scene Ep2 represents the epipolar curve corresponding to p2 on the right scene; EP is the approximate epipolar surface; ED is the approximate epipolar line; AP represents the rectangular area range of the epipolar line image on the reference plane PRP, and the horizontal side of the rectangle is parallel to the epipolar line direction.

[0052] For a co-track stereo pair with RPC parameters, the image to be updated is automatically extracted from the stereo pair based on the building outline area to be updated. The coordinates of the points with the same name, p1(c1, r1) and p2(c2, r2), are automatically extracted from the stereo pair based on the building outline area to be updated. c is the column number of the point in the image, and r is the row number of the point in the image. The geometric relationship between the object and image space of the foreground and backview images of the stereo pair can be described by building a rational function model using the RPC parameters of the stereo pair. The ground point coordinates are then calculated using the least squares method for stereo intersection, ultimately obtaining the height data for the white model area of the building to be updated.

[0053] In the case where the image to be updated is a monocular orthophoto, the open source DINV2 open source model is used to pre-train the building height inversion model based on the low-altitude data base. At the same time, the model is fine-tuned for the height data calculated for the area with stereo image pairs. Finally, the height data of the building white model area to be updated is deduced based on the model.

[0054] S8: Fusing the updated height data with the white model data of the low-altitude data base to generate updated building white model data. The fusion process includes spatially superimposing the newly added or changed building height data with the original white model data in the low-altitude data base and correcting the topological consistency.

[0055] S9, repeating steps S1-S8 for the newly added images to be updated, and continuously updating the white model data of the low-altitude data base.

[0056] This invention rapidly updates building white model data for large-scale regional scenarios by constructing a method for rapidly updating building white models. With the rapid development of the low-altitude economy, the timeliness and update frequency of building white models are required to be higher. The technical solution of this patent can effectively and quickly complete the rapid update of low-altitude urban bases, providing safe flight line protection for low-altitude unmanned aerial vehicles (UAVs) such as obstacle avoidance and path planning.

[0057] The present invention also discloses a computer-readable storage medium and a computer system. The medium stores a computer program, which, when executed, performs any of the methods described above. A computer system includes a processor and a storage medium, wherein the storage medium stores the computer program, and the processor reads and executes the computer program from the storage medium to perform any of the methods described above.

[0058] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. A skilled person may implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention.

[0059] In an embodiment, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or codes. Computer-readable media include both computer storage media and communication media, including any media that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium that can be accessed by a computer. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Any connection is also properly referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of medium. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0060] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0061] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for rapidly updating white models of buildings in large-scale regional scenes, characterized in that: The following steps are involved: S1, obtain the image to be updated, and preliminarily calculate the range of the image area to be updated in the low-altitude data base image based on the RPC parameter file of the image to be updated; S2, obtain the corresponding homonymous points of the low-altitude data base image and the image to be updated through an automated matching algorithm, and calculate the affine transformation parameters based on the corresponding relationship between the homonymous points; S3, using affine transformation parameters to perform polynomial geometric correction on the image to be updated so that it is aligned with the spatial position of the low-altitude data base image; S4, extracting building outlines from the low-altitude data base image and the image to be updated; S5, regularizing and adjusting the building outline to generate regularized polygons; S6, performing a difference operation on the low-altitude data base image and the building outline binary image of the image to be updated to determine the white model area of the building to be updated; S7, calculating the height data of the white mold area of the building to be updated; S8, fusing the updated height data with the low-altitude base white mold data to generate updated building white mold data; S9, repeating steps S1-S8 for the newly added images to be updated, and continuously updating the white model data of the low-altitude data base.

2. The method according to claim 1, wherein: The image to be updated in step S1 is a co-orbital stereo satellite image pair or a monocular orthophoto image.

3. The method according to claim 1, wherein: In step S4, HRnet-18 is used as the skeleton for building feature extraction, and spatial pyramid pooling is combined to fuse different depth features to extract building contours from the low-altitude data base image and the image to be updated.

4. The method according to claim 1, wherein: In step S5, regular adjustment of the building outline includes classification of long side directions, correction of short side directions and merging of line segments; specifically, different edge thresholds are set according to the building area, and the edges are divided into long sides and short sides. The direction of the longest side is added to the main direction list, and the directions of other long sides are compared with the main direction in the list. If the angle is within the threshold range, it is added to the list, the long side is rotated to the main direction or near its perpendicular direction, the short side is adjusted to be parallel or perpendicular to the main direction, and parallel lines with a distance less than the threshold are merged or connected. All adjusted lines are connected to form the final polygon outline.

5. The method according to claim 2, wherein: In step S7, if the image to be updated is a pair of co-orbital stereo satellite images, the height data of the area to be updated is calculated based on the RPC parameters and the forward intersection algorithm; if the image to be updated is a monocular orthophoto, the height data of the area to be updated is inverted based on the pre-trained deep learning model.

6. The method according to claim 5, characterized in that: In step S7, the coordinates of the same-name image points in the front and back images of the same-orbit stereo satellite image pair are used in combination with the RPC parameters to construct a rational function model, and the height of the ground points in the area to be updated is calculated by least squares forward intersection; in addition, the deep learning model adopts the DINV2 open source framework, and the building height inversion model is pre-trained based on the low-altitude data base. At the same time, the model is fine-tuned for the height data calculated for the area with stereo image pairs, and finally the height data of the building white model area to be updated is deduced and calculated based on the deep learning model.

7. The method according to claim 1, wherein: The fusion process in step S8 includes spatially superimposing the newly added or changed building height data with the original white model data in the low-altitude data base and correcting the topological consistency.

8. The method according to claim 1, wherein: The low-altitude data base image includes at least one of a real-scene three-dimensional model, an aerial photogrammetry image, or airborne LiDAR data.

9. A computer-readable storage medium, characterized in that: The medium stores a computer program, and after the computer program is run, the method according to any one of claims 1 to 8 is executed.

10. A computer system, characterized in that: The method comprises a processor and a storage medium, wherein the storage medium stores a computer program, and the processor reads and runs the computer program from the storage medium to execute the method according to any one of claims 1 to 8.

Images