A house adaptive vectorization method combining LiDAR point cloud and optical image

Abstract

The application discloses a method for automatically identifying and vectorizing houses by combining LiDAR point clouds and optical images, and uses the height optical image nDSM of the point cloud data and the edge profile information of the height optical image as input bands to participate in the multi-scale elevation collaborative segmentation of the optical image; an optimization method for sorting plants and houses based on the object gradient and the elevation spatial relationship is provided; for the edge information recognition result of the house, a fine vectorization method for the house profile driven by the corner point recognition is provided; the application can effectively identify the four-prism-shaped house and the quadrangle house, has strong adaptability to the house with small accessories on the roof surface, can correctly identify the house cluster composed of the herringbone-shaped house, the I-shaped house or various types of houses, and achieves high-quality identification and house self-adaptive vectorization effect based on the multi-level complementary advantages of the elevation, echo, spectrum and gradient and the like.

Description

Technical Field

[0001] This application relates to an adaptive vectorization method for houses from optical images, and more particularly to an adaptive vectorization method for houses that combines LiDAR point clouds and optical images, belonging to the field of remote sensing house vectorization technology. Background Technology

[0002] Buildings are the most prominent geographical element in cities. Building extraction based on remote sensing technology can be used for urban spatial planning and management. Traditional remote sensing technology mainly focuses on the distribution analysis of building rooftops. With the advent of the smart city construction boom, building extraction and modeling that combines multi-source data and multi-scale, multi-dimensional space has important applications. In smart city construction, the real-time, intelligent, and efficient management and planning of cities all require high-precision building models, primarily in the form of measurable, real-world 3D building models. Currently, 3D building modeling technology mainly focuses on stereoscopic measurement based on oblique photogrammetry. Figure 3 While 3D modeling is possible, it cannot materialize ground features. 3D modeling based on LiDAR point clouds and oblique photogrammetry allows for detailed modeling of building entities and spatial measurement, leading to a clearer spatial perception for urban planning. Buildings in urban spaces possess distinct shapes and spatial locations. Feature extraction based on the spectral characteristics of remote sensing images is feasible for automated building extraction, but it also faces numerous challenges. Therefore, as the most frequently updated geographic data in the urban environment, the automation of building identification and extraction based on high-resolution remote sensing images holds immense promise.

[0003] With the technological advancements in remote sensing data sensors, the diversity and resolution of remote sensing data sources have significantly improved. Regional-level remote sensing image interpretation and recognition can be achieved using low- to medium-resolution images, but the extraction and analysis of urban spatial features requires high-resolution images to provide detailed information. In high-resolution images, the detailed features and texture attributes of features are clearer and more apparent. Furthermore, the high resolution enhances spatial information such as feature shape and the relationships between adjacent features, expanding the feature space for feature identification. However, building extraction based on high-resolution remote sensing images faces numerous challenges. Issues such as spectral confusion between objects with similar or dissimilar spectra, and the excessive complexity and richness of feature types and texture details can lead to missed detections and misclassifications when using high-resolution remote sensing images alone for building extraction, and the level of automation is also low.

[0004] Airborne LiDAR is an active remote sensing technology. Its main data product is discrete 3D laser point clouds. Each discrete point contains 3D coordinates (x, y, z), laser corner reflection intensity, and laser echo information. LiDAR point cloud data can directly obtain the geographic reference coordinates and high-precision 3D spatial location of ground features. Furthermore, the acquisition method of point cloud data is flexible, and as an auxiliary remote sensing data source, it can further improve the accuracy of ground feature classification. While buildings exhibit relatively regular patterns in LiDAR point cloud datasets, and the spatial nature of point cloud data enhances the distinguishability of buildings from surrounding features, thus offering advantages in building extraction, several shortcomings remain. These shortcomings prevent a fundamental improvement in the accuracy of building extraction across urban areas. The main reasons include: First, limitations in point cloud sensor technology result in generally low planar resolution of point clouds scanned by airborne LiDAR. Even achieving perfect extraction accuracy in point clouds is not particularly meaningful, as the most significant drawback of point cloud data compared to remote sensing images is the lack of rich spectral and semantic information. Second, the increasing complexity of building materials, structures, and shapes makes using airborne LiDAR point cloud data alone inefficient and ineffective for building identification, as it involves significant manual editing and misclassification. Third, while LiDAR point cloud data includes laser signal intensity information, which reflects different ground materials and serves as a feature for ground feature identification, the intensity reflection signals acquired by sensors have large errors and cannot accurately reflect the category information of ground features. Therefore, ground feature classification based on intensity information and other features in LiDAR point cloud data fails to achieve the desired results and introduces numerous interfering factors.

[0005] Low-to-medium resolution remote sensing images are limited by their resolution, making it difficult to extract buildings. With the development of high-resolution remote sensing images and richer details of ground features, new challenges have emerged in building extraction, such as occlusion, spectral obfuscation, and overly complex ground features. Extracting buildings in built-up areas based on high-resolution remote sensing and aerial images involves two approaches: the first is a bottom-up feature combination approach, which starts with the low-level features of the remote sensing data source and identifies buildings based on the extraction of multiple features; the second is a top-down prior model approach, which abstracts relevant feature models based on target model features and identifies building targets in optical images based on this abstract model.

[0006] Existing technologies for house recognition based on remote sensing data primarily rely on the spectral features of remote sensing images. They extract distinguishing features from the shape, texture, and semantic characteristics of houses as they appear in the remote sensing images. In addition, supplementing the feature space with various remote sensing auxiliary data sources can also increase the accuracy of house recognition to some extent.

[0007] Existing technologies, based on spectral information from optical images and Hough line segment recognition, roughly obtain the outlines of houses. However, methods based on geometric boundaries do not utilize texture features or spatial relationships between individual houses. House extraction dominated by such methods often suffers from misidentification and missed detection. Simply fitting the roof area of ​​a house with its simple geometric shape lacks scalability for applications involving building complexes. Other optical image segmentation methods for house recognition require high initial segmentation quality and struggle to handle the occlusion of buildings, vegetation, and shadows in complex terrain environments.

[0008] Model-driven building identification methods utilize high-resolution remote sensing data from multiple data sources and multiple spatial and temporal dimensions to extract urban building targets from mathematically modeled feature models and prior expert knowledge. However, existing models, being model-based, do not consider building shape in the initial stage, resulting in irregularly shaped extracted building contours that require regularization. Furthermore, they struggle to simulate the spatial semantic relationships between ground features, leading to significant gross errors in the identification results. Existing technologies, based on shape-regular building knowledge models, lack robustness and adaptability in complex built-up areas.

[0009] When using LiDAR data sources alone for house extraction, the relevant filtering is difficult to determine, thus lacking universality. Moreover, the absence of spectral information from plants leads to numerous misclassifications. Filtering and identification of houses based on slope features is ineffective. Due to the resolution limitations of point cloud data sources and the easy confusion between plant and house elevations, the accuracy of house extraction is not high.

[0010] In summary, while some progress has been made in building identification and extraction based on remote sensing images or joint auxiliary information, many technical challenges and extraction bottlenecks remain. These include both algorithmic issues and problems in processing remote sensing data sources, primarily manifested in:

[0011] First, building identification in built-up areas based on high-resolution remote sensing images inevitably encounters the problem of shadow occlusion, which becomes more serious as the complexity of urban building shapes and heights increases. In the shadow areas of buildings and plants, the spectral information of ground objects will be distorted. If existing technologies do not take this into account, there will be missed detections and false detections. For example, the normalized plant index value of the roof of a building is similar to that of the plant under shadow, which leads to misclassification. This is because the plant index value calculated from the spectral value of the plant under shadow is lower. Therefore, in complex built-up areas, the accuracy of distinguishing plants and buildings in shadow areas by using high-resolution remote sensing images and plant indices alone is not high, which ultimately leads to a low accuracy rate of building identification.

[0012] Secondly, in the classification and extraction of ground features from high-resolution remote sensing images, existing technologies, in the absence of DSM height information, may misclassify rooftops and roads due to the similarity of their spectral features caused by the similarity of their building materials. In addition, the color of rooftops in built-up areas can also affect the accuracy of spectral information classification. Especially in recent years, with the improvement of building technology and aesthetic requirements, there are many blue, green, or cool-colored rooftops with planted lawns, which can cause serious confusion in plant index monitoring, increase the false negative rate of building identification, and result in very low building identification accuracy. As a result, adaptive vectorization of buildings loses its practical application value.

[0013] Third, using high-resolution remote sensing images alone for building extraction suffers from missed detections and misclassifications, and its automation level is low. Furthermore, due to limitations in point cloud sensor technology, the planar resolution of point clouds scanned by airborne LiDAR is generally low, lacking rich spectral and semantic information. Even if perfect extraction accuracy is achieved in point clouds, it still has little practical significance. Using airborne LiDAR point cloud data alone for building identification is ineffective and inefficient, involving significant manual editing and misclassification. The intensity reflection signals of ground features acquired by LiDAR sensors have large errors and cannot accurately reflect the category information of ground features. Ground feature classification based on intensity information and other features using LiDAR point cloud data fails to achieve the desired effect and instead introduces many interfering factors, failing to fundamentally improve the accuracy of building extraction in urban areas.

[0014] Fourth, the processes and steps of optical image data sources and image segmentation algorithms have not been scientifically validated. The parameter settings for image segmentation and recognition in existing technologies are highly arbitrary, and parameter errors can affect the quality of house extraction. The outlines extracted by existing technologies are not refined enough, with edge information loss or protrusion. The feature constraints are not strong, resulting in false detections and false negatives in the extraction results. The usability and reliability are poor, and it cannot be extended to many important fields such as house vectorization. At the same time, existing technologies only utilize the elevation features of LiDAR point cloud data and do not introduce other features for multi-feature house recognition, which greatly limits their application. The poor accuracy makes them almost useless in practical applications. Summary of the Invention

[0015] This application addresses the spectral obfuscation phenomenon in building identification using high-resolution remote sensing images. It incorporates elevation information from airborne LiDAR point clouds for optimization. In the segmentation-classification process of optical images, elevation features are included as one of the bands for segmentation and classification, increasing the vertical spatial differences of different ground features with the same spectral characteristics, thus enabling ground feature extraction and classification. High-resolution remote sensing image data and airborne LiDAR data have strong synergistic characteristics. Based on cutting-edge technologies of multi-source data fusion and ground feature recognition, this application comprehensively utilizes the advantages of both data sources to accurately and efficiently obtain the building outline distribution information of built-up areas. This is of great significance for many urban planning applications such as urban spatial management, land type distribution analysis, and natural disaster assessment, and also has important engineering application value for remote sensing image interpretation.

[0016] To achieve the above technical effects, the technical solution adopted in this application is as follows:

[0017] An adaptive vectorization method for houses combining LiDAR point cloud and optical imagery is proposed. This method extracts houses in built-up areas from high-resolution remote sensing images and airborne LiDAR point cloud data using multiple feature spaces, and then performs adaptive thinning and vectorization of the results.

[0018] (I) Core Method: An automatic house recognition vectorization method combining LiDAR point cloud and optical image, including: first, multi-scale elevation collaborative segmentation of images combining LiDAR point cloud data; second, optimized sorting of houses and plants based on gradient space analysis; and third, house recognition and fine-grained vectorization of contours.

[0019] Based on multi-scale elevation collaborative segmentation of remote sensing images, the nDSM (nearest-distributed surface image) of point cloud data and the edge contour information of the nDSM are used as input bands to participate in the multi-scale elevation collaborative segmentation of optical images, improving the segmentation quality and preparing for further classification. In the process of building contour recognition based on point cloud elevation images, a sorting optimization method for plants and buildings based on land feature gradient and elevation spatial relationship is proposed, including plant identification and filtering based on gradient spatial features and building edge restoration. For the building edge information recognition results, a corner recognition-driven fine vectorization method for building contours is proposed, mainly including: corner recognition-driven initial optimization of building contours and cyclic recursive compression of building contours fine vectorization.

[0020] (II) Process Steps: Automatic identification and adaptive vectorization of buildings in built-up areas, including:

[0021] Step 1: Preprocessing and feature recognition of LiDAR data;

[0022] Step 2: Feature analysis and setting of high-resolution remote sensing images;

[0023] Step 3: Multi-scale collaborative elevation segmentation based on multi-source data;

[0024] Step 4: Initial House Identification (Object-Oriented);

[0025] Step 5: Shadow recognition in remote sensing images;

[0026] Step Six: Detailed Identification of the House;

[0027] Step 7: Thinning and vectorizing the building outline;

[0028] An automatic building recognition vectorization method based on LiDAR point cloud and optical image is proposed. Following a multi-level recognition process of segmentation followed by classification, the method extracts two-dimensional roof patches of buildings with refined contours, thereby achieving adaptive building vectorization. This method has good adaptability to buildings in built-up areas with different shapes and occlusion.

[0029] An adaptive vectorization method for buildings, combining LiDAR point cloud and optical image, further involves multi-scale elevation collaborative segmentation of the point cloud nDSM and edge images: the initial LiDAR point cloud consists of discrete data points in 3D space, with point location information including 3D coordinates (x, y, z), laser corner echo information, and corner reflection intensity information. The acquired LiDAR point cloud elevation information is used as a collaborative feature for segmentation, and an absolute height information model nDSM for ground features is obtained. The absolute height optical image of the point cloud is used as an input band for multi-scale ground feature segmentation. In the optical image segmentation and classification, incoherent ground features are filtered out by setting elevation thresholds.

[0030] Edge recognition is used to obtain the outline information of major land features such as houses and plants in the nDSM elevation image. In the subsequent segmentation, the outline information of the land features is used as the input band for segmentation. In multi-scale elevation collaborative segmentation, the outline information of the land features is referenced for segmentation to improve the segmentation accuracy.

[0031] An adaptive vectorization method for houses combining LiDAR point clouds and optical images, further including plant identification and filtering based on gradient spatial features:

[0032] The elevation gradient between different ground features serves as an indicator for distinguishing between features at similar heights. The rate of elevation change is more pronounced at the edges of ground features or in surface areas with varying surface roughness. The gradient represents the inclination and roughness of the spatial surface. Gradient simulation is performed using the second derivative of the 3D surface, and the calculation formula is as follows:

[0033]

[0034] Where (x,y) are points on the surface. These are partial derivatives;

[0035] On relatively smooth building surfaces, ground surfaces, or other land features, local elevation changes are not prominent enough, i.e., the gradient spatial features are not large, and they appear as dark tones in gradient images. On uneven plant surfaces, not only are the elevation changes large at the edges, but there are also elevation changes in different directions within the regions, resulting in large gradient values, which appear as highlights in gradient images.

[0036] After obtaining the gradient space feature results, the house and plant areas have certain differences in gradient space features. By setting the gradient threshold, binary images of the plant area and other areas are obtained. After filtering out the low gradient area, the high gradient area is set to white (255) and the low gradient area is set to black (0), which makes it easier to observe the results visually. Finally, the output results are inverted.

[0037] An adaptive vectorization method for houses combining LiDAR point clouds and optical images is proposed. Further, the method involves house edge recovery: the first echo signal of the laser signal mainly includes the surface information of houses, plants and other relatively complete ground features. The house edges and plant areas contain areas with large gradient values. In the last echo, the main features are the information of houses, ground points or other relatively low-lying plants. The areas with large gradient values ​​are mainly at the house boundaries. The gradient of the last echo image is calculated to obtain the house boundary area. The other ground features included are mainly low-lying plants. The missing house edges are obtained by filtering by setting an elevation threshold.

[0038] After obtaining the surface regions of houses based on LiDAR point cloud nDSM optical images, the initial complete outline information of the houses is obtained. Based on prior knowledge, a square buffer with a point spacing of 1 is created in the house region. Plant filtering based on gradient spatial features will include some residual plant information. In addition, a small amount of plant information will also be brought in during house edge restoration. In the object-oriented image house extraction results, the plant recognition effect is excellent in house recognition based on image spectral features, but the edge information of the houses has the problem of shadow occlusion and missing information. Therefore, the point cloud data and the initial extraction results of houses from optical images have good complementarity.

[0039] An adaptive vectorization method for houses combining LiDAR point clouds and optical images is proposed. Further, corner recognition-driven initial optimization of the house outline is implemented: by utilizing the curvature and gradient changes of pixels, extreme points of curvature change are found as candidate corners for corner recognition. This method exhibits excellent performance and stability, and mainly includes:

[0040] Step 1: Calculate the correlation matrix N of the pixels and store the directional gradient values;

[0041]

[0042] J(x,y) represents grayscale pixels, k(x,y) represents the sliding window, and J xJ y The derivative of the image in the x and y directions;

[0043] Step 2: Calculate the corner response function value T of the pixel:

[0044] T = (DE - SA) 2 w(D+E) 2 Formula 6

[0045] w is an empirical constant;

[0046] Step 3: Find the extreme points in the response function, compare them with the given critical value, and set them as corner points.

[0047] An adaptive vectorization method for houses combining LiDAR point clouds and optical images is proposed. Further, a fine vectorization of the house outline is achieved through recursive compression: the corner point set identified by corner point recognition is connected end-to-end into a closed polygon based on least-squares matching Euclidean distance, simplifying the data volume while preserving the house outline information. The entire curve is recursively bisegmented. The fine vectorization of the house outline through recursive compression includes:

[0048] Step 1: Locate the feature point of the linear feature, i.e., the point Q with the maximum vector radius when connecting the corner point of corner identification to the start and end points Q1 and Q2 of the local curve. If the radius is greater than a given critical value Q... max If it is not present, keep it; otherwise, remove it.

[0049] Step 2: Find a new feature point Q, divide the local curve into two segments, and perform the calculation in step 1 on each segment;

[0050] Step 3: Iterate and calculate until no new feature points Q appear. Finally, a series of feature points and the start and end points of local curves are obtained. Connecting the start and end points with straight lines will yield the thinned vectorized linear features.

[0051] An adaptive vectorization method for houses combining LiDAR point clouds and optical images is further described in step one, which involves preprocessing and feature recognition of LiDAR data. First, point cloud filtering is performed. By utilizing the elevation changes in terrain undulations, non-ground points are filtered out to generate a digital elevation model (DEM). The point cloud data is filtered using a progressive triangulation method and high-resolution sampling based on the image is performed to provide high accuracy for generating an absolute elevation feature (nDSM). After upsampling, the 3D point cloud image and the aerial image are adjusted using the forward intersection method based on collinearity equations. The elevation image and the aerial image data are then registered.

[0052] (1) Elevation Feature Recognition: The initial airborne LiDAR point cloud data contains spatially resolved elevation features of ground objects. After obtaining the ground point elevation model, the absolute elevation model nDSM of the ground objects is obtained based on the DSM optical image obtained by interpolation from the last echo in the dataset. The computer is:

[0053] nDSM(x,y)=DSM(x,y)-DTM(x,y) Equation 7

[0054] (2) Gradient spatial feature recognition: On the uneven surface of plants or the edge of houses, not only is there a large elevation change in the edge area, but there are also elevation changes in different directions inside the area, resulting in a large gradient spatial value. The gradient optical image is obtained based on the gradient amplitude solution algorithm, and the plant area appears bright.

[0055] An adaptive vectorization method for houses using LiDAR point clouds and optical images is further described in step two: feature analysis and setting of high-resolution remote sensing images. The high-resolution remote sensing image, a color infrared image, contains three bands: near-infrared (NIR), red (R), and green (G). Plant information in the color infrared image is represented by red. The definition of the plant normalization index is as follows:

[0056]

[0057] Where E NIR For the near-infrared band, E R It is in the infrared band;

[0058] Step 3, Multi-scale Elevation Collaborative Segmentation Based on Multi-source Data: Multi-scale elevation collaborative segmentation of ground features is performed by integrating elevation information from LiDAR point cloud data and shape information from high-resolution remote sensing images. The information advantages of combining multiple data sources are combined to improve the accuracy of building identification.

[0059] After introducing LiDAR point cloud elevation information, to address the issues of shadow occlusion of plants and houses and spectral confusion of ground features at different heights, the weight of elevation information was increased and set to 5; accurate house segmentation is of paramount importance, so the shape factor of the segmentation parameter was set to 0.5, so that houses with regular geometric information can achieve better segmentation results.

[0060] Step 4, object-oriented initial building identification: Low-lying features are masked by setting an elevation threshold to reduce the complexity of building identification. The elevation threshold is set to a range of H = [1.5, 1.7] meters. Features with an elevation value greater than [1.5, 1.7] meters are defined as the initial building selection area.

[0061] After masking with an elevation threshold, some vegetation, ground features, and low-lying infrastructure such as cars are filtered out. At this point, the main feature types include houses, vegetation, and shadows. The plant normalization index Rn is obtained, and a threshold value Rn is set. 临界值 =0.208, for areas outside the masked region, the initial identification of houses is performed according to the following rules:

[0062] Rn≥Rn 临界值 Identified as plant-type 9

[0063] Rn<Rn 临界值 Identified as house type 10

[0064] Based on the above rules, the initial identification of the house is obtained. The initial identification result of the house includes plants and shadows.

[0065] An adaptive vectorization method for houses combining LiDAR point cloud and optical imagery, further, step four, fine identification of houses: using the height image nDSM generated by LiDAR point cloud data to further refine the identification results of houses. LiDAR laser signal has penetrability, and the shadow occlusion phenomenon shown in remote sensing image is not affected in point cloud echo information.

[0066] Although airborne LiDAR point clouds can identify the outline information of houses, the house outline serves as a reference constraint framework for fine house identification. The occlusion problem is solved based on the penetrability of laser signals. First, the house outline of the depth optical image nDSM is identified, and house patch optimization and sorting are performed on the nDSM single-band image.

[0067] (1) Set an elevation threshold to filter out low-lying features. In this application, the threshold value H = 1.7m is set to remove the interference of low-lying features through elevation filtering.

[0068] (2) After elevation filtering, the main land features in the optical image include plants and houses. Therefore, the house outline is identified by the house and plant optimization sorting based on gradient spatial analysis proposed in this application. After obtaining the gradient optical image, the gradient threshold is set, the plant information is filtered out, i.e., the house patch is identified, and the edge of the last echo house is added to the gradient optical image. Finally, an area filter with a threshold of 200 is performed.

[0069] After obtaining the outline of the house from the LiDAR point cloud nDSM image, the outline information of the house is basically complete. The point cloud data and the initial recognition result of the house in the image have good complementarity. By overlaying and parsing the initial recognition result obtained from the optical image of the house, the intersection of the two results can combine the advantages of the data and filter out messy misclassified ground features.

[0070] The two recognition results were overlaid and parsed using the MATLAB platform. The regular house edge information and high-quality plant recognition information in the optical image were complemented by the shadow occlusion suppression of houses or plants in the point cloud data to obtain high-quality recognition results. Finally, area filtering and morphological opening operations with a given threshold were performed to remove the holes in the patches and houses in the results.

[0071] A house adaptive vectorization method combining LiDAR point cloud and optical image, further, step seven, house outline thinning vectorization: based on the house roof patch recognition results obtained by overlay analysis, the occlusion of the house by plant shadows or house shadows is corrected to a certain extent, and the advantages of the two recognition results are complementary. The outline of the recognition results is finely vectorized according to the house outline fine vectorization method driven by corner point recognition.

[0072] Corner recognition-driven initial optimization of house outline preserves the house outline while reducing data volume, thus improving the efficiency of subsequent thinning processing.

[0073] Corner point recognition connects the corner points end to end to form a closed polygon, which simplifies the amount of data while preserving the outline information of the house. The refined vectorization of the house outline through recursive compression divides the entire curve into two segments. However, the large amount of data can easily lead to stack overflow due to excessive depth. The results of corner point recognition control the amount of data and improve recursion efficiency.

[0074] Compared with existing technologies, the innovations and advantages of this application are as follows:

[0075] First, this application addresses the spectral confusion phenomenon in building identification based on high-resolution remote sensing images by introducing elevation information from airborne LiDAR point clouds for optimization. In the segmentation and classification process of optical images, elevation features are included as one of the bands in the segmentation and classification, increasing the differences between different ground features in vertical space under the same spectral features, thus enabling the extraction and classification of ground features. High-resolution remote sensing image data and airborne LiDAR data have strong synergistic characteristics. Based on cutting-edge technologies of multi-source data fusion and ground feature recognition, this application comprehensively utilizes the advantages of both data sources to accurately and efficiently obtain the distribution of building outlines in built-up areas. This is of great significance for many urban planning applications such as urban spatial management, land type distribution analysis, and natural disaster assessment, and also has important engineering application value for remote sensing image interpretation.

[0076] Secondly, building identification in built-up areas based on high-resolution remote sensing images inevitably encounters the problem of shadow occlusion, which becomes more severe as the complexity of urban building shapes and heights increases. Within the shadow areas of buildings and vegetation, the spectral information of ground features is distorted. This application focuses on addressing the missed detections and false detections in existing technologies, improving the accuracy of building identification. Since LiDAR point clouds generally have low planar resolution and do not include rich spectral and semantic information, even achieving perfect extraction accuracy in point clouds is not very practical. Combining LiDAR point clouds with optical images significantly improves the effectiveness and efficiency of building identification. Considering issues such as shadow areas and spectral confusion in complex built-up areas, this application optimizes image segmentation algorithms, introduces multi-feature and contour thinning vectorization processing of LiDAR point cloud data for building identification, and improves the quality of roof patch extraction in built-up areas, fundamentally enhancing the accuracy of various types of building extraction in urban areas.

[0077] Third, while high-resolution remote sensing images are rich in ground feature details, they are prone to spectral confusion due to the presence of objects with the same spectral signature and the shadowing problem of tall objects. Using remote sensing images alone for building identification presents significant challenges. Furthermore, with the expansion of urban areas and the increasing complexity of environments, the accuracy of building identification urgently needs improvement. Airborne LiDAR data, however, contains 3D elevation information of ground features, effectively distinguishing objects with the same spectral signature and shadowed features that are difficult to differentiate in high-resolution remote sensing images in vertical space. Therefore, this application combines high-resolution remote sensing images with airborne LiDAR point cloud data to extract buildings in built-up areas using multiple features in a multi-feature spatial model. The results are then adaptively thinned and vectorized. This approach considers the various types of ground feature details and spectral confusion in complex urban environments. Based on multiple features such as elevation, echo, spectrum, and gradient, a multi-level complementary approach to building identification is adopted, resulting in faster identification and vectorization. The method exhibits high sensitivity, reliability, and practicality. Experimental results are consistent with theoretical methods, achieving excellent identification and adaptive building vectorization performance.

[0078] Fourth, the core method of this application is an automatic building recognition vectorization method that combines LiDAR point cloud and optical image. It uses the nDSM (nearest-distributed surface image) of the point cloud data and the edge contour information of the nDSM as input bands to participate in multi-scale elevation collaborative segmentation of the optical image, improving segmentation quality and preparing for further classification. It proposes an optimized sorting method for plants and buildings based on land feature gradients and elevation spatial relationships. For the edge information recognition results of buildings, it proposes a corner point recognition-driven method for fine vectorization of building contours. This application can effectively identify pyramidal buildings and courtyard-style buildings, and has strong adaptability to buildings with small attachments on the roof surface. It can correctly identify building clusters composed of A-frame, I-frame, or multiple types of buildings. Following a multi-level recognition process of segmentation followed by classification, it extracts two-dimensional roof patches with refined contours, achieving adaptive building vectorization. It has good adaptability to buildings of various shapes in built-up areas and shading. Attached Figure Description

[0079] Figure 1 This is a flowchart of a method for automatically identifying buildings by combining LiDAR point cloud data and high-resolution remote sensing images.

[0080] Figure 2 The point cloud absolute height image is used as the input band to perform multi-scale ground feature segmentation sample images.

[0081] Figure 3 It is a multi-scale elevation co-segmentation sample image of the combined point cloud nDSM and edge.

[0082] Figure 4 This is a diagram showing the effect of multi-scale segmentation of a single high-resolution remote sensing image.

[0083] Figure 5 This is a multi-scale segmentation effect diagram of high-resolution remote sensing images using LiDAR data.

[0084] Figure 6 It is a rendering of the plant and shadow segmentation effect from a single high-resolution remote sensing image.

[0085] Figure 7 This is a diagram showing the segmentation of plants and shadows in a high-resolution remote sensing image based on LiDAR data.

[0086] Figure 8 This is a schematic diagram of plant identification results based on gradient spatial features.

[0087] Figure 9 This is a schematic diagram of the gradient space detection and restoration process at the edge of a building.

[0088] Figure 10 The gradient optical image is obtained by solving the gradient space magnitude algorithm.

[0089] Figure 11 This is a schematic diagram of the multi-scale elevation collaborative segmentation parameter settings for multi-source data.

[0090] Figure 12 This is a schematic diagram illustrating the effect of multi-scale elevation collaborative segmentation based on multi-source data.

[0091] Figure 13 This is a schematic diagram illustrating the initial house identification effect using object-oriented methods.

[0092] Figure 14 A diagram illustrating the presence of residual plant information and voids caused by rooftop attachments.

[0093] Figure 15 This is a schematic diagram illustrating the effect of vectorization of the building area.

[0094] Figure 16 It is an evaluation diagram overlaid with the adaptive vectorization results of the house. Specific implementation methods

[0095] The technical solution of the adaptive vectorization method for buildings combining LiDAR point cloud and optical image provided in this application will be further described below with reference to the accompanying drawings, so that those skilled in the art can better understand this application and implement it.

[0096] Buildings are the most typical urban features, and the identification of buildings and houses plays a crucial role in the interpretation of remote sensing data. In urban planning and management, population density surveys, and urban emergency safety construction, large-scale building identification and change detection based on remote sensing images play a vital role. In complex urban environments, high-resolution remote sensing images are rich in feature details, but this also easily leads to spectral confusion between objects with similar or dissimilar spectra. The spectral information of buildings and other features, or between other features, exhibits irregular similarities, and there is the problem of shadows cast by tall objects. Furthermore, with the expansion of urban scale and the increasing complexity of the environment, the accuracy of building identification urgently needs to be improved. Building identification using high-resolution remote sensing images alone faces many challenges. Airborne LiDAR data contains 3D elevation information of features, effectively distinguishing between objects with similar or dissimilar spectra and shadowed features that are difficult to differentiate in high-resolution remote sensing images in the vertical space. Therefore, this application combines high-resolution remote sensing images and airborne LiDAR point cloud data to extract buildings in built-up areas using multi-feature space, and then adaptively thins and vectorizes the results.

[0097] (1) Core method: A vectorization method for automatic house recognition that combines LiDAR point cloud and optical image, including: first, multi-scale elevation collaborative segmentation of images that combine LiDAR point cloud data; second, optimized sorting of houses and plants based on gradient space analysis; and third, house recognition and fine vectorization of contours.

[0098] Based on multi-scale elevation collaborative segmentation of remote sensing images, the nDSM (nearest-distributed surface image) of point cloud data and the edge contour information of the nDSM are used as input bands to participate in the multi-scale elevation collaborative segmentation of optical images, improving the segmentation quality and preparing for further classification. In the process of building contour recognition based on point cloud elevation images, a sorting optimization method for plants and buildings based on land feature gradient and elevation spatial relationship is proposed, including plant identification and filtering based on gradient spatial features and building edge restoration. For the building edge information recognition results, a corner recognition-driven fine vectorization method for building contours is proposed, mainly including: corner recognition-driven initial optimization of building contours and cyclic recursive compression of building contours fine vectorization.

[0099] (2) Process steps: Automatic identification and adaptive vectorization of buildings in built-up areas, including:

[0100] Step 1: Preprocessing and feature recognition of LiDAR data;

[0101] Step 2: Feature analysis and setting of high-resolution remote sensing images;

[0102] Step 3: Multi-scale collaborative elevation segmentation based on multi-source data;

[0103] Step 4: Initial House Identification (Object-Oriented);

[0104] Step 5: Shadow recognition in remote sensing images;

[0105] Step Six: Detailed Identification of the House;

[0106] Step 7: Thinning and vectorizing the building outline;

[0107] An automatic building recognition vectorization method based on LiDAR point cloud and optical image is proposed. Following a multi-level recognition process of segmentation followed by classification, the method extracts two-dimensional roof patches of buildings with refined contours, thereby achieving adaptive building vectorization. This method has good adaptability to buildings in built-up areas with different shapes and occlusion.

[0108] I. A Vectorization Method for Automatic House Recognition Based on Combined LiDAR Point Cloud and Optical Imagery

[0109] The process of automatically identifying houses based on the joint use of LiDAR point cloud data and high-resolution optical images is as follows: Figure 1 As shown.

[0110] (I) Multi-scale Elevation Collaborative Segmentation of Images Using LiDAR Point Cloud Data

[0111] High-resolution remote sensing images possess rich semantic information, but their ability to distinguish feature differences in complex urban environments is insufficient. LiDAR point cloud data contains elevation information of ground features and maintains the regular geometric shape information of buildings, making it a valuable auxiliary data source for filtering and reducing the complexity of ground features. This application employs a multi-source information source fusion approach, integrating LiDAR point cloud elevation information, shape information, and high-resolution remote sensing images for multi-scale elevation collaborative segmentation of ground features. By combining the information advantages of multiple data sources, the accuracy of building extraction is improved.

[0112] 1. Multi-scale elevation co-segmentation of images using joint point cloud nDSM and edge data.

[0113] The initial LiDAR point cloud consists of discrete data points in 3D space. Point location information includes 3D coordinates (x, y, z), laser corner echo information, and corner reflection intensity information. While the spectral texture features in traditional 2D optical images can effectively identify various ground features, they do not consider the distribution of ground features in vertical space. Different urban ground features exhibit predictable distribution patterns in vertical space. Therefore, the elevation information carried in the point cloud data can suppress excessive complexity of ground features and has a sorting and optimization effect on phenomena such as different spectra for the same object and different objects with the same spectrum in high-resolution remote sensing images.

[0114] The acquired LiDAR point cloud elevation information is used as a collaborative feature for segmentation. An nDSM model is used to obtain the absolute height information of ground features. The absolute height optical image of the point cloud is used as an input band for multi-scale ground feature segmentation. Sample images are shown below. Figure 2 .

[0115] like Figure 2 As shown, the nDSM image represents the absolute height information of ground features and no longer contains ground point information. The grayscale level of its pixels is proportional to the elevation of the corresponding ground feature point; the higher the brightness value, the higher the height. Since houses are generally of similar height, in optical image segmentation and classification, irrelevant ground features such as cars and low-rise infrastructure are filtered out by setting elevation thresholds. Furthermore, the main ground feature types in the filtered elevation image include houses and vegetation, and the outlines of houses are clearly visible; edge information is obtained through edge recognition.

[0116] Edge recognition is used to obtain the contour information of major land features such as houses and vegetation in the nDSM elevation image. In subsequent segmentation, this contour information is used as input bands. Multi-scale elevation collaborative segmentation references the contour information to improve segmentation accuracy. Edge effects are shown below. Figure 3 As shown.

[0117] 2. Multi-scale elevation co-segmentation effect of images using LiDAR point cloud data

[0118] Multi-scale elevation collaborative segmentation of high-resolution remote sensing images is a two-dimensional optical image analysis method, which inevitably leads to the phenomenon of objects with the same spectrum but different features in complex terrain environments. By combining 3D elevation information from LiDAR point cloud data, we can identify differences in elevation features in the vertical space when dealing with objects with the same spectrum, thus resolving the erroneous segmentation of such objects.

[0119] Figure 4 and Figure 5 These images show the effects of LiDAR data participating in high-resolution remote sensing image segmentation at the same segmentation scale, highlighting the effects of heterogeneous data sharing and the improvement in segmentation results brought about by introducing multiple features. Figure 4 In this context, LiDAR data is not used for segmentation using only a single high-resolution remote sensing image. Because road surfaces and roads have similar spectral and textural features, they are easily misclassified as the same type of object during multi-scale elevation co-segmentation, leading to missegmentation (i.e., homospectral heterogeneity). In subsequent building object extraction, inaccurate segmentation results in unclear sample object features, thus reducing the accuracy of building extraction. Figure 5 In this study, the height image nDSM of the acquired LiDAR point cloud data and the contour feature images of houses and plants obtained by the edge recognition algorithm are used as input feature bands to participate in multi-scale elevation collaborative segmentation. Due to the difference in elevation features between houses and roads and the constraints of the contour information obtained by edge recognition, the two types of objects are successfully segmented.

[0120] Similarly, Figure 6 and Figure 7 As shown, when segmenting vegetation and shadows at the same segmentation scale, introducing LiDAR point cloud elevation information can effectively segment the two types of ground features.

[0121] The segmentation scale of an optical image determines the degree of fragmentation of the segmented objects, thus having a decisive impact on the segmentation quality and subsequent classification. Using the same segmentation parameters (shape factor, spectral factor), the number of segmented objects obtained from the same image with and without LiDAR point cloud height images and using different segmentation scales is shown in the following figures. Figure 7 As shown:

[0122] At the same segmentation scale, incorporating elevation features from LiDAR point clouds significantly reduces the number of objects. In a remote sensing image, the degree of object fragmentation affects not only segmentation accuracy but also processing complexity, and subsequently, classification. Using nDSM (nearest-dimension model) elevation images and contour images as input bands for segmentation reduces heterogeneity and increases homogeneity among objects at the same elevation, while increasing heterogeneity across different elevations, resulting in more complete segmented objects. Furthermore, using contour images of buildings and vegetation in segmentation allows the segmentation path to follow the contours of the features, promoting the merging of fragmented objects, reducing the number of segmented objects, and alleviating the complexity of feature analysis.

[0123] (II) Optimal Sorting of Houses and Plants Based on Gradient Space Analysis

[0124] In the process of building extraction and recognition, building recognition based solely on optical images mainly includes object-oriented multi-level recognition and machine learning-based classification recognition. Machine learning-based building recognition is prone to the salt-and-pepper phenomenon, resulting in classification noise. Object-oriented building recognition is affected by shadows and complex terrain features, leading to missing or prominent building outlines and low accuracy of outline information. LiDAR point clouds provide complete building outlines for roof patches, providing overlay analysis and filtering for misclassified terrain features in the final building extraction. The influence of shadows or complex terrain features in optical image building recognition can be supplemented and optimized by combining point cloud data with building outlines. After obtaining the absolute height optical image nDSM of the point cloud, the terrain feature types mainly include buildings, vegetation, and other low-lying features. Therefore, the main problem in building outline extraction based on LiDAR point cloud data is the separation of buildings and vegetation. In optimizing the separation of plants and buildings, this application uses the elevation change rate between the building area and the point cloud area in the elevation image to detect and filter vegetation.

[0125] 1. Plant identification and filtering based on gradient spatial features

[0126] The height image of a 3D laser point cloud region is a grayscale matrix composed of elevation values ​​of different ground features. While absolute height values ​​supplement the classification of ground features in vertical space, they are difficult to effectively distinguish between features at nearly identical heights. Therefore, the gradient of elevation values ​​between different ground features serves as an indicator for distinguishing features at similar heights. This gradient is more pronounced at the edges of ground features or on surfaces with varying roughness. The gradient represents the inclination and roughness of the spatial surface. Gradient simulation is performed using the second derivative of the 3D surface, and the calculation formula is as follows:

[0127]

[0128] Where (x,y) are points on the surface. These are partial derivatives;

[0129] On relatively smooth building surfaces, ground surfaces, or other terrain features, local elevation abrupt changes are not prominent enough, meaning the gradient spatial features are not significant, appearing as dark tones in gradient images. However, on uneven surfaces with vegetation, not only are elevation abrupt changes large at the edges, but there are also elevation abrupt changes in different directions within the regions, resulting in large gradient values, which appear as highlights in gradient images. Figure 8 As shown.

[0130] After obtaining the gradient space feature results, the house and plant areas have certain differences in gradient space features. By setting the gradient threshold, binary images of the plant area and other areas are obtained. After filtering out the low gradient area, the high gradient area is set to white (255) and the low gradient area is set to black (0), which makes it easier to observe the results visually. Finally, the output results are inverted.

[0131] 2. Restoration of house edges

[0132] The process of filtering plant information based on gradient spatial features can lead to the loss of house edges, resulting in information loss. Therefore, the edge information of houses must be recovered. The first echo signal of the laser signal mainly includes house, plant, and other relatively complete ground surface information. House edges and plant areas contain regions with large gradient values. In the last echo, however, it mainly contains house, ground points, or other relatively low-lying vegetation information, with large gradient values ​​primarily located at house boundaries. Therefore, gradient calculation is performed on the last echo image to obtain the house boundary region. Other ground features included are mainly low-lying vegetation. By setting an elevation threshold for filtering, the missing house edges can be obtained. Figure 9 As shown.

[0133] After obtaining the surface region of the house based on the LiDAR point cloud nDSM optical image, the initial complete outline information of the house is obtained. Due to the limitation of the planar resolution of the point cloud data, a square buffer with a point spacing of 1 is made in the house area based on prior knowledge. However, because the plant range plane appears in the local area inside the plant area, the gradient value is small in this area. Plant filtering based on gradient spatial features will include some residual plant information. In addition, a small amount of plant information will also be brought in during the house edge restoration. In the object-oriented image house extraction results, the plant recognition effect of house recognition based on image spectral features is excellent, but the edge information of the house has the problem of shadow occlusion and missing information. Therefore, the initial extraction results of the house from the point cloud data and the optical image have good complementarity.

[0134] (III) Building Recognition and Fine-Diffraction of Contours

[0135] LiDAR point cloud data possesses rich elevation information and can preserve building outline information, giving it a significant advantage in automatic building identification. This application proposes a corner-recognition-driven fine vectorization method for building outlines, which mainly consists of two parts: corner-recognition-driven initial optimization of building outlines and iterative recursive compression of building outlines for fine vectorization.

[0136] 1. Initial optimization of house outline driven by corner recognition

[0137] The resulting images from building recognition exhibit jagged edges, failing to meet relevant mapping standards when imported into a GIS vector database. Therefore, fine-grained vectorization of the contours is necessary. Corner points are the extreme points of curvature on a curve, displaying the curve's contour and key point information. Corner point recognition is highly effective for thinning and vectorizing polygon edges; the identified corner points approximate the polygon's contour, effectively reducing data volume and significantly improving the efficiency and quality of subsequent fine-grained vectorization processing.

[0138] This application utilizes the curvature and gradient changes of pixels to find extreme points of curvature change as candidate corner points for corner recognition, achieving excellent results and stability. The main features include:

[0139] Step 1: Calculate the correlation matrix N of the pixels and store the directional gradient values;

[0140]

[0141] J(x,y) represents grayscale pixels, k(x,y) represents the sliding window, and J x J y The derivative of the image in the x and y directions;

[0142] Step 2: Calculate the corner response function value T of the pixel:

[0143] T = (DE - SA) 2 w(D+E) 2 Formula 6

[0144] w is an empirical constant;

[0145] Step 3: Find the extreme points in the response function, compare them with the given critical value, and set them as corner points.

[0146] 2. Refined vectorization of house outlines obtained through iterative recursive compression

[0147] Corner recognition uses a set of corner points connected end-to-end in a closed polygon based on least-squares matching and Euclidean distance. This simplifies the data volume while preserving the building's outline information. Recursively dividing the entire curve into two segments can lead to stack overflow issues when dealing with large datasets. Therefore, corner recognition effectively controls the data volume and improves recursion efficiency.

[0148] Refined vectorization of house outlines through cyclic recursive compression includes:

[0149] Step 1: Locate the feature point of the linear feature, i.e., the point Q with the maximum vector radius when connecting the corner point of corner identification to the start and end points Q1 and Q2 of the local curve. If the radius is greater than a given critical value Q... max If it is not present, keep it; otherwise, remove it.

[0150] Step 2: Find a new feature point Q, divide the local curve into two segments, and perform the calculation in step 1 on each segment;

[0151] Step 3: Iterate and calculate until no new feature points Q appear. Finally, a series of feature points and the start and end points of local curves are obtained. Connecting the start and end points with straight lines will yield the thinned vectorized linear features.

[0152] II. Steps for Automatic Building Identification and Adaptive Vectorization in Built-up Areas

[0153] (I) Preprocessing and Feature Recognition of LiDAR Data

[0154] The first step is point cloud filtering. By utilizing the elevation changes caused by terrain undulations, non-ground points are filtered out to generate a digital elevation model (DEM) of the land surface. The point cloud data is filtered using a progressive triangulation method and high-resolution image sampling is performed to provide high accuracy for generating an absolute elevation feature (nDSM). After upsampling, the 3D point cloud image and the aerial image are adjusted using the forward intersection method based on collinearity equations to solve for exterior orientation elements. The height image and the aerial image data are then registered.

[0155] (1) Elevation feature identification

[0156] The initial airborne LiDAR point cloud data contains spatially resolved elevation features of ground features. After obtaining the ground point elevation model, the absolute elevation model nDSM of the ground features is obtained based on the DSM optical image obtained from the last echo interpolation in the dataset. The computer is:

[0157] nDSM(x,y)=DSM(x,y)-DTM(x,y) Equation 7

[0158] DTM(x,y) is a digital terrain model, and DSM(x,y) is a digital surface model.

[0159] (2) Gradient space feature recognition

[0160] While the Rn plant index can filter out some plant information, in complex urban environments, a significant portion of plants are obstructed by buildings, and even in shaded areas, there is confusion regarding plant spectral information. It is difficult to filter out all plant information in obstructed or shaded areas using only the Rn plant index.

[0161] On uneven plant surfaces or along building edges, not only are there large elevation abrupt changes at the edges, but there are also elevation abrupt changes in different directions within the region, resulting in large gradient spatial values. Gradient optical images are obtained based on gradient magnitude calculation algorithms, such as... Figure 10 As can be seen, the plant area is clearly highlighted.

[0162] (II) Feature Analysis and Setting of High-Resolution Remote Sensing Images

[0163] High-resolution remote sensing images, specifically color infrared images, contain three bands: near-infrared (NIR), red (R), and green (G). Plant information in color infrared images is represented in red. The definition of the plant normalization index is as follows:

[0164]

[0165] Where E NIR For the near-infrared band, E R It is in the infrared band;

[0166] (III) Multi-scale collaborative elevation segmentation based on multi-source data

[0167] Multi-scale elevation collaborative segmentation of ground features is performed by integrating elevation and shape information from LiDAR point cloud data and high-resolution remote sensing images. This combined advantage of multi-source data improves the accuracy of building identification. Segmentation factor parameters are as follows: Figure 11 The effect of multi-scale elevation collaborative segmentation is as follows: Figure 12 As shown.

[0168] After introducing LiDAR point cloud elevation information, to address the issues of shadow occlusion of plants and houses and spectral confusion of ground features at different heights, the weight of elevation information was increased and set to 5. Accurate house segmentation was given top priority, and the shape factor of the segmentation parameter was set to 0.5, so that houses with regular geometric information could achieve better segmentation results.

[0169] (iv) Object-oriented initial building identification

[0170] To reduce the complexity of building identification, low-lying features are masked by setting an elevation threshold. The elevation threshold is set to a range of H = [1.5, 1.7] meters, and features with an elevation value greater than [1.5, 1.7] meters are defined as the initial building selection area.

[0171] After masking with an elevation threshold, some vegetation, ground features, and low-lying infrastructure such as cars are filtered out. At this point, the main feature types include houses, vegetation, and shadows. The plant normalization index Rn is obtained, and a threshold value Rn is set. 临界值=0.208, for areas outside the masked region, the initial identification of houses is performed according to the following rules:

[0172] Rn≥Rn 临界值 Identified as plant-type 9

[0173] Rn<Rn 临界值 Identified as house type 10

[0174] Based on the above rules, the initial identification of houses is obtained, and the results are as follows: Figure 13 As shown:

[0175] The initial identification results of buildings included both vegetation and shade. Directly using elevation thresholds and vegetation index Rn thresholds for building identification has two problems: First, trees in shaded areas have lower Rn indices, which are close to the calculated vegetation index Rn value for buildings, so trees in shaded areas are misclassified as buildings; second, some small features are identified as buildings because their elevations are close to those of buildings and their Rn values ​​are distinct from those of vegetation.

[0176] (v) Shadow recognition in remote sensing images

[0177] In the initial house identification results, the presence of shadow areas causes trees and plants within these areas to have low vegetation indices and be misclassified as houses. Furthermore, plant shadows and the shadows of rooftop attachments also obscure the houses. This application uses a brightness histogram critical value segmentation method to identify shadow areas. By segmenting shadow areas based on their generally low brightness values, it eliminates the need to filter out low-lying features such as ground points in the initial house identification process, including low-elevation shadow areas.

[0178] (vi) Detailed identification of houses

[0179] The results of building shadow recognition show that shadows can obscure buildings, and the shadows of building appurtenances can also cause gaps in building recognition. Therefore, the height image nDSM generated from LiDAR point cloud data is used to further refine the building recognition results. LiDAR laser signals have penetrating power, and the shadow occlusion phenomenon shown in remote sensing images is not affected in point cloud echo information.

[0180] Although airborne LiDAR point clouds can identify the outline information of houses, they are limited by the horizontal resolution of the LiDAR sensor. Even if the identification result is completely correct, the quality and accuracy of house identification cannot be guaranteed. However, the house outline can serve as a reference constraint framework for fine house identification. Based on the penetrability of the laser signal, the occlusion problem is solved. First, the house outline of the depth optical image nDSM is identified, and house patch optimization and sorting are performed on the nDSM single-band image.

[0181] (1) Set an elevation threshold to filter out low-lying features. In this application, the threshold value H = 1.7m is set to remove the interference of low-lying features through elevation filtering.

[0182] (2) After elevation filtering, the main land cover types in the optical image include plants and houses. Therefore, the house outline recognition is performed by the house and plant optimization sorting based on gradient spatial analysis proposed in this application. After obtaining the gradient optical image, a gradient threshold is set to filter out plant information, i.e., house patch recognition, and the house edges of the last echo are added to the gradient optical image. Finally, an area filter with a threshold of 200 is performed.

[0183] After obtaining the outline of the house from the LiDAR point cloud nDSM image, the outline information of the house is basically complete. However, within the plant area, there are localized areas where plant range planes appear. These areas have low gradient values. In the gradient spatial feature-based plant filtering, some residual plant information is included, and the presence of roof attachments causes voids, such as... Figure 14 While plant identification based on image spectral features is highly effective, edge information of houses is often obscured or missing. Therefore, point cloud data and the initial identification results of houses in images are highly complementary. By overlaying and analyzing the initial identification results obtained from optical images of houses, and then performing intersection processing on the two results, the combined data advantages can be utilized to filter out messy misclassified features.

[0184] The two recognition results were overlaid and parsed using the MATLAB platform. The regular house edge information and high-quality plant recognition information in the optical image were complemented by the shadow occlusion suppression of houses or plants in the point cloud data to obtain high-quality recognition results. Finally, area filtering and morphological opening operations with a given threshold were performed to remove the holes in the patches and houses in the results.

[0185] (vii) Thinning and vectorization of building outline

[0186] In the roof patch recognition results obtained based on overlay analysis, the occlusion of the building by plant shadows or building shadows is corrected to a certain extent, and the advantages of the two recognition results are complementary. However, due to the pixel characteristics and resolution issues of raster data, some edges are still not refined enough. The resulting building recognition image is raster data with jagged edges, which does not meet the relevant mapping standards when entering GIS vector graphics into the database. Therefore, fine vectorization of the outline is performed. The fine vectorization of the building outline driven by corner recognition proposed in this application is used to perform fine vectorization processing of the recognition results.

[0187] Corner recognition-driven initial optimization of house outlines preserves the house outline while reducing data volume, significantly improving the efficiency of subsequent thinning processing.

[0188] Corner point recognition connects the corner points end-to-end to form a closed polygon, simplifying the data volume while preserving the building's outline information. Recursive compression of the building outline into fine vectorization involves bisegmenting the entire curve, which can easily lead to stack overflow issues due to excessive depth when dealing with large datasets. Therefore, corner point recognition effectively controls the data volume and improves recursion efficiency. The result of thinning and vectorizing the building region is shown below. Figure 15 .

[0189] III. Quantitative Evaluation and Experimental Comparison

[0190] (I) Overlay parsing of recognition results

[0191] This application establishes three study areas with varying terrain, landforms, and building shapes, each with a corresponding ground truth reference area. Vectorized overlay analysis of the corresponding building areas effectively utilizes the aforementioned evaluation indicators for accuracy assessment. The rasterized results of building identification are processed into vector patches in .shp format, while the building portions in the ground truth image are separately ground truth-valued and vectorized. In ArcGIS, the resulting image is overlaid with the ground truth image, with a strong color contrast set for analysis. The transparency of both colors is set to 60% to highlight the contrast between the identified and ground truth areas, or the intersection of the two vector maps can be directly determined, thus identifying misdetected and missed areas.

[0192] This application presents the latest results of four recognition methods for comparison. These include two sets of test results from Working Group III / 4 in 2021: a random forest classification method based on object-oriented image combined with LiDAR point cloud data, referred to as methods WCX1 and UDE2; recognition results from Mortid based on point cloud data image processing in 2020, referred to as method HUI3; and optical image house recognition results from Niem based on SVM in 2018, referred to as method GDFD4. The comparison of recognition results shows that the method in this application outperforms other methods on average across various metrics, achieving a maximum accuracy of 98.69%, a maximum completeness rate of 6.47%, and a recognition quality of 2.69% for region A. In summary, the method in this application maintains both high accuracy and high completeness in house recognition results, mainly due to the complementary data sources and post-processing algorithms that enable the recovery of house information, achieving high-quality adaptive vectorization of houses from optical images.

[0193] (II) Results Analysis

[0194] like Figure 16The vector layer of the identified houses is overlaid on the initial optical image, creating a vector overlay effect with the ground truth house region. Recognition results from multiple test areas demonstrate that the proposed method effectively identifies houses of various shapes, textures, and heights, maintaining good robustness in terms of completeness and accuracy. Figure 16 As can be seen from (a) and (c), the building also has a strong ability to identify buildings surrounded by dense vegetation and whose surfaces are obscured by the shadows of other buildings, plants, or its own roof; from the identification results Figure 16 As can be seen from (c), the method of this application can effectively identify pyramidal houses and courtyard houses, and also has a strong adaptability to houses with small attachments on the roof surface; from Figure 16 As shown in (b), it can correctly identify house clusters composed of A-shaped, I-shaped, or multiple types of houses. The algorithm in this application considers the details of multiple land features and spectral confusion in complex urban environments, and performs multi-level house recognition based on multiple features such as elevation, echo, spectrum, and gradient. The experimental results are consistent with the theoretical method, and excellent recognition and adaptive vectorization of houses have been achieved.

Claims

1. A method for adaptive vectorization of buildings using LiDAR point clouds and optical images, characterized in that, Buildings in built-up areas are extracted using multi-feature spatial analysis of high-resolution remote sensing images and airborne LiDAR point cloud data, and the results are adaptively thinned and vectorized. (i) Based on multi-scale elevation collaborative segmentation of remote sensing images, the nDSM elevation optical image of point cloud data and the edge contour information of the elevation optical image are used as input bands to participate in the multi-scale elevation collaborative segmentation of optical images, thereby improving the segmentation quality and preparing for further classification. (II) Process Steps: Automatic identification and adaptive vectorization of buildings in built-up areas, including: Step 1: Preprocessing and feature recognition of LiDAR data: Using the elevation changes of the terrain, non-ground points are filtered out to generate a digital elevation model (DEM) of the land surface. The point cloud data is filtered by the progressive triangulation method and image-based high-resolution sampling is performed. After upsampling, the 3D point cloud image and the aerial image are adjusted by the collinear equation forward intersection method to solve the exterior orientation elements. The height image and the aerial image data are then registered. Step 2: Feature analysis and setting of high-resolution remote sensing images; Step 3: Multi-scale elevation collaborative segmentation using multi-source data: Multi-scale elevation collaborative segmentation of ground features is performed by integrating elevation information, shape information from LiDAR point cloud data and high remote sensing imagery. After introducing LiDAR point cloud elevation information, the weight of elevation information is increased and set to 5; the shape factor of the segmentation parameter is set to 0.

5. Step 4: Object-oriented initial building identification: For object-oriented initial building identification, set the elevation threshold value range H = [1.5, 1.7] meters, and define features with elevation values ​​greater than [1.5, 1.7] meters as the initial building selection area; Step 5: Shadow Recognition in Remote Sensing Images: After masking with set elevation thresholds, the main ground feature types include houses, vegetation, and shadows. The vegetation normalization index Rn is obtained. E NIR For the near-infrared band, E R For the infrared band, a threshold value Rn is set. 临界值 =0.208; Step Six: Detailed Identification of the House; Step 7: House Outline Thinning and Vectorization: First, shadow areas are identified using the brightness histogram critical value segmentation method. Shadow areas are segmented based on their low brightness values. Then, the height image nDSM generated from LiDAR point cloud data is used to further refine the house identification results. The house outline serves as a reference constraint framework for fine-grained house identification. The house outlines in the depth optical image nDSM are identified. House patch optimization and sorting are performed on the nDSM single-band image. A height critical value is set to zero-filter out low-lying features. The critical value H = 1.7m, and the height filtering removes interference from low-lying features. House and vegetation optimization and sorting based on gradient space analysis are used to identify the house outline. After acquiring the gradient optical image, a gradient critical value is set to filter out vegetation information, i.e., house patch identification. The house edges from the last echo are then added to the gradient optical image. An area filter with a threshold value of 200 is applied. The two recognition results are then overlaid and analyzed using MATLAB. Regular house edge information and high-quality plant recognition information from the optical image complement the suppression of shadow occlusion of houses or plants in the point cloud data. Area filtering and morphological opening operations are performed at a given threshold value to remove holes in the resulting image features and houses. Finally, a corner-recognition-driven fine-vectorization method is used to refine the contours of the recognition results. Initial optimization of the house contours is performed, utilizing the curvature and gradient changes of pixels to find extreme points of curvature change as candidate corners. Corner recognition includes: Step 1: Calculating the correlation matrix N of the pixels and storing the directional gradient values; Step 2: Calculating the corner response function value T of the pixels: T = (DE - SA). 2 w(D+E) 2 w is an empirical constant, N is the correlation matrix of the pixel, and D, E, S, and A are the stored directional gradient values, respectively. The third step is to find the extreme points in the response function, compare them with a given critical value, and set them as corner points. Then, a recursive compression of the house outline is performed for fine vectorization. The corner point set for corner recognition is connected end-to-end to form a closed polygon based on the Euclidean distance of least-squares matching. The entire curve is recursively bisegmented, including: Step 1: Finding the feature points of linear elements, i.e., the point Q with the maximum vector radius on the line connecting the corner point of corner recognition to the start and end points Q1 and Q2 of the local curve. If the radius is greater than the given critical value Q... max If the feature point Q is found, it is retained; otherwise, it is removed. Step 2: Find a new feature point Q and divide the local curve into two segments. Perform the calculation in step 1 on each segment. Step 3: Iterate and calculate until no new feature point Q appears. Finally, a series of feature points and the start and end points of the local curve are obtained. Connect the start and end points with straight lines to obtain the thinned vectorized linear features.

2. The adaptive vectorization method for houses based on combined LiDAR point clouds and optical images according to claim 1, characterized in that, Multi-scale elevation collaborative segmentation of joint point cloud nDSM and edge images: The initial LiDAR point cloud consists of discrete data points in 3D space. The point location information includes 3D coordinates (x, y, z), laser corner echo information, and corner reflection intensity information. The acquired LiDAR point cloud elevation information is used as a collaborative feature for segmentation. An absolute height information model nDSM for ground objects is obtained. An optical image of the absolute height of the point cloud is used as an input band for multi-scale ground object segmentation. In the optical image segmentation and classification, incoherent ground objects are filtered out by setting elevation thresholds. Edge recognition is used to obtain the outline information of houses, plants and other main land features in the nDSM elevation image. In the subsequent segmentation, the outline information of the land features is used as the input band for segmentation. In multi-scale elevation collaborative segmentation, the outline information of the land features is referenced for segmentation to improve the segmentation accuracy.

3. The adaptive vectorization method for houses based on combined LiDAR point clouds and optical images according to claim 1, characterized in that, Plant identification and filtering based on gradient spatial features: The elevation gradient between different ground features serves as an indicator for distinguishing between features at similar heights. The rate of elevation change is more pronounced at the edges of ground features or in surface areas with varying surface roughness. The gradient represents the inclination and roughness of the spatial surface. Gradient simulation is performed using the second derivative of the 3D surface, and the calculation formula is as follows: Where (x,y) are points on the surface. These are partial derivatives; On relatively smooth building surfaces, ground surfaces, or other land features, local elevation changes are not prominent enough, i.e., the gradient spatial features are not large, and they appear as dark tones in gradient images. On uneven plant surfaces, not only are the elevation changes large at the edges, but there are also elevation changes in different directions within the regions, resulting in large gradient values, which appear as highlights in gradient images. After obtaining the gradient space feature results, the house and plant areas have certain differences in gradient space features. By setting the gradient threshold, binary images of the plant area and other areas are obtained. After filtering out the low gradient area, the high gradient area is set to white (255) and the low gradient area is set to black (0), which makes it easier to observe the results visually. Finally, the output results are inverted.

4. The adaptive vectorization method for houses based on combined LiDAR point clouds and optical images according to claim 1, characterized in that, Building edge restoration: The first echo signal of the laser signal mainly includes the surface information of buildings, plants and other relatively complete ground features. The building edges and plant areas contain areas with large gradient values. In the last echo, the main features are the building, ground points or other relatively low plants. The areas with large gradient values ​​are mainly at the building boundaries. Gradient calculation is performed on the last echo image to obtain the building boundary area. The other ground features included are mainly low plants. By setting an elevation threshold for filtering, the missing building edges are obtained. After obtaining the surface regions of houses based on LiDAR point cloud nDSM optical images, the initial complete outline information of the houses is obtained. Based on prior knowledge, a square buffer with a point spacing of 1 is created in the house region. Plant filtering based on gradient spatial features will include some residual plant information. In addition, a small amount of plant information will also be brought in during house edge restoration. In the object-oriented image house extraction results, the plant recognition effect is excellent in house recognition based on image spectral features, but the edge information of the houses has the problem of shadow occlusion and missing information. Therefore, the point cloud data and the initial extraction results of houses from optical images have good complementarity.

5. The adaptive vectorization method for houses based on combined LiDAR point clouds and optical images according to claim 1, characterized in that, Corner recognition-driven initial optimization of building outlines: This method utilizes the curvature and gradient changes of pixels to find extreme points of curvature change as candidate corners for corner recognition. It exhibits excellent performance and stability, and mainly includes: Step 1: Calculate the correlation matrix N of the pixels and store the directional gradient values; J(x,y) represents grayscale pixels, k(x,y) represents the sliding window, and J x J y The derivative of the image in the x and y directions; Step 2: Calculate the corner response function value T of the pixel: T = (DE - SA) 2 w(D + E) 2 Formula 6 w is an empirical constant; Step 3: Find the extreme points in the response function, compare them with the given critical value, and set them as corner points.

6. The adaptive vectorization method for houses based on combined LiDAR point clouds and optical images according to claim 1, characterized in that, Step 1, LiDAR data preprocessing and feature recognition: First, point cloud filtering is performed. Taking advantage of the elevation changes in terrain undulations, non-ground points are filtered out to generate a digital elevation model (DEM). The point cloud data is filtered using a progressive triangulation method and high-resolution image sampling is performed to provide high accuracy for generating an absolute elevation feature (nDSM). After upsampling, the 3D point cloud image and the aerial image are adjusted using the forward intersection method based on collinearity equations. The height image and the aerial image data are then registered. (1) Elevation Feature Recognition: The initial airborne LiDAR point cloud data contains spatially resolved elevation features of ground objects. After obtaining the ground point elevation model, the absolute elevation model nDSM of the ground objects is obtained based on the DSM optical image obtained by interpolation from the last echo in the dataset. The computer is: nDSM(x,y)=DSM(x,y)-DTM(x,y) Equation 7 (2) Gradient spatial feature recognition: On the uneven surface of plants or the edge of houses, not only is there a large elevation change in the edge area, but there are also elevation changes in different directions inside the area, resulting in a large gradient spatial value. The gradient optical image is obtained based on the gradient amplitude solution algorithm, and the plant area appears bright.

7. The adaptive vectorization method for houses based on combined LiDAR point clouds and optical images according to claim 1, characterized in that, Step 2, Feature Analysis and Setting of High-Resolution Remote Sensing Images: High-resolution remote sensing images, specifically color infrared images, contain three bands: near-infrared (NIR), red (R), and green (G). Plant information in color infrared images is represented by red. The definition of the plant normalization index is as follows: Where E NIR For the near-infrared band, E R It is in the infrared band; Step 3, Multi-scale Elevation Collaborative Segmentation Based on Multi-source Data: Multi-scale elevation collaborative segmentation of ground features is performed by integrating elevation information from LiDAR point cloud data and shape information from high-resolution remote sensing images. The information advantages of combining multiple data sources are combined to improve the accuracy of building identification. After introducing LiDAR point cloud elevation information, to address the issues of shadow occlusion of plants and houses and spectral confusion of ground features at different heights, the weight of elevation information was increased and set to 5; accurate house segmentation is of paramount importance, so the shape factor of the segmentation parameter was set to 0.5, so that houses with regular geometric information can achieve better segmentation results. Step 4, object-oriented initial building identification: Low-lying features are masked by setting an elevation threshold to reduce the complexity of building identification. The elevation threshold is set to a range of H = [1.5, 1.7] meters. Features with an elevation value greater than [1.5, 1.7] meters are defined as the initial building selection area. After masking with an elevation threshold, some vegetation, ground features, and low-lying infrastructure such as cars are filtered out. At this point, the main feature types include houses, vegetation, and shadows. The plant normalization index Rn is obtained, and a threshold value Rn is set. 临界值 =0.208, for areas outside the masked region, the initial identification of houses is performed according to the following rules: Rn≥Rn 临界值 Identified as plant-type 9 Rn<Rn 临界值 Identified as house type 10 Based on the above rules, the initial identification of the house is obtained. The initial identification result of the house includes plants and shadows.

8. The adaptive vectorization method for houses based on combined LiDAR point clouds and optical images according to claim 1, characterized in that, Step 4, fine-grained building identification: The height image nDSM generated using LiDAR point cloud data further refines the building identification results. LiDAR laser signals have penetrating power, and the shadow occlusion phenomenon shown in remote sensing images is not affected in point cloud echo information. Although airborne LiDAR point clouds can identify the outline information of houses, the house outline serves as a reference constraint framework for fine house identification. The occlusion problem is solved based on the penetrability of laser signals. First, the house outline of the depth optical image nDSM is identified, and house patch optimization and sorting are performed on the nDSM single-band image. (1) Set an elevation threshold to filter out low-lying features. In this application, the threshold value H = 1.7m is set to remove the interference of low-lying features through elevation filtering. (2) After elevation filtering, the main land features in the optical image include plants and houses. Therefore, the house outline is identified by the house and plant optimization sorting based on gradient spatial analysis proposed in this application. After obtaining the gradient optical image, the gradient threshold is set, the plant information is filtered out, i.e., the house patch is identified, and the edge of the last echo house is added to the gradient optical image. Finally, an area filter with a threshold of 200 is performed. After obtaining the outline of the house from the LiDAR point cloud nDSM image, the outline information of the house is basically complete. The point cloud data and the initial recognition result of the house in the image have good complementarity. By overlaying and parsing the initial recognition result obtained from the optical image of the house, the intersection of the two results can combine the advantages of the data and filter out messy misclassified ground features. The two recognition results were overlaid and parsed using the MATLAB platform. The regular house edge information and high-quality plant recognition information in the optical image were complemented by the shadow occlusion suppression of houses or plants in the point cloud data to obtain high-quality recognition results. Finally, area filtering and morphological opening operations with a given threshold were performed to remove the holes in the patches and houses in the results.

9. The adaptive vectorization method for houses based on combined LiDAR point clouds and optical images according to claim 1, characterized in that, Step 7, House outline thinning and vectorization: Based on the house roof patch recognition results obtained from overlay analysis, the occlusion of the house by plant shadows or house shadows is corrected to a certain extent, and the advantages of the two recognition results are complemented. The outline of the recognition results is processed by the fine vectorization method of house outline driven by corner point recognition. Corner recognition-driven initial optimization of house outline preserves the house outline while reducing data volume, thus improving the efficiency of subsequent thinning processing. Corner point recognition connects the corner points end to end to form a closed polygon, which simplifies the amount of data while preserving the outline information of the house. The refined vectorization of the house outline through recursive compression divides the entire curve into two segments. However, the large amount of data can easily lead to stack overflow due to excessive depth. The results of corner point recognition control the amount of data and improve recursion efficiency.

Images