Building vacancy automatic recognition method based on three-dimensional point cloud and image features

Abstract

The application provides a kind of building vacancy automatic identification method based on three-dimensional point cloud and image feature, it is related to image recognition technical field. Including multi-source data acquisition and three-dimensional reconstruction, ground object classification and building contour extraction, structural state feature (roof damage index, wall body inclination index, facade integrity loss index) is extracted based on building point cloud subset, structural vacancy state determination is carried out, and functional vacancy state determination is carried out by extracting yard use trace feature (weed invasion index, weed height feature, weed distribution dispersion). The application realizes the intelligent identification of structural vacancy and functional vacancy by establishing building structure integrity evaluation model based on point cloud structure feature and maintenance state quantitative index system based on three-dimensional vegetation structure.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to an automatic building vacancy identification method based on three-dimensional point cloud and image features. Background Technology

[0002] With increasing population mobility, continuous adjustments in residential structures, and changes in the supply and demand pattern of the real estate market, building vacancy is prevalent in central cities, suburban transition zones, and scattered rural settlements. According to internationally accepted standards, buildings that have been vacant for more than six consecutive months are generally considered vacant. Such buildings not only waste land and space resources but also affect community vitality, the allocation of public services, and the healthy development of regional spatial structures. Therefore, accurate identification and dynamic monitoring of vacancy status have become crucial foundational work in urban renewal, urban and rural built environment monitoring, resource optimization, and stock planning management.

[0003] Currently, methods for identifying vacant housing mainly include: remote sensing methods based on nighttime light, identification methods based on electricity big data, and manual on-site surveys. Nighttime remote sensing data is often used to estimate regional residential activity, but its low resolution makes it unable to identify individual buildings, and it is significantly affected by terrain, obstructions, and cloud cover, making it unsuitable for accurate assessment of small- to medium-scale residential areas or individual buildings. Electricity big data methods rely on smart meter records, which involve user privacy, have poor data availability, and are not fully implemented in rural areas. While manual surveys can obtain relatively accurate information to some extent, they are costly, time-consuming, and susceptible to subjective judgment by surveyors, making it difficult to generate scalable data results. Given the scarcity of data and the concealment of characteristics in low-intensity development areas, there is an urgent need for a dedicated vacancy identification method that does not rely on traditional big data and can adapt to the characteristics of low-intensity development environments. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to address the shortcomings of the prior art by providing an automatic identification method for vacant buildings based on three-dimensional point cloud and image features. Through non-contact, high-precision three-dimensional geometric perception, a building structural integrity assessment model based on point cloud structural features and a maintenance status quantitative index system based on three-dimensional vegetation structure are established to achieve dual-path intelligent identification of structural vacancy and functional vacancy.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] An automatic building vacancy identification method based on 3D point cloud and image features includes the following steps:

[0007] Step 1: Multi-source data acquisition and 3D reconstruction;

[0008] UAVs are used to collect regional images through four-way oblique photography to make roof and facade information visible; panoramic cameras are deployed at building entrances and street nodes to fill the visual blind spots of the UAV oblique photography on building facades and under eaves, ensuring the integrity of wall verticality and window opening detection; the above data are fused and a dense, high-precision 3D scene reconstruction is completed through a 3D reconstruction algorithm to obtain a unified point cloud model of the region.

[0009] Step 2: Land feature classification and building outline extraction;

[0010] The input 3D point cloud data is preprocessed, including noise point removal, ground point separation, and regional clustering of building point clouds; the 3D point cloud model is then subjected to a cloth simulation filtering algorithm for feature classification, separating ground and non-ground point sets.

[0011] The non-ground point set is classified into vegetation based on normalized height, a digital terrain model (DTM) is constructed, a reference plane is established, and the vertical distance (HAG) from all non-ground points to the DTM is calculated. Points with HAG greater than the maximum building height and points with HAG less than the distance threshold are directly removed. Then, color, echo intensity, and slope are used to distinguish vegetation from walls and roofs, and the courtyard boundary is determined based on the wall point cloud segmentation.

[0012] Step 3: Determine the structural vacancy status;

[0013] Automatic extraction of building structural state features based on building point cloud subsets, including roof damage index D1, wall tilt index D2, and facade integrity loss index D3;

[0014] A structural vacancy assessment model is established based on the roof damage index D1, wall tilt index D2, and facade integrity loss index D3:

[0015] ;

[0016] in, This indicates that the building is structurally vacant. T1 indicates that the building is considered non-structurally vacant; T2 and T3 are the thresholds for wall tilt and facade integrity loss, respectively. When either indicator exceeds the threshold, the building is considered structurally vacant.

[0017] Step 4: Determine the functional vacancy status;

[0018] Weeds are subdivided into trees and ground cover plants, with only ground cover plants included in the calculation as weeds; ground cover plants are represented as dense, near-ground point clusters in the point cloud; vegetation classification adopts a multi-level classification method based on height distribution and geometric morphology.

[0019] Extract the characteristics of courtyard usage traces, including weed invasion index G1, weed height characteristics G2, and weed distribution dispersion G3;

[0020] Based on indicators G1, G2 and G3, a comprehensive judgment model for the usage status of courtyards is constructed. Functional vacancy levels are generated through index weighting or rule integration. When the area occupied by weeds is large, the vegetation height is abnormal and the distribution shows a significant discrete pattern, the building is judged to be in a state of neglected vacancy.

[0021] Furthermore, in step 1, the 3D reconstruction algorithm uses motion recovery structure and multi-view stereo vision algorithm, or uses 3D Gaussian sputtering algorithm to generate point cloud model.

[0022] Furthermore, in step 2, spectral and geometric feature analysis is introduced; by establishing a mapping relationship between point clouds and original images, the reflection intensity information of point clouds is analyzed, and vegetation types are identified by combining two types of vegetation indices based on visible light: normalized vegetation index and supergreen index; the calculated vegetation indices are added as scalar domains to three-dimensional points to construct a multi-dimensional feature vector including spectral index, geometric normal vector, and local roughness, which is input into a random forest classifier for training and prediction. Thus, under the condition of similar geometric shape, green man-made objects and non-green debris are eliminated by using spectral differences, and ground cover weeds are located.

[0023] Considering the significant differences in vegetation growth status across different seasons, a seasonal compensation model is further constructed. This model utilizes the continuous temporal characteristics of macro-scale multi-temporal satellite remote sensing data to compensate for the discreteness of micro-scale UAV point clouds in the temporal dimension. First, historical multi-temporal optical satellite images of the target area are acquired, the time-series normalized vegetation index is calculated, and a high-precision conformal filter is used to remove noise. A continuous phenological curve P(t) reflecting the annual growth cycle of vegetation in the region is then fitted and generated. Second, a seasonal correction coefficient K(t) is constructed based on the relative position of the UAV acquisition time point in the phenological curve. Finally, the coefficient K(t) is introduced into the judgment model for adaptive threshold adjustment. Specifically, during the vegetation withering period, the judgment threshold for weed invasion characteristics is dynamically reduced by increasing the correction coefficient, thereby compensating for weak vegetation signals during non-growing seasons and avoiding missed detection of neglected maintenance status due to seasonal yellowing. This achieves robustness and generalization ability of the model throughout the entire time cycle.

[0024] Considering the interference from ground vegetation in the environment, the height difference threshold for filtering is set higher than that of ordinary turf.

[0025] Furthermore, in step 3, the roof damage index D1 is confirmed as follows: the Laplace smoothing shrinkage operator is introduced to analyze the local geometric features of the roof surface; a method combining outlier detection based on local curvature significance and density-based clustering is used to identify discontinuous abrupt regions of the roof.

[0026] Based on roof point set For each point P in the point seti A k-nearest neighbor search is performed to construct a local neighborhood, and the geometric deviation of each point is calculated using the Laplacian operator. The calculation formula is as follows:

[0027] ;

[0028] in, The coordinates of the current point. Let k be the coordinates of a neighboring point, and k be the number of neighboring points. The centroid representing the local micro-tangent plane. Characterizes the degree to which this point deviates from the local centroid; through Threshold filtering identifies anomalous points, when When the value exceeds the set threshold, it indicates that the current point is a point of change.

[0029] For the marked outlier points, a density-based clustering algorithm is used for spatial clustering; a minimum clustering point threshold MinPt is set. s and cluster radius To eliminate discrete noise points, several continuous high-residual connected components are extracted. If at least one connected component satisfies the area constraint, this connected component is defined as a suspected roof damage area, and its bounding box is generated. The bounding box is then visualized and marked in the 3D scene. Simultaneously, the roof status label of the building is assigned. Set to 1 to indicate damage, otherwise set to 0 to indicate intact;

[0030] The wall tilt index D2 is determined as follows: Buildings with damaged roofs are removed, and a point cloud set of the walls is extracted from the remaining buildings. For each point in the set, a local covariance matrix is ​​constructed using principal component analysis, and the eigenvector corresponding to its minimum eigenvalue is calculated as the local surface normal vector for that point. ; Calculate the local surface normal vector Angle with the vertical direction Then, the verticality deviation angle of the wall can be obtained. The calculation formula is:

[0031] ;

[0032] in, The vertical axis direction vector is [0,0,1]; the structural safety limit threshold is set. The statistical wall point cloud satisfies The percentage of points or the area of ​​a continuous region, recorded as ;

[0033] The facade integrity missing index D3 is determined as follows: remove buildings with sloping roofs and obtain the point cloud of the remaining building facades. By fitting the main plane of the wall using random sampling consistency, Orthographically project the image onto the fitted main plane of the wall; rasterize the projection plane to generate a facade density map; set a density scarcity threshold, extract low-density areas as empty areas, and calculate the maximum outer contour area of ​​the facade point cloud, denoted as . Calculate the total area of ​​all missing areas, and denote it as . ; Calculate the facade integrity missing index:

[0034] ;

[0035] At this point, the facade integrity loss index Exceeding the set threshold It is marked as a suspected facade void.

[0036] Further, in step 4, the weed invasion index G1 is confirmed as follows: the courtyard space is rasterized, the presence or absence of vegetation point clouds in each grid is calculated, and the proportion of vegetation grids to the total number of courtyard grids is counted; the weed invasion index G1 reflects the degree to which the courtyard is occupied by weeds, and a higher value indicates that it is more likely to be unmanaged; based on the vegetation point identification, the proportion of vegetation inside the courtyard is determined, and a 0.5m resolution DEM is generated through vegetation category points, and the number of grids in the vegetation-covered area is recorded as follows. The total number of grid cells in the courtyard is recorded as follows: Calculate the percentage of image pixels within the courtyard that are covered by vegetation, i.e., the weed invasion index G1: ;

[0037] The weed height feature G2 is confirmed as follows: The average height and height distribution statistics of the point cloud of vegetation classified as ground cover plants are used to distinguish between natural greening and uncontrolled weed growth. When the weed height exceeds a set upper limit or the height distribution shows a random diffusion trend, it is judged as abnormal growth. The Z-axis coordinates of the vegetation point cloud within each grid are statistically analyzed, and the local height mean of each grid is calculated. The local height mean of all grids containing vegetation is globally averaged to obtain the vegetation average height, which is defined as the weed height feature. Unit: meters; The higher the value, the more lush the weeds are, and the fewer traces of human pruning or trampling.

[0038] The dispersion of weed distribution, G3, was determined as follows: Spatial clustering algorithms were used to analyze the distribution pattern of weed point clouds within the yard; when vegetation exhibited multi-patterned dispersion, irregular expansion, and coverage of activity paths, it was determined to be in a state lacking routine maintenance; the dispersion of weed distribution was quantified based on a single index: the coefficient of variation of patch area-perimeter ratio, effectively distinguishing the distribution characteristics of artificially pruned vegetation from naturally growing weeds. The calculation formula is as follows:

[0039] ;

[0040] Where APR is the ratio of area to perimeter of a single vegetation patch. (APR) is the standard deviation of the APR values ​​for all vegetation patches. (APR) is the mean APR value of all vegetation patches. The min() function ensures that the G3 value does not exceed 1 and performs normalization.

[0041] Furthermore, the vacancy states in steps 3 and 4 are divided into three levels, as follows:

[0042] Class I Vacancy: Normal use condition, well maintained building and inhabited, structurally intact with flat roof and walls, and no obvious weeds;

[0043] Level II vacancy: Moderate vacancy, uninhabited for a long time and lacking maintenance, with slight structural aging but no substantial damage, and a small amount of weeds scattered in the area.

[0044] Level III Vacancy: Severely vacant, completely abandoned, uninhabitable, with collapsed roofs, tilted walls, missing doors and windows, overgrown with weeds, which may even cover the building.

[0045] Furthermore, in the setting of the vacancy status level, a key feature veto rule is set. When the damaged area of ​​the roof exceeds 50% or the tilt angle of the wall exceeds 15°, it is directly judged as Level III vacancy.

[0046] On the other hand, this application proposes an electronic device, including: one or more processors, and a memory for storing instructions, which, when executed by the one or more processors, cause the one or more processors to perform the automatic building vacancy identification method based on three-dimensional point cloud and image features.

[0047] Thirdly, this application proposes a computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the aforementioned method for automatic identification of vacant buildings based on three-dimensional point cloud and image features.

[0048] Fourthly, this application proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned method for automatic identification of vacant buildings based on three-dimensional point cloud and image features.

[0049] The beneficial effects of adopting the above technical solutions are as follows: The automatic building vacancy identification method based on 3D point cloud and image features provided by this invention significantly improves the automation level of building vacancy identification, effectively enhances identification accuracy, and is particularly suitable for low-intensity development areas. It offers strong data acquisition timeliness, can quickly respond to market and spatial changes, and forms a standardized processing flow, which is conducive to widespread application. This invention constructs a fully automated identification system based on point cloud semantic recognition, geometric structure analysis, and 3D vegetation feature extraction. It can complete building structure damage detection, vegetation maintenance status assessment, and vacancy type determination without extensive manual interpretation, significantly reducing the degree of manual intervention and survey costs. Compared with methods relying on 2D images, this invention is particularly suitable for low-intensity development environments with complex building materials, diverse courtyard structures, and high vegetation coverage, significantly improving identification accuracy. It improves the timeliness of data acquisition, enabling rapid response to market changes. This invention can complete data collection and automated processing within a short period, making vacancy building monitoring highly timely and able to reflect changes in urban and rural spatial patterns, population migration, and building utilization in real time. It provides multi-dimensional criteria for judging vacancy status, improving the reliability of the results. This invention establishes a standardized workflow from data acquisition, point cloud preprocessing, structural analysis, vegetation structure interpretation to vacancy classification, which is easy to promote and use in natural resources departments, local governments, planning and design institutions and third-party surveying and mapping units. Attached Figure Description

[0050] Figure 1 The flowchart of the automatic building vacancy identification method based on three-dimensional point cloud and image features provided in the first embodiment of the present invention is shown below.

[0051] Figure 2 This is a schematic diagram of the vacancy index decision model provided in the first embodiment of the present invention. Detailed Implementation

[0052] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0053] Example 1:

[0054] An automatic building vacancy identification method based on 3D point cloud and image features, such as Figure 1 As shown, the method of this embodiment is described below.

[0055] Step 1: Multi-source data acquisition and 3D reconstruction.

[0056] Drones were used to acquire regional images via four-directional oblique photography, making roof and facade information visible. Panoramic cameras were deployed at building entrances and street nodes to fill in the visual blind spots of the drone's oblique photography on building facades and under eaves, ensuring the completeness of wall verticality and window opening detection. The above data were then fused together to complete a dense, high-precision 3D scene reconstruction using a 3D reconstruction algorithm, resulting in a unified point cloud model for the region.

[0057] The 3D reconstruction algorithm uses motion recovery structure and multi-view stereo vision algorithm, or uses 3D Gaussian sputtering algorithm to generate point cloud model.

[0058] Step 2: Land feature classification and building outline extraction.

[0059] The input 3D point cloud data is preprocessed, including noise point removal, ground point separation, and regional clustering of building point clouds; the Cloth Simulation Filter (CSF) algorithm is used to classify ground features on the 3D point cloud model, separating ground and non-ground point sets.

[0060] Non-ground point sets are used to classify vegetation based on normalized height, constructing a digital terrain model (DTM), establishing a reference surface, and calculating the vertical distance (HAG) from all non-ground points to the DTM. Points with HAG greater than the maximum building height and points with HAG less than the distance threshold are directly removed. Color, echo intensity, and slope are then used to distinguish vegetation from walls and roofs, and courtyard boundaries are determined based on wall point cloud segmentation. To improve classification accuracy, this embodiment introduces spectral and geometric feature analysis. By establishing a mapping relationship between the point cloud and the original image, the reflectance intensity information of the point cloud is analyzed. Combining the Normalized Difference Vegetation Index (NDVI) and the Supergreen Index (ExG), two vegetation indices based on visible light, more accurately identifies vegetation types. In specific implementation, the calculated vegetation indices are appended as scalar domains to the three-dimensional points, constructing a multi-dimensional feature vector containing spectral indices, geometric normals, and local roughness. This vector is input into a random forest classifier for training and prediction, thereby effectively removing green man-made structures and non-green debris based on spectral differences, and accurately locating ground cover weeds, even when geometrically similar.

[0061] Considering the significant differences in vegetation growth status across different seasons, a further constructed seasonal compensation model aims to leverage the continuous temporal characteristics of macro-scale multi-temporal satellite remote sensing data to compensate for the temporal discreteness of micro-scale UAV point clouds. Specifically, the model first acquires historical multi-temporal optical satellite images of the target area, calculates the time-series normalized vegetation index, and uses a Savitzky-Golay (high-precision conformal filter) to remove noise and fit a continuous phenological curve P(t) reflecting the annual growth cycle of vegetation in the region. Secondly, based on the relative position of the UAV acquisition time point within the phenological curve, a seasonal correction coefficient K(t) is constructed. Finally, K(t) is introduced into the decision model for adaptive threshold adjustment. Specifically, during the vegetation withering period, the correction coefficient is increased to dynamically lower the threshold for weed invasion characteristics, thereby compensating for weak vegetation signals during non-growing seasons and avoiding missed detections of neglected maintenance states due to seasonal yellowing, thus achieving robustness and generalization ability of the model throughout the entire time cycle.

[0062] Considering the interference from ground vegetation in the environment, the height difference threshold for filtering is set higher than that of ordinary turf.

[0063] Step 3: Determine the structural vacancy status.

[0064] The building structural state features are automatically extracted based on the building point cloud subset, including roof damage index D1, wall tilt index D2, and facade integrity missing index D3.

[0065] The roof damage index D1 is determined as follows: the Laplace smoothing shrinkage operator is introduced to analyze the local geometric features of the roof surface; and a method combining outlier detection based on local curvature significance and density-based clustering is used to identify discontinuous abrupt regions of the roof.

[0066] Based on roof point set For each point P in the point set i A k-nearest neighbor search is performed to construct a local neighborhood, and the geometric deviation of each point is calculated using the Laplacian operator. The calculation formula is as follows:

[0067] ;

[0068] in, The coordinates of the current point. Let k be the coordinates of a neighboring point, and k be the number of neighboring points. The centroid representing the local micro-tangent plane. This characterizes the degree to which the point deviates from the local centroid. (Through...) Threshold filtering identifies anomalous points, when When the value exceeds the set threshold, it indicates that the current point is a point of change. The calculation is based on the curvature change of a point. If the curvature change is large, it means that the point is not a smooth transition, but rather a point of anomaly.

[0069] For the marked outliers, a density-based clustering algorithm (DBSCAN) is used for spatial clustering. A minimum clustering threshold, MinPt, is set. s and cluster radius To eliminate discrete noise points, several continuous high-residual connected components are extracted. If at least one connected component satisfies the area constraint, this connected component is defined as a suspected roof damage area, and its bounding box is generated. The bounding box is then visualized and marked in the 3D scene. Simultaneously, the roof status label of the building to which it belongs is assigned. Set to 1 (indicating damage), otherwise set to 0 (indicating intact).

[0070] The wall tilt index D2 is determined as follows: Buildings with damaged roofs are removed, and a point cloud set of the walls is extracted from the remaining buildings. For each point in the set, a local covariance matrix is ​​constructed using Principal Component Analysis (PCA), and the eigenvector corresponding to its smallest eigenvalue is calculated as the local surface normal vector for that point. Calculate the local surface normal vector. Angle with the vertical direction Then, the verticality deviation angle of the wall can be obtained. The calculation formula is:

[0071] ;

[0072] in, The direction vector is the vertical axis, i.e., [0,0,1]. The structural safety limit threshold is set. The statistical wall point cloud satisfies The percentage of points or the area of ​​a continuous region, recorded as .

[0073] The facade integrity missing index D3 is determined as follows: remove buildings with sloping roofs and obtain the point cloud of the remaining building facades. The main plane of the wall was fitted using Random Sample Consensus (RANSAC). Orthographically project the image onto this local plane. Rasterize the projection plane to generate a facade density map. Set a density scarcity threshold, extract low-density areas as void regions, and calculate the maximum outer contour area of ​​the facade point cloud, denoted as . Calculate the total area of ​​all missing areas, and denote it as . Calculate the facade integrity missing index:

[0074] ;

[0075] At this point, the facade integrity loss index Exceeding the set threshold (This embodiment takes) (i.e., a vacancy rate exceeding 35%) is marked as a suspected facade void.

[0076] A structural vacancy assessment model is established based on the roof damage index D1, wall tilt index D2, and facade integrity loss index D3:

[0077] ;

[0078] in, This indicates that the building is structurally vacant. This indicates that the building is considered non-structurally vacant. T2 and T3 are the thresholds for wall tilt and facade integrity loss, respectively. When either indicator exceeds the threshold, the building is considered structurally vacant.

[0079] Step 4: Determine the functional vacancy status;

[0080] Weeds were further subdivided into trees and ground cover plants, with only ground cover plants included in the calculations as weeds. Ground cover plants appear as dense, near-ground point clusters in the point cloud. Vegetation classification employed a multi-level classification method based on height distribution and geometric morphology.

[0081] Extract the characteristics of courtyard usage traces, including weed invasion index G1, weed height characteristics G2, and weed distribution dispersion G3.

[0082] The weed invasion index G1 is determined as follows: the courtyard space is rasterized, the presence or absence of vegetation point clouds in each grid is calculated, and the proportion of vegetation grids to the total number of courtyard grids is counted. The weed invasion index G1 reflects the degree to which the courtyard is occupied by weeds; a higher value indicates a more likely state of neglect. Based on vegetation point identification, the proportion of vegetation within the courtyard is determined, and a 0.5m resolution DEM is generated using vegetation category points. The number of grids in the vegetation-covered area is recorded. The total number of grid cells in the courtyard is recorded as follows: Calculate the percentage of image pixels within the courtyard that are covered by vegetation, i.e., the weed invasion index G1: .

[0083] The weed height feature G2 is confirmed as follows: The average height and height distribution statistics of the point cloud of vegetation classified as ground cover plants are used to distinguish between natural vegetation and uncontrolled weed growth. When the weed height exceeds a set upper limit or the height distribution shows a random diffusion trend, it is judged as abnormal growth. The Z-axis coordinates of the vegetation point cloud within each grid are statistically analyzed, and the local average height of each grid is calculated. The local average height of all grids containing vegetation is then globally averaged to obtain the average vegetation height, which is defined as the weed height feature. (Unit: meters). The higher the value, the more lush the weeds are, and the less trace of human pruning or trampling.

[0084] The dispersion of weed distribution, G3, was determined as follows: Spatial clustering algorithms were used to analyze the point cloud distribution pattern of weeds within the yard. When vegetation exhibited multi-patterned dispersion, irregular expansion, and coverage of activity paths, it was deemed to be in a state of lack of routine maintenance. The dispersion of weed distribution was quantified using a single index: the coefficient of variation of patch area-perimeter ratio. This effectively distinguishes the distribution characteristics of artificially pruned vegetation from those of naturally creeping weeds. The calculation formula is as follows:

[0085] ;

[0086] Where APR is the ratio of area to perimeter of a single vegetation patch. (APR) is the standard deviation of the APR values ​​for all vegetation patches. (APR) is the mean APR value of all vegetation patches. The min() function ensures that the G3 value does not exceed 1 and performs normalization.

[0087] A comprehensive judgment model for usage status is established based on the weed invasion index G1, weed height characteristics G2, and weed distribution dispersion G3. When any two indicators exceed the threshold, it is judged as unmaintained and vacant.

[0088] Based on indicators G1, G2 and G3, a comprehensive judgment model for the usage status of courtyards is constructed. Functional vacancy levels are generated through index weighting or rule integration. When the area occupied by weeds is large, the vegetation height is abnormal and the distribution shows a significant discrete pattern, the building is judged to be in a state of neglected vacancy.

[0089] The vacancy status in steps 3 and 4 is divided into three levels, as shown in Table 1.

[0090] Table 1 Classification of Vacancy Status Levels

[0091] grade code Definitions Structural vacancy features Functional vacancy characteristics Level I Normal use The building is well-maintained and inhabited. The structure is intact, and the roof and walls are flat. No obvious weeds Level II Moderate vacancy Long-term uninhabited and lack of maintenance Slight structural aging, no substantial damage There are a few weeds, scattered in a dispersed manner. Level III Heavy vacancy Completely abandoned and uninhabitable Roof collapsed, walls tilted, facade missing Weeds grew rampant, even covering the buildings.

[0092] In the vacancy status level setting, a key feature veto rule is set. When there is structural damage, it is directly judged as Level III vacancy.

[0093] In this embodiment, the determination of each indicator of structural vacancy and functional vacancy is shown in Table 2.

[0094] Table 2. Judgment of various indicators for structural and functional vacancy.

[0095] Factor categories index formula Structural vacancy <![CDATA[Roof damage index D1]]> <![CDATA[D1=1, Roof damaged]]> Structural vacancy <![CDATA[Wall inclination angle D2]]> <![CDATA[D2≥5°, the wall is inclined]]> Structural vacancy <![CDATA[Lack of facade integrity D3]]> <![CDATA[D3≥0.35, lack of facade integrity]]> Functional vacancy <![CDATA[Vegetation encroachment index G1]]> <![CDATA[G1≥0.35, there is a large amount of vegetation]]> Functional vacancy <![CDATA[Weed height G2]]> <![CDATA[G2≥0.6m, long vegetation growth time]]> Functional vacancy <![CDATA[Weed dispersion degree G3]]> <![CDATA[G3≥0.6, natural vegetation growth]]>

[0096] The vacancy index decision model obtained by the method in this embodiment is as follows: Figure 2 As shown.

[0097] The vacancy type is determined as shown in Table 3. When any one of the structural indicators exceeds the threshold, it is judged as structural vacancy. When any two of the functional indicators exceed the threshold, it is unmaintainable vacancy.

[0098] Table 3 Vacancy Type Determination Table

[0099] Structural vacancy Unmaintained vacancy Final type 1 0 Structural vacancy 0 1 Functional vacancy 1 1 Severely vacant (completely vacant) 0 0 Not empty (normal use)

[0100] Example 2:

[0101] This embodiment proposes an electronic device, including: one or more processors, and a memory, wherein the memory is used to store instructions, and when the instructions are executed by the one or more processors, the one or more processors execute the automatic building vacancy identification method based on three-dimensional point cloud and image features.

[0102] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the automatic building vacancy identification method based on 3D point cloud and image features as described in the embodiments. It is understood that the electronic device may also include an input / output (I / O) interface and communication components.

[0103] The processor is used to execute all or part of the steps in the automatic building vacancy identification method based on 3D point cloud and image features as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.

[0104] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the automatic building vacancy identification method based on three-dimensional point cloud and image features described in the above embodiments.

[0105] Example 3:

[0106] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0107] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the automatic building vacancy identification method based on three-dimensional point cloud and image features described in the various embodiments of this application.

[0108] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disk, optical disk, server, APP (Application) application store, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the aforementioned automatic building vacancy identification method based on 3D point cloud and image features.

[0109] Example 4:

[0110] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned method for automatic identification of vacant buildings based on 3D point cloud and image features.

[0111] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the present invention.

Claims

1. A method for automatic identification of vacant buildings based on 3D point cloud and image features, characterized in that: Includes the following steps: Step 1: Multi-source data acquisition and 3D reconstruction; UAVs are used to collect regional images through four-way oblique photography to make roof and facade information visible; panoramic cameras are deployed at building entrances and street nodes to fill the visual blind spots of the UAV oblique photography on building facades and under eaves, ensuring the integrity of wall verticality and window opening detection; the above data are fused and a dense, high-precision 3D scene reconstruction is completed through a 3D reconstruction algorithm to obtain a unified point cloud model of the region. Step 2: Land feature classification and building outline extraction; The input 3D point cloud data is preprocessed, including noise point removal, ground point separation, and regional clustering of building point clouds; A cloth-simulation filtering algorithm is used to classify ground features in a 3D point cloud model, separating ground and non-ground point sets. The non-ground point set is classified into vegetation based on normalized height, a digital terrain model (DTM) is constructed, a reference plane is established, and the vertical distance (HAG) from all non-ground points to the DTM is calculated. Points with HAG greater than the maximum building height and points with HAG less than the distance threshold are directly removed. Then, color, echo intensity, and slope are used to distinguish vegetation from walls and roofs, and the courtyard boundary is determined based on the wall point cloud segmentation. Step 3: Determine the structural vacancy status; Automatic extraction of building structural state features based on building point cloud subsets, including roof damage index D1, wall tilt index D2, and facade integrity loss index D3; A structural vacancy assessment model is established based on the roof damage index D1, wall tilt index D2, and facade integrity loss index D3: ； in, This indicates that the building is structurally vacant. T1 indicates that the building is considered non-structurally vacant; T2 and T3 are the thresholds for wall tilt and facade integrity loss, respectively. When either indicator exceeds the threshold, the building is considered structurally vacant. Step 4: Determine the functional vacancy status; Weeds are subdivided into trees and ground cover plants, with only ground cover plants included in the calculation as weeds; ground cover plants are represented as dense, near-ground point clusters in the point cloud; vegetation classification adopts a multi-level classification method based on height distribution and geometric morphology. Extract features of courtyard usage traces, including weed invasion index G1, weed height feature G2, and weed distribution dispersion G3; Based on indicators G1, G2 and G3, a comprehensive judgment model for the usage status of courtyards is constructed. Functional vacancy levels are generated through index weighting or rule integration. When the area occupied by weeds is large, the vegetation height is abnormal and the distribution shows a significant discrete pattern, the building is judged to be in a state of neglected vacancy.

2. The method for automatic identification of vacant buildings based on three-dimensional point cloud and image features according to claim 1, characterized in that: In step 1, the 3D reconstruction algorithm uses motion recovery structure and multi-view stereo vision algorithm, or uses 3D Gaussian sputtering algorithm to generate point cloud model.

3. The method for automatic identification of vacant buildings based on three-dimensional point cloud and image features according to claim 1, characterized in that: In step 2, spectral and geometric feature analysis is introduced. By establishing a mapping relationship between point clouds and original images, the reflectance intensity information of point clouds is analyzed. Combined with the normalized vegetation index and the supergreen index, two types of vegetation indices based on visible light, vegetation types are identified. The calculated vegetation indices are added to the three-dimensional points as scalar domains to construct a multi-dimensional feature vector including spectral indices, geometric normals, and local roughness. This feature vector is then input into a random forest classifier for training and prediction. Thus, under the condition of similar geometric shapes, spectral differences are used to eliminate green man-made objects and non-green debris, and to locate ground cover weeds. Considering the significant differences in vegetation growth status across different seasons, a seasonal compensation model is further constructed. This model utilizes the continuous temporal characteristics of macro-scale multi-temporal satellite remote sensing data to compensate for the discreteness of micro-scale UAV point clouds in the temporal dimension. First, historical multi-temporal optical satellite images of the target area are acquired, the time-series normalized vegetation index is calculated, and a high-precision conformal filter is used to remove noise. A continuous phenological curve P(t) reflecting the annual growth cycle of vegetation in the region is then fitted and generated. Second, a seasonal correction coefficient K(t) is constructed based on the relative position of the UAV acquisition time point in the phenological curve. Finally, the coefficient K(t) is introduced into the judgment model for adaptive threshold adjustment. That is, during the vegetation withering period, the judgment threshold for weed invasion characteristics is dynamically reduced by increasing the correction coefficient, thereby compensating for weak vegetation signals during non-growing periods and avoiding missed judgments of neglected maintenance status due to seasonal yellowing. This achieves robustness and generalization ability of the model throughout the entire time cycle. Considering the interference from ground vegetation in the environment, the height difference threshold for filtering is set higher than that of ordinary turf.

4. The method for automatic identification of vacant buildings based on three-dimensional point cloud and image features according to claim 1, characterized in that: In step 3, the roof damage index D1 is confirmed as follows: the Laplace smoothing shrinkage operator is introduced to analyze the local geometric features of the roof surface; and a method combining outlier detection based on local curvature significance and density-based clustering is used to identify discontinuous abrupt regions of the roof. Based on roof point set For each point P in the point set i A k-nearest neighbor search is performed to construct a local neighborhood, and the geometric deviation of each point is calculated using the Laplacian operator. The calculation formula is as follows: ； in, The coordinates of the current point. Let k be the coordinates of a neighboring point, and k be the number of neighboring points. The centroid representing the local micro-tangent plane. Characterizes the degree to which this point deviates from the local centroid; through Threshold filtering identifies anomalous points, when When the value exceeds the set threshold, it indicates that the current point is a point of change. For the marked outlier points, a density-based clustering algorithm is used for spatial clustering; a minimum clustering point threshold MinPt is set. s and cluster radius To eliminate discrete noise points, several continuous high-residual connected components are extracted. If at least one connected component satisfies the area constraint, this connected component is defined as a suspected roof damage area, and its bounding box is generated. The bounding box is then visualized and marked in the 3D scene. Simultaneously, the roof status label of the building is assigned. Set to 1 to indicate damage, otherwise set to 0 to indicate intact; The wall tilt index D2 is determined as follows: Buildings with damaged roofs are removed, and a point cloud set of the walls is extracted from the remaining buildings. For each point in the set, a local covariance matrix is ​​constructed using principal component analysis, and the eigenvector corresponding to its minimum eigenvalue is calculated as the local surface normal vector for that point. ; Calculate the local surface normal vector Angle with the vertical direction Then, the verticality deviation angle of the wall can be obtained. The calculation formula is: ； in, The vertical axis direction vector is [0,0,1]; the structural safety limit threshold is set. The statistical wall point cloud satisfies The percentage of points or the area of ​​a continuous region, recorded as ; The facade integrity missing index D3 is determined as follows: remove buildings with sloping roofs and obtain the point cloud of the remaining building facades. By fitting the main plane of the wall using random sampling consistency, Orthographically project the image onto the fitted main plane of the wall; rasterize the projection plane to generate a facade density map; set a density scarcity threshold, extract low-density areas as empty areas, and calculate the maximum outer contour area of ​​the facade point cloud, denoted as . Calculate the total area of ​​all missing areas, and denote it as . ; Calculate the facade integrity missing index: ； At this point, the facade integrity loss index Exceeding the set threshold It is marked as a suspected facade void.

5. The method for automatic identification of vacant buildings based on three-dimensional point cloud and image features according to claim 4, characterized in that: In step 4, the weed invasion index G1 is determined as follows: the courtyard space is rasterized, the presence or absence of vegetation point clouds in each grid is calculated, and the proportion of vegetation grids to the total number of courtyard grids is counted; the weed invasion index G1 reflects the degree to which the courtyard is occupied by weeds, and a higher value indicates a more likely state of neglect; based on vegetation point identification, the proportion of vegetation inside the courtyard is determined, and a 0.5m resolution DEM is generated through vegetation category points, with the number of grids in the vegetation-covered area recorded as follows. The total number of grid cells in the courtyard is recorded as follows: Calculate the percentage of image pixels within the courtyard that are covered by vegetation, i.e., the weed invasion index G1: ; The weed height feature G2 is confirmed as follows: The average height and height distribution statistics of the point cloud of vegetation classified as ground cover plants are used to distinguish between natural greening and uncontrolled weed growth. When the weed height exceeds a set upper limit or the height distribution shows a random diffusion trend, it is judged as abnormal growth. The Z-axis coordinates of the vegetation point cloud within each grid are statistically analyzed, and the local height mean of each grid is calculated. The local height mean of all grids containing vegetation is globally averaged to obtain the vegetation average height, which is defined as the weed height feature. Unit: meters; The higher the value, the more lush the weeds are, and the fewer traces of human pruning or trampling. The dispersion of weed distribution, G3, was determined as follows: Spatial clustering algorithms were used to analyze the distribution pattern of weed point clouds within the yard; when vegetation exhibited multi-patterned dispersion, irregular expansion, and coverage of activity paths, it was determined to be in a state lacking routine maintenance; the dispersion of weed distribution was quantified based on a single index: the coefficient of variation of patch area-perimeter ratio, effectively distinguishing the distribution characteristics of artificially pruned vegetation from naturally growing weeds. The calculation formula is as follows: ； Where APR is the ratio of area to perimeter of a single vegetation patch. (APR) is the standard deviation of the APR values ​​for all vegetation patches. (APR) is the mean APR value of all vegetation patches. The min() function ensures that the G3 value does not exceed 1 and performs normalization.

6. The method for automatic identification of vacant buildings based on three-dimensional point cloud and image features according to claim 5, characterized in that: The vacancy state in steps 3 and 4 is divided into three levels, as follows: Class I Vacancy: Normal use condition, well maintained building and inhabited, structurally intact with flat roof and walls, and no obvious weeds; Level II vacancy: Moderate vacancy, uninhabited for a long time and lacking maintenance, with slight structural aging but no substantial damage, and a small amount of weeds scattered in the area. Level III Vacancy: Severely vacant, completely abandoned, uninhabitable, with collapsed roofs, tilted walls, missing doors and windows, overgrown with weeds, which may even cover the building.

7. The method for automatic identification of vacant buildings based on three-dimensional point cloud and image features according to claim 6, characterized in that: In the classification of the vacancy status, a key feature veto rule is set. When the damaged area of ​​the roof exceeds 50% or the tilt angle of the wall exceeds 15°, it is directly judged as Level III vacancy.

8. An electronic device, characterized in that: The method includes one or more processors and a memory for storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the automatic building vacancy identification method based on three-dimensional point cloud and image features as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that: It stores executable instructions that, when executed, cause the processor to perform the automatic building vacancy identification method based on three-dimensional point cloud and image features as described in any one of claims 1-7.

10. A computer program product, characterized in that: Includes a computer program or instructions that, when executed by a processor, implement the automatic building vacancy identification method based on three-dimensional point cloud and image features as described in any one of claims 1-7.

Images