A building layering intelligent recognition method and system based on multi-modal remote sensing data

Abstract

The application provides a building layering intelligent recognition method and system based on multi-modal remote sensing data, and belongs to the technical field of building detection. Multi-modal and multi-view remote sensing images including optical remote sensing images and radar remote sensing images are continuously acquired; the remote sensing images are input into a deep learning prediction model to realize accurate prediction of the height of urban building groups; the building height information predicted at multiple time nodes is calculated by a difference analysis method and a change vector analysis to obtain the change value of height data in different periods; the buildings whose height change values exceed a first threshold value are preliminarily marked as layering buildings; a weighted local spatial anomaly measurement method is further applied to calculate a spatial consistency index of height change in the region, and if the index exceeds a second threshold value, the building is finally determined as a layering building. The application can realize automatic and intelligent identification of illegal layering buildings in urban areas, and provide a scientific basis for urban management and law enforcement.

Description

Technical Field

[0001] This invention belongs to the field of building inspection technology, specifically relating to a method and system for intelligent identification of building additions based on multimodal remote sensing data. Background Technology

[0002] With the acceleration of urbanization and the continuous increase in urban building density, a large number of unapproved self-built houses have appeared in some areas. These buildings generally lack professional design and construction, and some residents have added floors without authorization to expand their living space, resulting in serious structural safety hazards. Therefore, it is necessary to conduct regular inspections of building additions to eliminate potential risks.

[0003] Traditional building addition inspections mainly rely on manual inspections, which suffer from high labor costs, low efficiency, and long cycles, making it difficult to achieve continuous monitoring and timely intervention in large urban areas.

[0004] In recent years, with the development of image recognition technology, some technologies have adopted aerial photography to identify building additions. However, such methods still have the following drawbacks: First, aerial photography is limited by factors such as flight path, weather, and operating conditions, and it is difficult to ensure that the position and angle of the images are completely consistent when taking multiple shots, which affects the accuracy of subsequent change detection. Second, the coverage of aerial photography operations is relatively limited, which is difficult to meet the needs of efficient and continuous monitoring in large urban areas, thus restricting the application and promotion of the technology in urban refined management.

[0005] Therefore, it is necessary to provide a method and system for intelligent identification of building additions based on multimodal remote sensing data to solve the above problems. Summary of the Invention

[0006] This invention provides a method and system for intelligent identification of building additions based on multimodal remote sensing data. It utilizes multi-view, multimodal remote sensing images of buildings to train a deep learning prediction model to predict building height. The remote sensing image detection method is well-suited for large-scale building addition detection, and the use of data from different viewpoints and modalities avoids identification errors caused by single-source data. Furthermore, by employing adaptive statistical thresholds and spatial consistency analysis methods, the target building and neighboring buildings within a certain range are treated as a whole for addition determination, effectively improving identification accuracy. This addresses at least one of the technical problems mentioned in the background section.

[0007] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0008] A method for intelligent identification of building additions based on multimodal remote sensing data includes the following steps:

[0009] Step S1: Construct an observation dataset, which includes remote sensing images of multiple different buildings from different perspectives and modalities, with the height of the corresponding building in each remote sensing image as the label.

[0010] Step S2: Construct a deep learning prediction model and train the deep learning prediction model using the observation dataset to learn the mapping relationship between remote sensing images of buildings and their height.

[0011] Step S3: For any building addition identification task, select a time series containing multiple consecutive time nodes. At each time node, collect remote sensing images of the target building, input the trained deep learning prediction model, output the height prediction value of the target building at the current time node, calculate the first difference between the height prediction value of the target building at the current time node and the height prediction value at the previous time node as the height change value of the target building, and traverse all time nodes to form the height change sequence of the target building.

[0012] Step S4: Select the target area and calculate the height change sequence of all buildings in the target area according to step S3. At any time point, set a first threshold based on the height change value of all buildings in the target area. If the height change value of the target building exceeds the first threshold, it is initially determined that the target building has added floors, and step S5 is executed; otherwise, it is determined that the target building does not have added floors, and the determination ends.

[0013] Step S5: The weighted local spatial anomaly measurement method is used to calculate the height change consistency index between the target building and other buildings in the target area. A second threshold is set. If the height change consistency index exceeds the second threshold, it is finally determined that the target building has added floors, and the determination ends; otherwise, it is finally determined that the target building does not have added floors, and the determination ends.

[0014] As a preferred improvement, building remote sensing imagery includes optical remote sensing imagery and radar remote sensing imagery, wherein optical remote sensing imagery includes high-resolution optical imagery and wide-swath optical imagery.

[0015] As a preferred improvement, the observation dataset needs to be preprocessed before being input into the deep learning prediction model. The preprocessing process includes geometric fine correction, orthorectification, radiometric correction and image enhancement to ensure data quality. At the same time, the sliding window technique is used to crop the remote sensing image to the required standard size, and a zero-value filling strategy is adopted for edge areas that cannot be divided evenly to ensure data integrity.

[0016] As a preferred improvement, the deep learning prediction model includes a multimodal feature extraction module, a feature fusion module, a spatial context information enhancement module, and an output module. The multimodal feature extraction module uses long and short attention mechanisms to extract multi-scale building features from optical remote sensing images and radar remote sensing images, respectively. The feature fusion module uses an adaptive fusion attention mechanism to fuse building features of different modalities and scales to obtain fused features. The spatial context information enhancement module captures the relationship between buildings and their surrounding environment to enhance the fused features. The output module processes the fused features and outputs the predicted height of the buildings.

[0017] As a preferred improvement, the deep learning prediction model training process employs a fusion dice loss. Boundary loss And Lovas loss Composite loss function , is represented as:

[0018]

[0019] In the formula, , , These represent the losses from dice rolls. Boundary loss And Lovas loss The weighting coefficients; where:

[0020] Dice loss Represented as:

[0021]

[0022] In the formula, Represents any pixel in a remotely sensed image; This represents the total number of pixels in the remote sensing image. This indicates that the deep learning prediction model predicts the pixel... The predicted probability; Represents pixels The actual pixel value; This represents the minimum value and is used to prevent the denominator from being 0;

[0023] Boundary loss Represented as:

[0024]

[0025] In the formula, Represents pixels SDM value;

[0026] Lovas loss Represented as:

[0027]

[0028] In the formula, Represents a set of pixels; This represents the sorted IoU loss gradient sequence; This represents any element in the IoU loss gradient sequence.

[0029] As a preferred improvement, in step S4, the target region is determined through the following process:

[0030] Step S41: The remote sensing image is gridded, and the building density within each grid cell is calculated. The calculation process is as follows:

[0031]

[0032] In the formula, Indicates the building density within a grid cell; This represents the total building area within the grid cell, calculated from the building outline data. Represents the total area of ​​the grid cells;

[0033] Step S42: Within each grid cell, the target region is determined using the sliding window method. The sliding window size is expressed as:

[0034]

[0035] In the formula, Indicates the size of the sliding window; Indicates the reference window size; Indicates the density adjustment coefficient;

[0036] Step S43: Select multiple target regions one by one according to the preset window overlap rate and step size, wherein:

[0037] Window overlap rate Represented as:

[0038]

[0039] In the formula, Indicates the baseline overlap rate; Indicates the overlap adjustment coefficient;

[0040] Step length Represented as:

[0041] .

[0042] As a preferred improvement, the process of setting the first threshold specifically includes the following steps:

[0043] If the height variation values ​​of all buildings in the target area follow a normal distribution, then the first threshold is... Set as:

[0044]

[0045] In the formula, This represents the average value of all building height variations within the target area. The coefficient factor representing control sensitivity has a value range of 2-3; The standard deviation of all building height variations within the target area; This represents the regional heterogeneity adjustment factor, with a value ranging from 0.1 to 0.5. Represents the coefficient of variation. ; This represents the total number of buildings in the target area; This represents the baseline sample size, ranging from 50 to 100.

[0046] If the height variation values ​​of all buildings in the target area are severely skewed, then the first threshold... Configure it using any of the following methods:

[0047] Firstly:

[0048]

[0049] In the formula, This represents the median of all building height variations within the target area; This represents the absolute median difference of all building height variations within the target area;

[0050] Secondly:

[0051]

[0052] In the formula, This represents the 95th percentile of all building height changes within the target area. In other words, if the height change of the target building exceeds 95% of the building height changes in the sample within the area, it is considered abnormal.

[0053] As a preferred improvement, the high variability consistency index is represented by a spatial anomaly metric, the calculation process of which is as follows:

[0054]

[0055] In the formula, Indicates the target building Spatial anomaly measurement; Indicates the target building The change in altitude; Indicates buildings within the target area The change in altitude; Represents the spatial weight matrix, and its relation to the target building. and neighboring buildings The distance is inversely proportional to the distance; Indicates that the target building is included. The target area; Indicates the target area The standard deviation of the height variation of all buildings within the area.

[0056] As a preferred improvement, the value of the second threshold is in the range of 2.5-3.5, where: for densely built areas, the second threshold is 3.0; for sparsely built areas, the second threshold is 2.5; and for special regulatory areas, the second threshold is 3.5.

[0057] A system for implementing the above-described intelligent building addition identification method based on multimodal remote sensing data includes:

[0058] The observation dataset includes remote sensing images of multiple buildings from different perspectives and modalities, with the height of the corresponding building in each remote sensing image as the label;

[0059] The deep learning prediction model is trained using an observation dataset to learn the mapping relationship between remote sensing images of buildings and their heights. For any building addition identification task, a time series containing multiple consecutive time nodes is selected. At each time node, remote sensing images of the target building are collected, input into the trained deep learning prediction model, and the predicted height of the target building at the current time node is output. The first-order difference between the predicted height of the target building at the current time node and the predicted height at the previous time node is calculated as the height change value of the target building. This process is repeated for all time nodes to form a sequence of height change of the target building.

[0060] The preliminary judgment module is used to select a target area, calculate the height change sequence of all buildings in the target area, and set a first threshold based on the height change value of all buildings in the target area at any time node. If the height change value of the target building exceeds the first threshold, it is preliminarily judged that the target building has added floors, and step S5 is executed; otherwise, it is judged that the target building does not have added floors, and the judgment ends.

[0061] The final determination module is used to calculate the height change consistency index between the target building and other buildings in the target area using a weighted local spatial anomaly measurement method. A second threshold is set. If the height change consistency index exceeds the second threshold, the target building is determined to have an added floor, and the determination ends; otherwise, the target building is determined not to have an added floor, and the determination ends.

[0062] The beneficial effects of this invention are as follows:

[0063] (1) The detection method based on high-resolution remote sensing images breaks through the spatial limitations of traditional monitoring methods and can achieve full coverage monitoring of large urban areas. Whether it is the core area of ​​the city or the edge area, the system can provide a unified standard monitoring service, which solves the problems of uneven coverage and blind spots in supervision of traditional methods and provides a full-area perspective for urban management.

[0064] (2) By constructing observation datasets using remote sensing images from different perspectives and modalities, the problem of blind spots and occlusion in traditional single-view remote sensing monitoring is effectively solved. The system can capture the three-dimensional structural features of buildings from different angles. Through perspective complementarity and ensemble learning methods, the accuracy and completeness of building height estimation are significantly improved. The multi-view observation strategy is well applicable to high-density urban environments and can effectively reduce the monitoring blind spots caused by the occlusion of high-rise buildings, achieving comprehensive coverage of illegal addition monitoring.

[0065] (3) Deep learning is used to directly predict the height of buildings instead of the traditional image difference-based change detection method. This method significantly improves the detection accuracy and effectively avoids the inherent defects of the image difference method, which is affected by lighting conditions, seasonal changes and shooting angle. It can accurately identify the real vertical changes of buildings and reduce the false alarm rate and false negative rate.

[0066] (4) The design of adaptive statistical threshold and spatial consistency threshold can effectively distinguish between normal building changes and illegal additions, significantly reducing the false alarm rate. The system can intelligently identify and filter changes in non-illegal addition types such as road vehicles and temporary buildings, improving the accuracy and reliability of identification.

[0067] (5) By setting a reasonable data collection frequency, a continuous monitoring mechanism for illegal addition of floors can be realized. The system regularly acquires the latest remote sensing data of the target area, which can capture information on changes in building height in a timely manner, and realize early detection and intervention of illegal addition of floors. This dynamic monitoring mode significantly improves the timeliness of supervision and effectively prevents the spread of illegal construction.

[0068] (6) Based on the real-time detection results, a list of illegal additions to buildings and a spatial distribution map can be automatically generated, providing decision support for urban management and law enforcement departments. This is of great significance for promoting urban building safety supervision, preventing building safety risks, and protecting the lives and property of urban residents. Detailed Implementation

[0069] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] This embodiment provides a method for intelligent identification of building additions based on multimodal remote sensing data, including the following steps:

[0071] Step S1: Construct an observation dataset, which includes remote sensing images of multiple different buildings from different perspectives and modalities, with the height of the corresponding building in each remote sensing image as the label.

[0072] Remote sensing imagery technology utilizes sensors mounted on satellites, drones, and other carriers to detect and record electromagnetic wave information emitted and reflected by buildings from a distance, presenting it in image form. In this embodiment, a satellite is used as the sensor carrier, and the sensors include optical sensors and radar sensors. Therefore, the obtained building remote sensing images include optical remote sensing images and radar remote sensing images, whereby the optical remote sensing images include high-resolution optical images and wide-swath optical images. The detection method based on high-resolution remote sensing images breaks through the spatial limitations of traditional monitoring methods, enabling full-coverage monitoring of large urban areas. Whether in the core urban area or the periphery, a unified standard monitoring service can be provided, solving problems such as uneven coverage and blind spots in traditional methods, and providing a comprehensive perspective for urban management.

[0073] During the remote sensing image acquisition process, image data from multiple satellites should be acquired simultaneously. The optical remote sensing images should be grouped according to the shooting time and coverage area. Each group of optical remote sensing images can be matched with multiple radar remote sensing images to form a multi-view observation dataset. Each observation dataset contains building remote sensing images from different perspectives and in different modes (optical mode and radar mode), which can ensure the diversity of data samples, improve the generalization ability of subsequent models, help the models better adapt to different inputs, avoid errors caused by a single data source, and increase the accuracy and robustness of predictions.

[0074] The building height is obtained from radar point cloud data, DSM or DEM data. It is important to note that during the labeling process, spatiotemporal consistency issues need to be considered. That is, the time and spatial location of the label need to be consistent with the time and spatial location of the target building in the building remote sensing image.

[0075] The observation dataset needs to be preprocessed, including geometric correction, orthorectification, radiometric correction, and image enhancement to ensure data quality. At the same time, the sliding window technique is used to crop the remote sensing images to the required standard size, and a zero-value filling strategy is adopted for edge areas that cannot be divided evenly to ensure data integrity.

[0076] Step S2: Construct a deep learning prediction model and train it using the observation dataset to learn the mapping relationship between remote sensing images of buildings and their height.

[0077] It should be noted that the construction and processing of the deep learning prediction model are existing technologies in this field. In this embodiment, the deep learning prediction model improves the loss function based on the traditional deep learning prediction model, while the other architectures remain consistent with those of the traditional deep learning prediction model.

[0078] Instead of traditional image-based change detection methods, this method uses deep learning to directly predict building heights. This significantly improves detection accuracy and effectively avoids the inherent limitations of image-based methods, which are affected by lighting conditions, seasonal changes, and shooting angles. It can accurately identify the true vertical changes of buildings and reduce false alarm and false negative rates.

[0079] Specifically, the deep learning prediction model includes a multimodal feature extraction module, a feature fusion module, a spatial context information enhancement module, and an output module. The multimodal feature extraction module uses long and short attention mechanisms to extract multi-scale building features from optical remote sensing images and radar remote sensing images, respectively. The feature fusion module uses an adaptive fusion attention mechanism to fuse building features of different modalities and scales to obtain fused features. The spatial context information enhancement module captures the relationship between buildings and their surrounding environment to enhance the fused features. The output module processes the fused features and outputs the predicted height of the buildings.

[0080] In this embodiment, a fusion dice loss is designed. Boundary loss And Lovas loss Composite loss function , is represented as:

[0081]

[0082] In the formula, , , These represent the losses from dice rolls. Boundary loss And Lovas loss The weighting coefficients; where:

[0083] Dice loss The focus is on improving the overall segmentation quality and resolving the class imbalance problem, as shown below:

[0084]

[0085] In the formula, Represents any pixel in a remotely sensed image; This represents the total number of pixels in the remote sensing image. This indicates that the deep learning prediction model predicts the pixel... The predicted probability; Represents pixels The actual pixel value; This represents the minimum value and is used to prevent the denominator from being 0;

[0086] Boundary loss Focusing on improving the accuracy of building boundary prediction, it is expressed as:

[0087]

[0088] In the formula, Represents pixels SDM value;

[0089] Lovas loss To optimize the model's IoU performance and improve the spatial consistency of highly predictive data, it is represented as:

[0090]

[0091] In the formula, Represents a set of pixels; This represents the sorted IoU loss gradient sequence; This represents any element in the IoU loss gradient sequence.

[0092] The composite loss function provided in this embodiment Incorporating dice loss Boundary loss And Lovas loss The advantages of these three elements can greatly improve the prediction accuracy of the model.

[0093] During the training of deep learning prediction models, a preheating learning rate adjustment strategy is adopted to optimize the model convergence process. A phased training strategy is used, combined with an early stopping mechanism and model checkpoint saving, to ensure the model's generalization ability.

[0094] Step S3: For any building addition identification task, select a time series containing multiple consecutive time nodes. At each time node, collect remote sensing images of the target building, input the trained deep learning prediction model, output the height prediction value of the target building at the current time node, calculate the first difference between the height prediction value of the target building at the current time node and the height prediction value at the previous time node as the height change value of the target building, and traverse all time nodes to form the height change sequence of the target building.

[0095] The height difference between two adjacent time points reflects the height change of the target building in a short period of time, including whether a height change has occurred, the rate of change, and the magnitude of the change. Preferably, the time series includes at least three time points to eliminate interference from seasonal factors and temporary structural changes.

[0096] Step S4: Select the target area and calculate the height change sequence of all buildings in the target area according to step S3. At any time node, set a first threshold based on the height change value of all buildings in the target area. If the height change value of the target building exceeds the first threshold, it is initially determined that the target building has added floors, and step S5 is executed; otherwise, it is determined that the target building does not have added floors, and the determination ends.

[0097] The size of the target region is determined through the following process:

[0098] Step S41: The remote sensing image is gridded, and the building density within each grid cell is calculated. The calculation process is as follows:

[0099]

[0100] In the formula, Indicates the building density within a grid cell; This represents the total building area within the grid cell, calculated from the building outline data. This represents the total area of ​​the grid cells.

[0101] In this embodiment, the size of the grid cell is 10km × 10km, and the total area of ​​the grid cells is 100km². 2 .

[0102] Step S42: Within each grid cell, the target region is determined using the sliding window method. The sliding window size is expressed as:

[0103]

[0104] In the formula, Indicates the size of the sliding window; The reference window size can be defined according to different regions; in this implementation, it is selected as 0.5km × 0.5km. This represents the density adjustment coefficient, which ranges from 0.5 to 1.5. In this embodiment, it is selected as 1.0.

[0105] The sliding window size is dynamically adjusted according to the building density, ensuring that areas with low building density use larger windows to include sufficient building samples, while areas with high building density use smaller windows to reduce mutual interference between buildings.

[0106] Step S43: Select multiple target regions one by one according to the preset window overlap rate and step size, wherein:

[0107] Window overlap rate Represented as:

[0108]

[0109] In the formula, Indicates the baseline overlap rate; This represents the overlap adjustment coefficient.

[0110] The window overlap rate is also dynamically adjusted according to the building density to ensure that areas with higher building density have a higher overlap rate, thereby improving the accuracy and continuity of building addition identification.

[0111] Step length Represented as:

[0112] .

[0113] When a building has added floors, its height will change significantly. The first threshold is set as the allowable height change value of the building. Once the height change value of the target building exceeds the first threshold, it indicates that the target building has added floors.

[0114] The process of setting the first threshold includes the following steps:

[0115] If the height variation values ​​of all buildings in the target area follow a normal distribution, then the first threshold is... Set as:

[0116]

[0117] In the formula, This represents the average value of all building height variations within the target area. The coefficient factor representing control sensitivity ranges from 2 to 3. A value of 2 covers approximately 95% of normal variations, making it suitable for scenarios that tolerate a small number of false alarms. A value of 3 covers approximately 99.7% of normal variations, making it suitable for scenarios with strict regulatory requirements. The standard deviation of all building height variations within the target area; This represents the regional heterogeneity adjustment factor, with a value ranging from 0.1 to 0.5. Represents the coefficient of variation. ; This represents the total number of buildings in the target area; This represents the baseline sample size, ranging from 50 to 100.

[0118] If the height variation values ​​of all buildings in the target area are severely skewed, then the first threshold... Configure it using any of the following methods:

[0119] Firstly:

[0120]

[0121] In the formula, This represents the median of all building height variations within the target area; This represents the absolute median difference of all building height variations within the target area;

[0122] Secondly:

[0123]

[0124] In the formula, This represents the 95th percentile of all building height changes within the target area. In other words, if the height change of the target building exceeds 95% of the building height changes in the sample within the area, it is considered abnormal.

[0125] Understandably, the first threshold is not a fixed value in traditional judgment techniques, but is adaptively adjusted according to the height changes of all buildings in the target area, which can significantly improve the accuracy of the judgment.

[0126] Step S5: The weighted local spatial anomaly measurement method is used to calculate the height change consistency index between the target building and other buildings in the target area. A second threshold is set. If the height change consistency index exceeds the second threshold, it is finally determined that the target building has added floors, and the determination ends; otherwise, it is finally determined that the target building does not have added floors, and the determination ends.

[0127] Illegal additions to buildings typically manifest as abrupt changes in localized heights, while the heights of surrounding buildings remain constant. To quantify this phenomenon, this invention studies the surrounding area within a certain range of the target building as a whole. It comprehensively considers the height changes of all buildings within the target building's vicinity and uses a height change consistency index to characterize the target building's height change trend compared to neighboring buildings. This consistency index is represented by a spatial anomaly metric. If the spatial anomaly metric exceeds a second threshold, it indicates that the target building's height change is inconsistent with that of neighboring buildings, and an illegal addition to the building can be determined. Conversely, if the value is below a threshold, it indicates consistency between the target building's height change and that no illegal addition has occurred. This determination method effectively eliminates height prediction errors caused by model accuracy limitations, improving the model's prediction accuracy.

[0128] The calculation process for spatial anomaly metrics is expressed as follows:

[0129]

[0130] In the formula, Indicates the target building Spatial anomaly measurement; Indicates the target building The change in altitude; Indicates buildings within the target area The change in altitude; Represents the spatial weight matrix, and its relation to the target building. and neighboring buildings The distance is inversely proportional to the distance; Indicates that the target building is included. The target area; Indicates the target area The standard deviation of the height variation of all buildings within the area.

[0131] The second threshold should be set between 2.5 and 3.5. For densely built-up areas, a second threshold of 3.0 is recommended; for sparsely built-up areas, a second threshold of 2.5 is recommended; and for special regulatory areas, a second threshold of 3.5 is recommended to reduce false alarms. The threshold selection should be based on experimental verification. The optimal threshold should be determined by analyzing the sample area. If necessary, receiver operating characteristic (ROC) curve analysis can be used to determine the optimal detection threshold for a specific area.

[0132] This application employs a two-stage assessment. The first assessment uses the height variation value of a single target building to initially screen for buildings with abnormal heights; however, this still cannot rule out changes in height caused by non-illegal additions, such as road traffic or temporary structures. The second assessment uses the height variation trend of the target building relative to other buildings in the target area to ultimately determine whether the target building has engaged in illegal additions. If the height variation trend of the target building is consistent with that of other buildings, illegal additions can be ruled out. This combination of assessments significantly improves the accuracy of identifying illegal additions and avoids misjudgments.

[0133] After the determination is completed, the system generates and outputs key information such as the precise coordinates, height of the added floors, and remote sensing image screenshots of the illegal additions, forming a list of illegal additions and a spatial distribution map, providing scientific basis and technical support for subsequent law enforcement actions.

[0134] Once the model framework of this application is established, by simply setting a reasonable data acquisition frequency to collect remote sensing images of all buildings within the target area, continuous monitoring of illegal additions can be achieved, enabling early detection and intervention of such activities. This dynamic monitoring mode significantly improves the timeliness of supervision and effectively prevents the spread of illegal construction. Furthermore, as monitoring progresses, the network parameters of the deep learning prediction model will be continuously updated and corrected, further improving the accuracy of predictions.

[0135] This embodiment also provides a system for the above-described intelligent identification method for building additions based on multimodal remote sensing data, comprising:

[0136] The observation dataset includes remote sensing images of multiple buildings from different perspectives and modalities, with the height of the corresponding building in each remote sensing image as the label;

[0137] The deep learning prediction model is trained using an observation dataset to learn the mapping relationship between remote sensing images of buildings and their heights. For any building addition identification task, a time series containing multiple consecutive time nodes is selected. At each time node, remote sensing images of the target building are collected, input into the trained deep learning prediction model, and the predicted height of the target building at the current time node is output. The first-order difference between the predicted height of the target building at the current time node and the predicted height at the previous time node is calculated as the height change value of the target building. This process is repeated for all time nodes to form a sequence of height change of the target building.

[0138] The preliminary judgment module is used to select a target area, calculate the height change sequence of all buildings in the target area, and set a first threshold based on the height change value of all buildings in the target area at any time node. If the height change value of the target building exceeds the first threshold, it is preliminarily judged that the target building has added floors, and step S5 is executed; otherwise, it is judged that the target building does not have added floors, and the judgment ends.

[0139] The final determination module is used to calculate the height change consistency index between the target building and other buildings in the target area using a weighted local spatial anomaly measurement method. A second threshold is set. If the height change consistency index exceeds the second threshold, the target building is determined to have an added floor, and the determination ends; otherwise, the target building is determined not to have an added floor, and the determination ends.

[0140] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the building addition intelligent recognition method described above.

[0141] The embodiments of the present invention have been described above. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A method for intelligent identification of building additions based on multimodal remote sensing data, characterized in that, Includes the following steps: Step S1: Construct an observation dataset, which includes remote sensing images of multiple different buildings from different perspectives and modalities, with the height of the corresponding building in each remote sensing image as the label. Step S2: Construct a deep learning prediction model and train the deep learning prediction model using the observation dataset to learn the mapping relationship between remote sensing images of buildings and their height. Step S3: For any building addition identification task, select a time series containing multiple consecutive time nodes. At each time node, collect remote sensing images of the target building, input the trained deep learning prediction model, output the height prediction value of the target building at the current time node, calculate the first difference between the height prediction value of the target building at the current time node and the height prediction value at the previous time node as the height change value of the target building, and traverse all time nodes to form the height change sequence of the target building. Step S4: Select the target area, calculate the height change sequence of all buildings in the target area according to step S3, set a first threshold based on the height change value of all buildings in the target area at any time node, if the height change value of the target building exceeds the first threshold, it is preliminarily determined that the target building has added floors, and proceed to step S5. Conversely, if no additional floors are added to the target building, the determination process ends. Step S5: The weighted local spatial anomaly measurement method is used to calculate the height change consistency index between the target building and other buildings in the target area. A second threshold is set. If the height change consistency index exceeds the second threshold, it is finally determined that the target building has added floors, and the determination ends; otherwise, it is finally determined that the target building does not have added floors, and the determination ends. The high degree of variation consistency index is represented by a spatial anomaly metric, and its calculation process is as follows: In the formula, Indicates the target building Spatial anomaly measurement; Indicates the target building The change in altitude; Indicates buildings within the target area The change in altitude; Represents the spatial weight matrix, and its relation to the target building. and neighboring buildings The distance is inversely proportional to the distance; Indicates that the target building is included. The target area; Indicates the target area The standard deviation of the height variation of all buildings within the area.

2. The intelligent identification method for building additions based on multimodal remote sensing data according to claim 1, characterized in that, Building remote sensing imagery includes optical remote sensing imagery and radar remote sensing imagery. Optical remote sensing imagery includes high-resolution optical imagery and wide-swath optical imagery.

3. The intelligent building addition identification method based on multimodal remote sensing data according to claim 1, characterized in that, Before being input into the deep learning prediction model, the observation dataset needs to be preprocessed. The preprocessing process includes geometric fine correction, orthorectification, radiometric correction and image enhancement to ensure data quality. At the same time, the sliding window technique is used to crop the remote sensing images to the required standard size, and a zero-value filling strategy is adopted for edge areas that cannot be divided evenly to ensure data integrity.

4. The intelligent building addition identification method based on multimodal remote sensing data according to claim 2, characterized in that, The deep learning prediction model includes a multimodal feature extraction module, a feature fusion module, a spatial context information enhancement module, and an output module. The multimodal feature extraction module uses long and short attention mechanisms to extract multi-scale building features from optical remote sensing images and radar remote sensing images, respectively. feature The fusion module employs an adaptive fusion attention mechanism to fuse building features of different modalities and scales to obtain fused features; The spatial context information enhancement module captures the relationship between the building and its surrounding environment, thereby enhancing the fusion features; the output module processes the fusion features and outputs the predicted height of the building.

5. The intelligent building addition identification method based on multimodal remote sensing data according to claim 4, characterized in that, Deep learning prediction model training uses fusion dice loss Boundary loss And Lovas loss Composite loss function , is represented as: In the formula, , , These represent the losses from dice rolls. Boundary loss And Lovas loss The weighting coefficients; where: Dice loss Represented as: In the formula, Represents any pixel in a remotely sensed image; This represents the total number of pixels in the remote sensing image. This indicates that the deep learning prediction model predicts the pixel... The predicted probability; Represents pixels The actual pixel value; This represents the minimum value and is used to prevent the denominator from being 0; Boundary loss Represented as: In the formula, Represents pixels SDM value; Lovas loss Represented as: In the formula, Represents a set of pixels; This represents the sorted IoU loss gradient sequence; This represents any element in the IoU loss gradient sequence.

6. The intelligent identification method for building additions based on multimodal remote sensing data according to claim 1, characterized in that, In step S4, the target area is determined through the following process: Step S41: The remote sensing image is gridded, and the building density within each grid cell is calculated. The calculation process is as follows: In the formula, Indicates the building density within a grid cell; This represents the total building area within the grid cell, calculated from the building outline data. Represents the total area of ​​the grid cells; Step S42: Within each grid cell, the target region is determined using the sliding window method. The sliding window size is expressed as: In the formula, Indicates the size of the sliding window; Indicates the reference window size; Indicates the density adjustment coefficient; Step S43: Select multiple target regions one by one according to the preset window overlap rate and step size, wherein: Window overlap rate Represented as: In the formula, Indicates the baseline overlap rate; Indicates the overlap adjustment coefficient; Step length Represented as: 。 7. The intelligent identification method for building additions based on multimodal remote sensing data according to claim 1, characterized in that, The process of setting the first threshold includes the following steps: If the height variations of all buildings in the target area follow a normal distribution, then the first threshold... Set as: In the formula, This represents the average value of all building height variations within the target area. The coefficient factor representing control sensitivity has a value range of 2-3; The standard deviation of all building height variations within the target area; This represents the regional heterogeneity adjustment factor, with a value ranging from 0.1 to 0.

5. Represents the coefficient of variation. ; This represents the total number of buildings in the target area; This represents the baseline sample size, ranging from 50 to 100. If the height variation values ​​of all buildings in the target area are severely skewed, then the first threshold... Configure it using any of the following methods: Firstly: In the formula, This represents the median of all building height variations within the target area; This represents the absolute median difference of all building height variations within the target area; Secondly: In the formula, This represents the 95th percentile of all building height changes within the target area. In other words, if the height change of the target building exceeds 95% of the building height changes in the sample within the area, it is considered abnormal.

8. The intelligent identification method for building additions based on multimodal remote sensing data according to claim 1, characterized in that, The second threshold ranges from 2.5 to 3.5, where: for densely built areas, the second threshold is 3.0; for sparsely built areas, the second threshold is 2.5; and for special regulatory areas, the second threshold is 3.

5.

9. A system for executing the intelligent building addition identification method based on multimodal remote sensing data as described in any one of claims 1-8, characterized in that, include: The observation dataset includes remote sensing images of multiple buildings from different perspectives and modalities, with the height of the corresponding building in each remote sensing image as the label; The deep learning prediction model is trained using an observation dataset to learn the mapping relationship between remote sensing images of buildings and their heights. For any building addition identification task, a time series containing multiple consecutive time nodes is selected. At each time node, remote sensing images of the target building are collected, input into the trained deep learning prediction model, and the predicted height of the target building at the current time node is output. The first-order difference between the predicted height of the target building at the current time node and the predicted height at the previous time node is calculated as the height change value of the target building. This process is repeated for all time nodes to form a sequence of height change of the target building. The preliminary judgment module is used to select a target area, calculate the height change sequence of all buildings in the target area, and set a first threshold based on the height change value of all buildings in the target area at any time node. If the height change value of the target building exceeds the first threshold, it is preliminarily determined that the target building has added floors, and step S5 is executed. Conversely, if no additional floors are added to the target building, the determination process ends. The final determination module is used to calculate the height change consistency index between the target building and other buildings in the target area using a weighted local spatial anomaly measurement method. A second threshold is set. If the height change consistency index exceeds the second threshold, the target building is determined to have an added floor, and the determination ends; otherwise, the target building is determined not to have an added floor, and the determination ends.

Images