Urban building change remote sensing detection method integrating building and shadow direction relation automatic modeling

Abstract

The invention relates to an urban building change remote sensing detection method integrating building and shadow direction relation automatic modeling, and belongs to the field of building change remote sensing detection. And designing a fuzzy landscape generator and integrating the fuzzy landscape generator into the multi-modal change detection model. The fuzzy landscape generator is used for generating a fuzzy landscape map representing direction characteristics through a neural network which takes the OfficientNetV2 as a main network through a directivity relation modeling module, and automatically modeling a directivity relation between a building and a shadow to form the fuzzy landscape map; therefore, the problems that parameters are difficult to control and metadata lacks angle information in a physical model for generating the fuzzy landscape are solved; and a multi-modal feature cross fusion module is combined with the directivity relationship between the building and the shadow, so that change features and directivity features can be effectively utilized, and redundant information can be eliminated. According to the method, the end-to-end deep neural network is integrated, the change detection efficiency is improved, and technical support is provided for acquiring urban building change information in a high-resolution image.

Description

Technical Field

[0001] The invention belongs to the field of remote sensing detection of building changes, and in particular relates to a remote sensing detection method for urban building changes that integrates automatic modeling of the relationship between buildings and shadow directions. Background Art

[0002] Buildings in urban areas are basic places for living, business, work and residence in urban environments. The rapid urbanization process has brought about rapid changes in urban surfaces, among which changes in buildings (demolition, new construction) are particularly significant. Accurately identifying information on building changes in urban areas is of great significance for improving the scientificity and effectiveness of urban management, planning and disaster response. It also provides important data support for applications such as land resource management, urban environmental monitoring and urban sustainable development strategy formulation.

[0003] The continuous development of remote sensing technology has improved the spatial resolution of images, making high-resolution remote sensing images have great potential in fine-grained identification of building changes. However, there are still many challenges in the current urban building change detection methods. Affected by different phases, high-resolution remote sensing images of the same location often lead to obvious false changes due to the influence of different image perspectives. Therefore, based on high-resolution remote sensing images, how to solve the pseudo-changes caused by shooting angles at different phases and accurately identify building change information in urban areas has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The purpose of the present invention is to provide a remote sensing detection method for urban building changes that integrates automatic modeling of the directional relationship between buildings and shadows. By automatically modeling the directional characteristics of buildings and shadows, the problem of pseudo-changes that occur in traditional twin neural networks in remote sensing image change detection is solved. The method of the present invention not only improves the efficiency of fuzzy landscape map generation through an automatic modeling module, but also solves the problem of dependence of the physical model generation process on solar altitude angle information. In addition, the end-to-end model architecture effectively improves the efficiency of change detection, providing technical support for efficiently acquiring detailed urban building change information in high-resolution images.

[0005] To achieve the above object, the technical solution of the present invention is: a remote sensing detection method for urban building changes that integrates automatic modeling of the relationship between buildings and shadow directions, comprising the following steps:

[0006] Step S1, obtaining dual-phase high-resolution remote sensing images of the study area;

[0007] Step S2, producing a fuzzy landscape image representing the directional relationship between the building and the shadow as multimodal data representing the directional relationship between the building and the shadow;

[0008] Step S3, designing a fuzzy landscape generator to automatically model the directional relationship between buildings and shadows, the fuzzy landscape generator includes a building and shadow directional relationship modeling module and a multimodal feature fusion module; constructing a building and shadow directional relationship modeling module based on EfficientNetV2 as the backbone, the module automatically models the directional relationship between buildings and shadows, and generates a fuzzy landscape map representing the directional relationship; then using a directional feature extractor to extract high-level directional features to enhance the ability to distinguish between real changes and false changes;

[0009] Step S4, integrating the fuzzy landscape generator into the twin multi-task change detection model, the model consists of three parts: a change feature encoder, a building and shadow directional relationship modeling module and a multimodal feature fusion module; using PVT-V2 as a change feature encoder to extract image features, learn the change information between dual-phase high-resolution remote sensing images, and use boundary information as an auxiliary task to constrain the learning of high-level semantic features;

[0010] Step S5, through the multimodal feature fusion module, the high-level directional features extracted in step S3 are fused to construct a dual-phase building binary change map and a change boundary decoder to make full use of the change features and directional features and effectively eliminate redundant information;

[0011] Step S6, designing special loss functions for different tasks to optimize model training; for the change detection task, using an improved version of the Focal Loss function based on the cross entropy loss function to improve the imbalance problem between changed samples and unchanged samples; using the negative log-likelihood NLL loss function for the boundary detection task; using the mean square error MSE loss function for the fuzzy landscape task; finally, by aggregating the loss values ​​of each task, forming a total loss function for overall optimization of the model;

[0012] Step S7, model training and prediction, finally obtaining a building change area map, a change boundary map and a fuzzy landscape map of the dual-temporal high-resolution remote sensing image.

[0013] In one embodiment of the present invention, in step S1, a preprocessing operation is performed on the acquired dual-phase high-resolution remote sensing image of the study area, including radiation correction, orthorectification, and image fusion.

[0014] In one embodiment of the present invention, step S2 specifically includes the following steps:

[0015] Step S21, firstly, shadow extraction is performed, and the shadow area of ​​the building is automatically extracted from the high-resolution remote sensing image by using the method based on the Otsu threshold; specifically, the high-resolution remote sensing image is converted from the RGB space to the HIS space, and the Otsu threshold segmentation is performed based on the ratio of saturation to brightness, so as to accurately extract the shadow area of ​​the building;

[0016] Step S22: In order to eliminate the interference of vegetation on shadow extraction, the normalized difference vegetation index NDVI is used to remove the vegetation area in the shadow area; by calculating the NDVI value, the vegetation and non-vegetation areas are effectively distinguished, and the vegetation area is removed before shadow extraction to ensure that the extracted shadow information is not affected by vegetation;

[0017] Step S23: The direction angle β derived based on the sun azimuth in the metadata and the given shadow object R S , define the membership function to quantify the image pixel x satisfies R s The degree of directional relationship; the angle θ needs to be considered when calculating the membership function value β (x,b) and the distance d(x,b), where b∈R s ; Angle θ β (x,b) refers to the angle between the vector from x to b and the unit vector along the horizontal direction, and the distance d(x,b) is the Euclidean distance between x and b;

[0018] Step S24: The calculation of the fuzzy structure element v consists of two parts: the angle-dependent v α (x) and distance-dependent v d (x), v α (x) through the angle θ α (x,o) is used to quantify, where o is the reference point, ensuring that the closer the angle is, the larger the value is; and v d (x) is quantized by the distance d(x,o), ensuring that the closer the distance, the larger the value; v α (x) and v d (x) are multiplied to obtain the final fuzzy structure element v; the specific calculation is as follows:

[0019] v=v α (x)×v d (x)

[0020]

[0021] In order to improve the computational efficiency, R s The boundary pixel δR s Perform dilation operation; the following formula represents the fuzzy landscape L β (x,R s ) calculation process:

[0022]

[0023] in, is the expansion operator, YesR s The complement of , v(x) is the fuzzy structure element.

[0024] In one embodiment of the present invention, step S3 specifically includes the following steps:

[0025] Step S31: The encoder part of the building and shadow directional relationship modeling module uses EfficientNetV2 as the backbone for feature extraction. EfficientNetV2 introduces the Fuse-MBConv block to reduce the computational complexity. In the decoder part, multiple deconvolution layers are used to gradually upsample the resolution, enhance the regional detail performance, restore the building and shadow directional information, and generate more detailed and accurate results. In addition, the channel attention mechanism SE is added to the building and shadow directional relationship modeling module to give different weights to different channels to better capture the directionality and distance correlation information.

[0026] Step S32, design a directional feature extractor, which is used to extract high-level directional features related to buildings and their shadows from the blurred landscape image obtained in step S2 that represents the directional relationship between buildings and shadows; the encoder part of the directional feature extractor includes two downsampling modules, each of which performs two 3×3 convolution operations and one 2×2 maximum pooling operation; the decoder part also includes two upsampling modules, which perform four 3×3 convolution operations, two 2×2 deconvolution operations, and perform final feature integration through a 1×1 convolution; the directional feature extractor outputs fine high-level directional features to enhance the model's ability to capture directional information.

[0027] In one embodiment of the present invention, step S4 specifically includes the following steps:

[0028] Step S41, the change feature encoder is composed of two PVT-V2 pre-trained based on the ImageNet dataset, which is used for change feature extraction;

[0029] Step S42: After extracting the change features of different levels from the dual-phase high-resolution remote sensing image using the change feature encoder, the features of the two phases are differentially aligned and fused at each scale through a multi-scale feature fusion strategy; while maintaining the feature information of different resolutions, the change area is highlighted and the interference of the non-change area is suppressed, so as to effectively capture the change details of the objects at each scale;

[0030] Step S43: introduce the fuzzy landscape generator and directional feature extractor proposed in step S3 to generate fuzzy landscape images of two time phases and their high-level directional features respectively.

[0031] In one embodiment of the present invention, step S5 specifically includes the following steps:

[0032] Step S51, obtaining change feature information and directional feature information through step S42 and step S43; since the difference between the two modal features is large, and the directional feature has a unique advantage in capturing the geometric relationship between the building and the shadow, a multimodal feature fusion module is introduced to perform feature integration; in the change detection process, the complementarity between the directional feature and the change feature is fully utilized;

[0033] Step S52, the multimodal feature fusion module includes two parts: feature fusion and feature enhancement; the feature fusion part includes two cross-attention layers: the first cross-attention layer uses the change feature as the key and value, the directional feature as the query, and calculates the attention of the directional feature to the change feature; the second cross-attention layer uses the directional feature as the key and value, the change feature as the query, and calculates the attention of the change feature to the directional feature; the results of the two are spliced ​​together to form a new difference feature; then, the difference feature is further processed using the self-attention mechanism to enhance its semantic representation;

[0034] Step S53, design a change area decoder and a change boundary decoder, the change area decoder is used to reconstruct the details of the change area, and the change boundary decoder focuses on restoring the change boundary to ensure that the contour of the change area is clear, thereby improving the overall performance of the model in the change detection task; the enhanced difference features obtained in step S52 are input into the change area decoder and the change boundary decoder respectively.

[0035] In one embodiment of the present invention, step S6 specifically includes the following steps:

[0036] Step S61: Based on the twin multi-task change detection model constructed in step S4, a multi-task loss function is designed to update the network weights and reduce the error between the actual and predicted results; specifically, the FocalLoss function is used for the change detection task to deal with the class imbalance problem; for the boundary detection task, the negative log-likelihood loss is used; for the fuzzy landscape task, the mean square error MSE loss is used;

[0037] Step S62: define the Focal loss function as follows:

[0038] L mask = -α(1-ρ t ) λ log(ρ t )

[0039] In the formula, α controls the weight of the share of positive and negative samples in the total loss; λ refers to an adjustment factor that reduces the weight of easily classified instances so that the model focuses on difficult-to-classify instances; ρ t Represents the predicted probability value, which represents the predicted probability of the model on the current category (i.e. positive sample or negative sample);

[0040] Step S63, define the negative log-likelihood loss as follows:

[0041]

[0042] Among them, L boundary represents the pixel classification error, x represents a pixel position in the image space, ω represents the pixel set in the image, logp boundary (x; L boundary (x)) represents the logarithm of the predicted probability of the true label;

[0043] Step S64, define the MSE loss as follows:

[0044]

[0045] L shadow The loss obtained for the blurred landscape of the two phases, represents the predicted value of the model, D(x) represents the true value; N represents the total number of samples;

[0046] Step S65: Based on the losses of the change detection task, the boundary detection task and the fuzzy landscape task, a joint loss is obtained.

[0047] L Siam multask =w1L mask +w2L boundary +w3L shadow

[0048] Where: L Siam multask represents the total loss of the twin multi-task change detection model, and w1, w2, and w3 represent the weights corresponding to different losses.

[0049] In one embodiment of the present invention, step S7 specifically includes the following steps:

[0050] Step S71: After the parameter setting is completed, the twin multi-task change detection model is trained and the optimal model weights are saved;

[0051] Step S72: Use the weights obtained in step S71 to perform change detection on the dual-phase high-resolution remote sensing images of the study area, and stitch the detection results to complete the detection of the entire area.

[0052] The present invention also provides a remote sensing detection system for urban building changes that integrates automatic modeling of the relationship between buildings and shadow directions, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0053] The present invention also provides a computer-readable storage medium, on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] (1) The present invention automatically models the directional relationship between buildings and shadows through deep learning, thus overcoming the problems of errors caused by artificially controlling physical parameters and lack of image modeling parameters in metadata in existing methods for generating fuzzy landscapes.

[0056] (2) The present invention integrates the fuzzy landscape generator into the change detection model and combines the directional characteristics of buildings and shadows to solve the problem of false changes caused by the dual-phase perspective in the change detection method, thereby improving the accuracy of urban building change detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of a method flow chart of a preferred embodiment of the present invention;

[0058] Figure 2 Generate fuzzy landscape flow charts for physical models;

[0059] Figure 3 This is a model framework diagram of the present invention;

[0060] Figure 4 This is a structural diagram of the multimodal feature fusion module of the present invention;

[0061] Figure 5 This is a diagram of the change detection results of the study area in the preferred embodiment of the present invention; from left to right, they are T1, T2, label, non-integrated fuzzy landscape change detection model, and integrated fuzzy landscape change detection model. DETAILED DESCRIPTION

[0062] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0063] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0064] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0065] The present invention provides a remote sensing detection method for urban building changes that integrates automatic modeling of building and shadow direction relationships, comprising the following steps:

[0066] Step S1, obtaining dual-phase high-resolution remote sensing images of the study area;

[0067] Step S2, producing a fuzzy landscape image representing the directional relationship between the building and the shadow as multimodal data representing the directional relationship between the building and the shadow;

[0068] Step S3, designing a fuzzy landscape generator to automatically model the directional relationship between buildings and shadows, the fuzzy landscape generator includes a building and shadow directional relationship modeling module and a multimodal feature fusion module; constructing a building and shadow directional relationship modeling module based on EfficientNetV2 as the backbone, the module automatically models the directional relationship between buildings and shadows, and generates a fuzzy landscape map representing the directional relationship; then using a directional feature extractor to extract high-level directional features to enhance the ability to distinguish between real changes and false changes;

[0069] Step S4, integrating the self-building and shadow directional relationship modeling module into the twin multi-task change detection model, the model consists of three parts: a change feature encoder, a building and shadow directional relationship modeling module and a multimodal feature fusion module; using PVT-V2 as a change feature encoder to extract image features, learning the change information between dual-phase high-resolution remote sensing images, and using boundary information as an auxiliary task to constrain the learning of high-level semantic features;

[0070] Step S5, through the multimodal feature fusion module, the high-level directional features extracted in step S3 are fused to construct a dual-phase building binary change map and a change boundary decoder to make full use of the change features and directional features and effectively eliminate redundant information;

[0071] Step S6, designing special loss functions for different tasks to optimize model training; for the change detection task, using an improved version of the Focal Loss function based on the cross entropy loss function to improve the imbalance problem between changed samples and unchanged samples; using the negative log-likelihood NLL loss function for the boundary detection task; using the mean square error MSE loss function for the fuzzy landscape task; finally, by aggregating the loss values ​​of each task, forming a total loss function for overall optimization of the model;

[0072] Step S7, model training and prediction, finally obtaining a building change area map, a change boundary map and a fuzzy landscape map of the dual-temporal high-resolution remote sensing image.

[0073] The following is the specific implementation process of the present invention.

[0074] like Figure 1 As shown, this embodiment provides a method for monitoring changes in urban buildings based on automatic modeling of building and shadow directional characteristics by twin networks, comprising the following steps:

[0075] Step S1: Obtain dual-phase high-resolution remote sensing images of the study area and perform preprocessing operations on the images, including radiation correction, orthorectification, image fusion, etc.;

[0076] Step S2: Create a fuzzy landscape image representing the directional relationship between the building and the shadow as multimodal data representing the directional relationship between the building and the shadow. Figure 2 To generate a fuzzy landscape flow chart.

[0077] Step S3: Design a fuzzy landscape generator to automatically model the directional relationship between buildings and shadows. The fuzzy landscape generator includes a building and shadow directional relationship modeling module and a multimodal feature fusion module. Construct a building and shadow directional relationship modeling module based on EfficientNetV2 as the backbone. This module automatically models the directional relationship between buildings and shadows and generates a fuzzy landscape map representing their directional relationship. Then use a directional feature extractor to extract high-level directional features to enhance the ability to distinguish between real changes and false changes.

[0078] Step S4: Integrate the fuzzy landscape generator into the twin multi-task change detection model. The model structure is as follows: Figure 3 As shown in the figure, it consists of three parts: change feature encoder, building and shadow directional relationship modeling module and multimodal feature fusion module. PVT-V2 is used as a change feature encoder to extract image features and learn the change information between dual-phase high-resolution remote sensing images. Boundary information is used as an auxiliary task to constrain the learning of high-level semantic features.

[0079] Step S5: Through the multimodal feature fusion module (see Figure 4), fuse the high-level directional features extracted in step S3, and construct a dual-phase building binary change map and change boundary decoder. This step makes full use of the change features and directional features to effectively eliminate redundant information.

[0080] Step S6: Design special loss functions for different tasks to optimize model training. For the change detection task, an improved version of the Focal Loss function based on the cross entropy loss function is used to improve the imbalance between changed samples and unchanged samples. The boundary detection task uses the NLL (negative log likelihood) loss function, while the blurred landscape task uses the MSE (mean square error) loss function. Finally, by aggregating the loss values ​​of each task, a total loss function is formed for the overall optimization of the model.

[0081] Step S7: Model training and prediction, finally obtaining the building change area map, change boundary map and blurred landscape map of the dual-temporal high-resolution remote sensing image.

[0082] In a preferred embodiment, step S2 specifically includes the following steps:

[0083] Step S21: First, shadow extraction is performed, and the shadow of the building is automatically extracted from the high-resolution remote sensing image using the Otsu threshold method. The Otsu threshold method is a technique that automatically determines the image segmentation threshold by maximizing the inter-class variance, effectively distinguishing the foreground from the background. In the specific operation, we convert the image from the RGB space to the HIS space, and perform Otsu threshold segmentation based on the ratio of saturation to brightness, so as to accurately extract the shadow area of ​​the building.

[0084] Step S22: In order to eliminate the interference of vegetation on shadow extraction, the normalized difference vegetation index (NDVI) is used to remove the vegetation area in the shadow. By calculating the NDVI value, the vegetation and non-vegetation areas can be effectively distinguished. After the vegetation area is removed, the shadow extraction is performed to ensure that the extracted shadow information is not affected by vegetation.

[0085] Step S23: The direction angle β derived from the sun azimuth in the metadata and the given shadow object R S , define the membership function to quantify the image pixel x satisfies R s The degree of directional relationship. The angle θ needs to be considered when calculating the membership function value β (x,b) and the distance d(x,b), where b∈R s Angle θ β (x,b) refers to the angle between the vector from x to b and the unit vector along the horizontal direction, and the distance d(x,b) is the Euclidean distance between x and b.

[0086] Step S24: The calculation of the fuzzy structure element v consists of two parts: the angle-dependent v α (x) and distance-dependent v d (x), v α (x) through the angle θ α (x,o) is used to quantify, where o is the reference point, ensuring that the closer the angle is, the larger the value is; and v d (x) is quantized by the distance d(x,o), ensuring that the closer the distance, the larger the value. The two values ​​are multiplied to obtain the final fuzzy structure element v. The specific calculation is as follows:

[0087] v=v α (x)×v d (x)

[0088]

[0089] In order to improve the computational efficiency, R s The boundary pixel δR s Perform expansion operation. Formula

[0090] Represents the fuzzy landscape L β (x,R s ) calculation process:

[0091]

[0092] in, is the expansion operator, YesR s The complement of . v is the fuzzy structure element.

[0093] In a preferred embodiment, step S3 specifically includes the following steps:

[0094] Step S31: The encoder part of the building and shadow directional relationship modeling module uses EfficientNetV2 as the backbone network for feature extraction. EfficientNetV2 introduces the Fuse-MBConv block to reduce computational complexity. In the decoder part, multiple deconvolution layers are used to gradually upsample the resolution, enhance the regional detail performance, restore the building and shadow directional information, and generate more detailed and accurate results. In addition, a channel attention mechanism (SE) is added to the module to assign different weights to different channels to better capture the directional and distance correlation information.

[0095] Step S32: Design a directional feature extractor to extract high-level directional features related to buildings and their shadows from the blurred landscape image of the dual-phase high-resolution remote sensing image obtained in step S2. The encoder part of the directional feature extractor contains two downsampling modules, each of which performs two 3×3 convolution operations and one 2×2 maximum pooling operation. The decoder part also contains two upsampling modules, which perform four 3×3 convolution operations, two 2×2 deconvolution operations, and perform final feature integration through a 1×1 convolution. Through these operations, the directional relationship feature enhancer outputs refined high-level directional features to enhance the model's ability to capture directional information.

[0096] In a preferred embodiment, step S4 specifically includes the following steps:

[0097] Step S41: The change feature encoder is composed of two PVT-V2 pre-trained on the ImageNet dataset for change feature extraction. PVT-V2 can obtain more local information of the image and change information feature map, and can flexibly process inputs of different resolutions and adapt to changes of various sizes and scales. In addition, PVT-V2 has a low computational cost and is suitable for high-resolution remote sensing images.

[0098] Step S42: After extracting the different levels of change features from the dual-phase images using the change feature encoder, the features of the two phases are differentiated and fused at each scale through a multi-scale feature fusion strategy. While maintaining feature information of different resolutions, the change area is highlighted and the interference of the non-change area is suppressed, thereby effectively capturing the details of the changes of the objects at each scale.

[0099] Step S43: introducing the fuzzy landscape generator and high-level directional feature extractor proposed in step S3 to generate fuzzy landscape images of two time phases and their high-level directional features respectively.

[0100] In a preferred embodiment, step S5 specifically includes the following steps:

[0101] Step S51: After step S42 and step S43, change feature information and directional feature information are obtained. Since the difference between the two modal features is large, and the directional feature has a unique advantage in capturing the geometric relationship between buildings and shadows, a multimodal feature fusion module is introduced to integrate the features. In the change detection process, the complementarity between directional information and change features is fully utilized.

[0102] Step S52: The multimodal feature fusion module includes two parts: feature fusion and feature enhancement. The feature fusion part contains two cross-attention layers: the first cross-attention layer uses the change feature as the key and value, the direction feature as the query, and calculates the attention of the direction feature to the change feature; the second cross-attention layer uses the direction feature as the key and value, the change feature as the query, and calculates the attention of the change feature to the direction feature. The results of the two are spliced ​​together to form a new difference feature representation. Subsequently, the self-attention mechanism is used to further process the feature to enhance its semantic representation.

[0103] Step S53: Design a change region decoder and a change boundary decoder. The change region decoder is used to reconstruct the details of the change region, while the change boundary decoder focuses on restoring the change boundary to ensure that the contour of the change region is clear, thereby improving the overall performance of the model in the change detection task. The enhanced difference features obtained in step S52 are input into the two decoders respectively.

[0104] In a preferred embodiment, step S6 specifically includes the following steps:

[0105] Step S61: Based on the network model constructed in step S4, a multi-task loss function is designed to update the network weights and reduce the error between the actual and predicted results. Specifically, the Focal Loss function is used for the changing region task to deal with the class imbalance problem; the negative log-likelihood loss is used for the boundary detection task; and the mean square error MSE loss is used for the fuzzy landscape task.

[0106] Step S62: The Focal loss function is defined as follows:

[0107] L mask = -α(1-ρ t ) λ log(ρ t )

[0108] Where α controls the weight of the share of positive and negative samples in the total loss. λ refers to a regulation factor that makes the model focus on difficult-to-classify instances by reducing the weight of easily classified instances. ρ t It represents the predicted probability value, which represents the predicted probability of the model on the current category (i.e., positive sample or negative sample). In our study, α is set to 0.25% and λ is set to 2.

[0109] Step S63: The negative log-likelihood loss is defined as follows:

[0110]

[0111] Among them, L boundaryrepresents the pixel classification error, x represents a pixel position in the image space, ω represents the pixel set in the image, logp boundary (x; L boundary (x)) represents the logarithm of the predicted probability of the true label;

[0112] Step S64: The MSE loss is defined as follows:

[0113]

[0114] represents the predicted value of the model, D(x) represents the true value. N represents the total number of samples. The final L shadow The loss obtained for the blurred landscape of the two phases.

[0115] Step S65: Based on the losses of the change detection task, boundary detection task, and blurred landscape task, we get the joint loss

[0116] L Siam multask =w1L mask +w2L boundary +w3L shadow

[0117] Where: L Siam multask represents the total loss of the network, w1, w2, and w3 represent the weights corresponding to different losses.

[0118] In a preferred embodiment, step S7 specifically includes the following steps:

[0119] Step S71: After the parameter setting is completed, train the twin multi-task neural network change detection model and save the optimal model weights.

[0120] Step S72: Use the weight file obtained in step S71 to perform change detection on the change detection samples, and splice the detection results to complete the detection of the entire area.

[0121] In this example, the main urban area of ​​City X is used as the research object, and high-resolution remote sensing image data of Pleiades-1B in the study area in 2018 and 2019 are used to detect building changes in key disturbance areas in the study area, using red, green and blue bands. This example uses 1600 pairs of images with a pixel size of 256×256 and corresponding annotation data for model training.

[0122] like Figure 5FIG. 4 is a diagram showing some experimental results of this embodiment on the Pleiades-1B image of City X. It can be seen from the figure that the change area obtained by the proposed method is highly consistent with the ground truth label; secondly, the pseudo error of buildings and shadows caused by different sun altitude angles is eliminated; the experimental results show the effectiveness of the proposed method for urban building change detection.

[0123] The present invention also provides a remote sensing detection system for urban building changes that integrates automatic modeling of the relationship between buildings and shadow directions, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0124] The present invention also provides a computer-readable storage medium, on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0125] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0126] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0129] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A remote sensing detection method for urban building changes that integrates automatic modeling of the relationship between buildings and shadow directions, characterized in that: The following steps are involved: Step S1, obtaining dual-phase high-resolution remote sensing images of the study area; Step S2, producing a fuzzy landscape image representing the directional relationship between the building and the shadow as multimodal data representing the directional relationship between the building and the shadow; Step S3, designing a fuzzy landscape generator to automatically model the directional relationship between buildings and shadows, the fuzzy landscape generator comprising a building and shadow directional relationship modeling module and a multimodal feature fusion module; A building-shadow directional relationship modeling module based on EfficientNetV2 is constructed. This module automatically models the directional relationship between buildings and shadows to generate a fuzzy landscape map representing the directional relationship. A directional feature extractor is then used to extract high-level directional features to enhance the ability to distinguish between real and false changes. Step S4, integrating the fuzzy landscape generator into the twin multi-task change detection model, the model consists of three parts: a change feature encoder, a building and shadow directional relationship modeling module and a multimodal feature fusion module; using PVT-V2 as a change feature encoder to extract image features, learn the change information between dual-phase high-resolution remote sensing images, and use boundary information as an auxiliary task to constrain the learning of high-level semantic features; Step S5, through the multimodal feature fusion module, the high-level directional features extracted in step S3 are fused to construct a dual-phase building binary change map and a change boundary decoder to make full use of the change features and directional features and effectively eliminate redundant information; Step S6, designing special loss functions for different tasks to optimize model training; for the change detection task, using an improved version of the Focal Loss function based on the cross entropy loss function to improve the imbalance problem between changed samples and unchanged samples; using the negative log-likelihood NLL loss function for the boundary detection task; using the mean square error MSE loss function for the fuzzy landscape task; finally, by aggregating the loss values ​​of each task, forming a total loss function for overall optimization of the model; Step S7, model training and prediction, finally obtaining a building change area map, a change boundary map and a fuzzy landscape map of the dual-temporal high-resolution remote sensing image.

2. The method for remote sensing detection of urban building changes with integrated automatic modeling of building and shadow direction relationship according to claim 1 is characterized in that: In step S1, the acquired dual-phase high-resolution remote sensing images of the study area are preprocessed, including radiation correction, orthorectification, and image fusion.

3. The remote sensing detection method for urban building changes integrating automatic modeling of building and shadow direction relationship according to claim 1 is characterized in that: Step S2 specifically includes the following steps: Step S21, firstly, shadow extraction is performed, and the shadow area of ​​the building is automatically extracted from the high-resolution remote sensing image by using the method based on the Otsu threshold; specifically, the high-resolution remote sensing image is converted from the RGB space to the HIS space, and the Otsu threshold segmentation is performed based on the ratio of saturation to brightness, so as to accurately extract the shadow area of ​​the building; Step S22: In order to eliminate the interference of vegetation on shadow extraction, the normalized difference vegetation index NDVI is used to remove the vegetation area in the shadow area; by calculating the NDVI value, the vegetation and non-vegetation areas are effectively distinguished, and the vegetation area is removed before shadow extraction to ensure that the extracted shadow information is not affected by vegetation; Step S23: The direction angle β derived based on the sun azimuth in the metadata and the given shadow object R S , define the membership function to quantify the image pixel x satisfies R s The degree of directional relationship; the angle θ needs to be considered when calculating the membership function value β (x,b) and the distance d(x,b), where b∈R s ; Angle θ β (x,b) refers to the angle between the vector from x to b and the unit vector along the horizontal direction, and the distance d(x,b) is the Euclidean distance between x and b; Step S24: The calculation of the fuzzy structure element v consists of two parts: the angle-dependent v α (x) and distance-dependent v d (x), v α (x) through the angle θ α (x,o) is used to quantize, where o is the reference point; the closer the angle is, the larger the value is; and v d (x) is quantized by the distance d(x,o), ensuring that the closer the distance, the larger the value; v α (x) and v d (x) are multiplied to obtain the final fuzzy structure element v; the specific calculation is as follows: v=v α (x)×v d (x) In order to improve the computational efficiency, R s The boundary pixel δR s Perform dilation operation; the following formula represents the fuzzy landscape L β (x,R s ) calculation process: in, is the expansion operator, YesR s The complement of , v(x) is the fuzzy structure element.

4. The method for remote sensing detection of urban building changes integrating automatic modeling of building and shadow direction relationship according to claim 1 is characterized in that: Step S3 specifically includes the following steps: Step S31: The encoder part of the building and shadow directional relationship modeling module uses EfficientNetV2 as the backbone for feature extraction. EfficientNetV2 introduces the Fuse-MBConv block to reduce the computational complexity. In the decoder part, multiple deconvolution layers are used to gradually upsample the resolution, enhance the regional detail performance, restore the building and shadow directional information, and generate more detailed and accurate results. In addition, the channel attention mechanism SE is added to the building and shadow directional relationship modeling module to give different weights to different channels to better capture the directionality and distance correlation information. Step S32, design a directional feature extractor, which is used to extract high-level directional features related to buildings and their shadows from the blurred landscape image obtained in step S2 that represents the directional relationship between buildings and shadows; the encoder part of the directional feature extractor includes two downsampling modules, each of which performs two 3×3 convolution operations and one 2×2 maximum pooling operation; the decoder part also includes two upsampling modules, which perform four 3×3 convolution operations, two 2×2 deconvolution operations, and perform final feature integration through a 1×1 convolution; the directional feature extractor outputs fine high-level directional features to enhance the model's ability to capture directional information.

5. The method for remote sensing detection of urban building changes integrating automatic modeling of building and shadow direction relationship according to claim 1 is characterized in that: Step S4 specifically includes the following steps: Step S41, the change feature encoder is composed of two PVT-V2 pre-trained based on the ImageNet dataset, which is used for change feature extraction; Step S42: After extracting the change features of different levels from the dual-phase high-resolution remote sensing image using the change feature encoder, the features of the two phases are differentially aligned and fused at each scale through a multi-scale feature fusion strategy; while maintaining the feature information of different resolutions, the change area is highlighted and the interference of the non-change area is suppressed, so as to effectively capture the change details of the objects at each scale; Step S43: introduce the fuzzy landscape generator and directional feature extractor proposed in step S3 to generate fuzzy landscape images of two time phases and their high-level directional features respectively.

6. The method for remote sensing detection of urban building changes integrating automatic modeling of building and shadow direction relationship according to claim 5 is characterized in that: Step S5 specifically includes the following steps: Step S51, obtaining change feature information and directional feature information through step S42 and step S43; since the difference between the two modal features is large, and the directional feature has a unique advantage in capturing the geometric relationship between the building and the shadow, a multimodal feature fusion module is introduced to perform feature integration; in the change detection process, the complementarity between the directional feature and the change feature is fully utilized; Step S52, the multimodal feature fusion module includes two parts: feature fusion and feature enhancement; the feature fusion part includes two cross-attention layers: the first cross-attention layer uses the change feature as the key and value, the directional feature as the query, and calculates the attention of the directional feature to the change feature; the second cross-attention layer uses the directional feature as the key and value, the change feature as the query, and calculates the attention of the change feature to the directional feature; the results of the two are spliced ​​together to form a new difference feature; then, the difference feature is further processed using the self-attention mechanism to enhance its semantic representation; Step S53, design a change area decoder and a change boundary decoder, the change area decoder is used to reconstruct the details of the change area, and the change boundary decoder focuses on restoring the change boundary to ensure that the contour of the change area is clear, thereby improving the overall performance of the model in the change detection task; the enhanced difference features obtained in step S52 are input into the change area decoder and the change boundary decoder respectively.

7. The method for detecting urban building changes by remote sensing with integrated automatic modeling of building and shadow direction relationship according to claim 1, characterized in that: Step S6 specifically includes the following steps: Step S61: Based on the twin multi-task change detection model constructed in step S4, a multi-task loss function is designed to update the network weights and reduce the error between the actual and predicted results; specifically, the Focal Loss function is used for the change detection task to deal with the class imbalance problem; for the boundary detection task, the negative log-likelihood loss is used; for the fuzzy landscape task, the mean square error MSE loss is used; Step S62: define the Focal loss function as follows: L mask =-α(1-ρ t ) λ log(ρ t ) In the formula, α controls the weight of the share of positive samples and negative samples in the total loss; λ refers to an adjustment factor that reduces the weight of easily classified instances so that the model focuses on difficult-to-classify instances; ρt represents the predicted probability value, which represents the predicted probability of the model on the current category, that is, positive samples or negative samples; Step S63, define the negative log-likelihood loss as follows: Among them, L boundary represents the pixel classification error, x represents a pixel position in the image space, ω represents the pixel set in the image, log p boundary (x; L boundary (x)) represents the logarithm of the predicted probability of the true label; Step S64, define the MSE loss as follows: L shadow the loss gained for the blurred landscape of the two phases; represents the predicted value of the model, D(x) represents the true value; N represents the total number of samples; Step S65: Based on the losses of the change detection task, the boundary detection task and the fuzzy landscape task, a joint loss is obtained. L Siam mul task =w1L mask +w2L boundary +w3L shadow Where: L Siammultask represents the total loss of the twin multi-task change detection model, and w1, w2, and w3 represent the weights corresponding to different losses.

8. The method for detecting urban building changes by remote sensing with integrated automatic modeling of the relationship between building and shadow direction according to claim 1, characterized in that: Step S7 specifically includes the following steps: Step S71: After the parameter setting is completed, the twin multi-task change detection model is trained and the optimal model weights are saved; Step S72: Use the weights obtained in step S71 to perform change detection on the dual-phase high-resolution remote sensing images of the study area, and stitch the detection results to complete the detection of the entire area.

9. A remote sensing detection system for urban building changes that integrates automatic modeling of the relationship between buildings and shadow directions, characterized in that: The method comprises a memory, a processor and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps as claimed in any one of claims 1 to 8 can be implemented.

10. A computer-readable storage medium having stored thereon computer program instructions that can be executed by a processor, and when the processor executes the computer program instructions, the method steps according to any one of claims 1 to 8 can be implemented.

Images