A poverty space extraction method based on edge multi-scale attention network

Abstract

The application discloses a poverty space extraction method based on an edge multi-scale attention network, which comprises the following steps: 1, obtaining a high-resolution visible light remote sensing image and performing normalization pretreatment; 2, constructing an edge multi-scale attention network EMANet; 3, introducing a multi-scale jump refinement module at the encoder end; 4, introducing an edge fusion attention module at the decoder end; and 5, completing model training. The application has the following beneficial effects: the EMANet constructed in the application can fully utilize global semantic information and local edge information, and can realize high-precision automatic extraction of a poverty space region only by using a single high-resolution visible light remote sensing image, thereby reducing the dependence on multi-source heterogeneous data and having a good application prospect.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image interpretation and urban spatial information extraction technology, specifically to a method for impoverished spatial extraction based on deep learning and high-resolution remote sensing images. Background Technology

[0002] Urban poverty spaces are characterized by dense building density, irregular layout, and complex spatial structures. As one of the world's most concentrated areas of poverty, the Greater Mumbai region makes accurate acquisition of its spatial information crucial for urban planning, infrastructure development, population management, and the allocation of public service resources. In recent years, with the continuous improvement of the spatial resolution and acquisition capabilities of high-resolution visible light remote sensing imagery, the identification and extraction of poverty spaces using high-resolution remote sensing imagery has become an important research direction in urban remote sensing monitoring.

[0003] Existing methods for extracting impoverished spaces mainly include manual visual interpretation, traditional pixel-based machine learning methods, and remote sensing image semantic segmentation methods based on deep learning. Among these, manual visual interpretation is labor-intensive and inefficient, making it difficult to meet the needs of rapid monitoring over large areas. Traditional machine learning methods, such as support vector machines and random forests, rely primarily on manually designed spectral, textural, and geometric features, which have limited ability to represent features in complex scenes. Furthermore, impoverished spaces often feature complex roof materials, small building spacing, and are interspersed with bare land and vegetation, easily leading to "same object, different spectrum" and "different object, same spectrum" phenomena, significantly impacting extraction accuracy.

[0004] In recent years, Convolutional Neural Networks (CNNs) and their encoder-decoder structures have been widely applied to semantic segmentation of high-resolution remote sensing images, achieving good extraction results. However, existing deep learning methods still have certain shortcomings in complex, high-density urban scenes. On the one hand, to improve extraction accuracy, some methods need to fuse multi-source remote sensing data such as multispectral images, synthetic aperture radar (SAR) images, and lidar (LiDAR) images, which not only increases data acquisition costs but also causes problems such as complex registration and large preprocessing workload among multi-source data. On the other hand, when using only a single high-resolution visible light remote sensing image as input, existing networks are limited by insufficient multi-scale feature representation capabilities and inadequate utilization of boundary information. In densely built-up areas, problems such as blurred target edges, overlapping of adjacent buildings, omission of small-scale targets, and poor integrity of segmentation results easily occur, affecting the extraction accuracy of impoverished spaces and the generalization ability of the model.

[0005] Therefore, how to improve the boundary identification capability and target segmentation accuracy of impoverished spaces in complex, high-density urban scenes using only a single high-resolution visible light remote sensing image, while reducing the dependence on multi-source remote sensing data, is a technical problem that urgently needs to be solved by technicians in this field. Summary of the Invention

[0006] The purpose of this invention is to provide a method for extracting impoverished space from high-resolution visible light remote sensing images for complex, high-density urban scenes, in order to solve the problem of insufficient accuracy in extracting impoverished space under complex background interference by existing deep learning models.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A method for impoverished space extraction based on edge multi-scale attention networks includes the following steps:

[0009] 1. Acquire high-resolution visible light remote sensing images of the target area and perform normalization preprocessing;

[0010] II. Construction of Edge Multi-Scale Attention Network: An edge multi-scale attention network is constructed using a "hierarchical encoder-decoder" architecture. The edge multi-scale attention network includes: a backbone encoder PVTv2-B0 network, a multi-scale skip refinement MSR module, an edge fusion attention EFA module, and a U-Net-like decoder.

[0011] 3. Introduce a multi-scale jump refinement module at the encoder end: global context features are extracted by the backbone encoder, and the MSR module is cascaded at the output of each stage of the encoder. The features processed by the MSR module are input to the decoder for feature fusion and upsampling restoration.

[0012] IV. Introducing an edge fusion attention module at the decoder end: Utilizing the EFA module attention mechanism to extract boundary information and spatial overlap features of the target region;

[0013] V. Complete Model Training: Introduce a composite loss function that combines binary cross-entropy loss and Dice loss to iteratively optimize the network parameters until the preset stopping condition is met, and obtain the optimal network model after training.

[0014] The beneficial effects of this invention are as follows:

[0015] (1) This invention introduces a multi-scale skip refinement (MSR) module at the encoder end and an edge fusion attention (EFA) module at the decoder end, providing a technical solution to the technical problems of insufficient multi-scale feature expression and inadequate utilization of boundary information under single image source conditions, thereby improving the recognition accuracy, boundary integrity and spatial continuity of impoverished spatial areas in complex high-density urban scenes. In the encoding stage, the MSR module adopts a parallel dual-branch structure: the spatial branch extracts multi-scale spatial context features, breaking through the limitation of a single receptive field, enabling the model to simultaneously capture the local details and wide-area spatial dependencies of building targets of different sizes; the channel branch applies global average pooling and max pooling to the input features respectively, and generates channel importance weights after fusion by a shared multilayer perceptron, realizing adaptive enhancement of key semantic channels and effective suppression of irrelevant background noise. In addition, the MSR module introduces a learnable scaling factor initialized to 0, and gradually learns the optimal fusion weights of the spatial branch and the channel branch during training, realizing dynamic adaptive optimization of feature expression in the form of residuals. During the decoding stage, the EFA module designed differentiated structure extraction strategies based on the characteristics of features at different levels: In the deep stage, for low-resolution features with blurred spatial boundaries due to multiple downsampling, a learnable edge detector is introduced, which dynamically fuses the gradient prior information captured by the Scharr operator with the semantic edge features extracted by the learnable convolution through learnable weights, effectively restoring clear semantic boundaries; In the shallow stage, for the characteristics of high-resolution features containing rich details but being susceptible to texture noise interference, a multi-scale Gaussian module is adopted, which effectively suppresses high-frequency noise while preserving multi-scale contextual information.

[0016] (2) Based on high-resolution visible light remote sensing images, this invention addresses the need for automated extraction of large-scale impoverished spatial areas, improves the precision and automation of spatial information acquisition in complex urban contexts, reduces the manual costs and computational overhead in traditional urban remote sensing monitoring, and provides efficient and scalable technical means for poverty spatial identification and spatial information extraction in the Greater Mumbai area and similar high-density urban areas around the world. Attached Figure Description

[0017] Figure 1 This is a flowchart of the overall method proposed in this invention;

[0018] Figure 2 This is a structural diagram of the multi-scale jump refinement module of the present invention;

[0019] Figure 3 This is a diagram of the edge blending attention module of the present invention;

[0020] Figure 4 Extract attention module diagrams for local features within the edge fusion attention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings and examples.

[0022] like Figure 1 As shown, this invention proposes an Edge Multi-Scale Attention Network (EMANet), employing an encoder-decoder architecture to construct a spatial extraction model for urban poverty in high-resolution visible light remote sensing imagery. The encoder uses PVTv2-B0 as its backbone to extract global contextual features and leverages the Transformer mechanism to enhance long-range dependency modeling capabilities, compensating for the insufficient global feature representation of convolutional neural networks. The decoder employs a U-Net-like progressive upsampling structure to restore spatial resolution, achieving effective fusion of shallow detail information and deep semantic information. Building upon this, this invention introduces a Multi-Scale Skip Refinement (MSR) module and an Edge Fusion Attention (EFA) module. The MSR module enhances the multi-scale contextual representation capability of the encoder's output features, while the EFA module adaptively optimizes the feature fusion process based on the semantic information of features at different levels, strengthening the representation of target boundary information and improving the accuracy of urban poverty spatial identification in complex scenarios. The main components can be summarized as follows:

[0023] (1) Data acquisition and preprocessing stage:

[0024] High-resolution visible light remote sensing imagery of Greater Mumbai was acquired and preprocessed using normalization to construct a dataset for network training. Specifically, the original large-format images were cropped and uniformly adjusted to 512×512 pixels to balance computational efficiency and feature representation capabilities. ImageNet standard mean and variance were used to normalize the images, ensuring pixel values ​​met uniform data distribution requirements, improving network training stability, and enhancing the model's ability to extract features from impoverished spatial targets in complex scenes.

[0025] (2) Semantic segmentation network construction:

[0026] The core architecture is EMANet. Its overall architecture consists of a feature encoder, an intermediate refinement module, and a feature decoder. The encoder uses a PVTv2-B0 backbone network and is initialized with pre-trained weights on large public datasets (such as ImageNet), giving the model sufficient prior visual representation capabilities.

[0027] During feature extraction, multi-scale skip refinement (MSR) modules are cascaded after the encoder's feature extraction. The feature maps output from each stage are input into the corresponding MSR modules to address the issues of insufficient contextual information and semantic ambiguity in single-scale feature extraction.

[0028] MSR employs a parallel dual-branch structure, consisting of a spatial branch and a channel branch, which comprehensively refine the feature map from two dimensions: spatial context and channel semantic response.

[0029] Spatial Branch: To capture spatial context information across different receptive fields, this branch employs a set of parallel depthwise convolutions with kernel sizes of 3×3, 5×5, and 7×7. This structure leverages the parallel combination of multi-sized kernels, enabling the model to focus on both fine local features and broader spatial dependencies, thus overcoming the limitations of a single receptive field. Subsequently, the extracted multi-scale spatial features are fused to generate a more refined spatial feature representation.

[0030] Channel Branch: To highlight key semantic features and suppress irrelevant noise, this branch focuses on utilizing the relevant information between channels. Using a channel attention mechanism, global average pooling and max pooling are applied to the input features respectively. Information is then fused using a shared multilayer perceptron (Shared MLP), and a bottleneck structure is employed to first compress and then restore the feature dimension, generating channel importance weights. These weights are then adaptively adjusted to the input features, resulting in an optimized channel feature representation.

[0031] A learnable residual fusion mechanism is introduced. MSR utilizes two learnable parameters. and Adaptive fusion is achieved between the original features and the outputs of the two branches. The final output is a multi-scale skip-refinement enhanced feature. Expressed as:

[0032] (1)

[0033] in This represents the input feature map, while and These represent the refined features generated by spatial branching and channel branching, respectively. Parameters and It is a learnable scaling factor initialized to 0, which enables the model to gradually learn the feature enhancement mechanism in the form of residuals during training. This allows the model to adaptively allocate the contribution of each branch according to the task requirements, thereby achieving dynamic optimization of feature representation.

[0034] After the decoder's convolutional blocks have finished processing and before performing linear upsampling to restore spatial resolution, the network incorporates an Edge Fusion Attention (EFA) module. The EFA module enhances the model's sensitivity to boundary details and strengthens the expression of structured semantics. This module adaptively adjusts its architecture based on the network's current stage, selecting an appropriate feature extraction method for the current depth. The EFA primarily consists of two parts: a Structure Extraction Branch and a Local Feature Extraction Attention (LFEA) module.

[0035] Structure extraction branch: To adapt to the features at different depths, this branch employs different mechanisms in the shallow and deep layers of the network:

[0036] Deep Stage: To address the issue of blurred spatial boundaries in low-resolution deep features due to multiple downsampling, a Learnable Edge Detector (LED) is introduced. Deep networks contain rich semantic information, but their structural information is relatively coarse. LED combines prior knowledge with data-driven learning, using the Scharr operator to capture gradient abrupt changes in the feature map, while simultaneously utilizing learnable convolutional layers to extract semantic edges and restore sharp boundaries. These two parts are connected by a learnable parameter. The final structure diagram is generated by fusion. The expression is defined as:

[0037] (2)

[0038] in, This represents the gradient edges captured by the Scharr operator, while This represents the semantic edge features extracted by learnable convolutions. Parameters It is a learnable weight used to dynamically balance the contributions of prior knowledge (Scharr gradient) and data-driven features (learnable convolution).

[0039] Shallow Stage: Addressing the issue that high-resolution shallow features contain rich details but are also highly susceptible to high-frequency noise and texture interference, a Multi-Scale Gaussian (MSG) module was employed. This module uses three different scale Gaussian kernels to process features in parallel, effectively smoothing irregular textures in the background and suppressing high-frequency noise while preserving contextual information at different scales.

[0040] Local feature extraction attention module:

[0041] To effectively integrate structural information into semantic representation while suppressing high-frequency noise, a Local Feature Enhancement Attention (LFEA) module is proposed. LFEA utilizes joint spatial and channel attention to transform the initial structural prior into content-aware feature weights. Since the lower-level structural prior focuses on spatial details, while the higher-level content features focus on abstract semantics, direct concatenation often fails to achieve effective synergy between the two. To address this issue, a structure-semantic pre-fusion step is first performed to align the structural prior with the content features. This process can be represented as:

[0042] (3)

[0043] in, and These represent the input content features and the extracted structural features, respectively. (Item) Content information is used to adjust structural priors, thereby matching the spatial structure with the local semantic context. Subsequently, This represents a convolutional module used to map adjusted features to intermediate feature representations. This is used for attention calculation.

[0044] The signals are fed into a parallel branch for processing to filter out informative signals. The Channel Attention Branch captures local channel interaction information through 1D convolution, thereby generating a channel weight map. Meanwhile, the spatial attention branch utilizes a large convolutional kernel ( This is used to locate the target region aligned with the boundary structure, thereby obtaining the spatial weight map. To dynamically balance the two branches, a learnable parameter is introduced. Final edge-blending attention enhancement features It is obtained through residual join, expressed as:

[0045] (4)

[0046] in, This represents the Sigmoid function. This represents element-wise multiplication. Combined with residual connections, the final feature output not only preserves the integrity of the original semantics but also achieves a balance between spatial and channel information through an adaptive weight allocation mechanism.

[0047] (3) Model training and parameter optimization

[0048] The network model is trained on a computing platform equipped with a high-performance graphics processing unit. In this embodiment, an NVIDIA RTX 5090 GPU with 32 GB of video memory is used as the computing device. The model parameters are updated using the AdamW optimizer, with a weight decay set to 0.01 to improve the model's generalization ability and suppress overfitting. The initial learning rate of the network is set to 1×10^-4, and optimization is achieved by combining a warm-up strategy and a dynamic learning rate adjustment strategy: a warm-up strategy is used for the first 200 iterations of training to gradually increase the learning rate and improve the stability of gradient updates in the early stages of training; after the warm-up, the ReduceLROnPlateau learning rate scheduling strategy is used. When the validation set loss does not decrease for 5 consecutive epochs, the current learning rate is adjusted to 0.5 times the original value to promote continuous model convergence.

[0049] During training, the batch size was set to 24, the maximum training epochs were set to 60, and an early stopping mechanism was introduced, using the validation set Dice coefficient as a performance monitoring metric. Training was automatically terminated when the Dice coefficient did not improve for 15 consecutive epochs, and the optimal model parameters were saved to avoid overfitting and improve training efficiency.

[0050] To address the significant class imbalance between impoverished spatial regions and background areas in high-resolution remote sensing images, this invention employs a composite loss function constructed from Binary Cross Entropy (BCE) and Dice loss functions, weighted and fused with a ratio of 0.5:0.5. The BCE loss optimizes global pixel classification accuracy, while the Dice loss enhances the overlap between predicted and ground truth labels, increasing the optimization weight of foreground targets and reducing the impact of background pixel dominance. This synergistic optimization of the two loss functions effectively mitigates missed detections caused by class imbalance while maintaining overall classification accuracy. It also improves the completeness, boundary continuity, and spatial consistency of impoverished spatial regions extracted in complex backgrounds, ensuring high-precision segmentation results for the network.

[0051] (4) Model performance evaluation

[0052] A test set, independent of the training data, is obtained. This test set consists of high-resolution remote sensing image patches uniformly adjusted to 512×512 pixels. The image to be predicted is input into the trained EMANet model for forward inference, outputting a pixel-level segmentation mask for the corresponding region. The Dice coefficient and mean intersection over union (mIoU) are used as evaluation metrics to quantitatively evaluate the model's impoverished space extraction performance.

[0053] Test results show that the proposed EMANet model achieves a Dice coefficient of 0.8769 and an average intersection-union ratio (IUR) of 0.8436. Compared with mainstream semantic segmentation models such as Swin-UperNet, DeepLab v3+, UNet, SegFormer, and Mask2Former, this invention achieves superior results in the IUR metric, with an improvement of approximately 0.97% over the best-performing comparative model, Swin-UperNet. These results demonstrate that by introducing a multi-scale skip refinement (MSR) module between the encoder and decoder and setting an edge fusion attention (EFA) module in the decoding stage, this invention effectively enhances the network's ability to jointly model multi-scale contextual information, target boundary features, and spatial structure information. In scenarios with complex backgrounds, dense buildings, and blurred boundaries, it can further improve the recognition accuracy, boundary integrity, and spatial continuity of impoverished spatial regions, while also exhibiting good robustness and generalization ability. The EMANet constructed in this invention can make full use of global semantic information and local edge information, and can achieve high-precision automatic extraction of impoverished spatial regions using only a single high-resolution visible light remote sensing image, reducing the dependence on multi-source heterogeneous data and showing good application prospects.

[0054] Therefore, this invention can meet the needs of large-scale, high-precision automated extraction of urban poverty space, reduce the cost of manual interpretation, improve the efficiency of urban remote sensing monitoring, and provide a high-precision, high-efficiency and easy-to-promote technical solution for poverty space identification, spatial information extraction and dynamic monitoring in the Greater Mumbai area and similar high-density urban areas around the world.

[0055] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for impoverished space extraction based on edge multi-scale attention networks, comprising the following steps:

1. Acquire high-resolution visible light remote sensing images of the target area and perform normalization preprocessing; II. Construction of Edge Multi-Scale Attention Network: An edge multi-scale attention network is constructed using a "hierarchical encoder-decoder" architecture. The edge multi-scale attention network includes: a backbone encoder PVTv2-B0 network, a multi-scale skip refinement MSR module, an edge fusion attention EFA module, and a U-Net-like decoder.

3. Introduce a multi-scale jump refinement module at the encoder end: global context features are extracted by the backbone encoder, and the MSR module is cascaded at the output of each stage of the encoder. The features processed by the MSR module are input to the decoder for feature fusion and upsampling restoration. IV. Introducing an edge fusion attention module at the decoder end: Utilizing the EFA module attention mechanism to extract boundary information and spatial overlap features of the target region; V. Complete Model Training: Introduce a composite loss function that combines binary cross-entropy loss and Dice loss to iteratively optimize the network parameters until the preset stopping condition is met, and obtain the optimal network model after training.

2. The method for impoverished space extraction based on edge multi-scale attention network as described in claim 1, characterized in that, In step three, the multi-scale jump refinement MSR module adopts a parallel dual-branch structure, including a spatial branch and a channel branch: Spatial branch: A set of parallel deep convolutions are used to extract spatial context information. The kernel sizes of the deep convolutions are 3×3, 5×5 and 7×7, respectively, and the extracted multi-scale spatial features are fused. Channel branching: Utilizing the channel attention mechanism, global average pooling and max pooling are applied to the input features respectively. By sharing information from the multilayer perceptron fusion, channel importance weights are generated and the input features are adaptively adjusted. The MSR module adaptively fuses the original features with the outputs of the two branches through a learnable residual fusion mechanism, and its output expression is: in Indicates the input feature map, and These represent the refined features generated by spatial branching and channel branching, respectively, with parameters... and It is a learnable scaling factor initialized to 0; This is to refine and enhance the features for the final multi-scale jump.

3. A method for impoverished space extraction based on an edge multi-scale attention network according to claim 1 or 2, characterized in that, Step four, the Edge Fusion Attention (EFA) module, includes a structure extraction branch and a Local Feature Extraction Attention (LFEA) module; the structure extraction branch adaptively adjusts the feature extraction method according to the network depth stage. (1) Deep stage: A learnable edge detector is introduced, and the Scharr operator is used to capture gradient abrupt change information of the feature map. At the same time, a learnable convolutional layer is used to extract semantic edges, and the final generated structure map is obtained. The expression is: in, This represents the gradient edges captured by the Scharr operator, while Represents semantic edge features extracted by learnable convolution; parameters Learnable weights for dynamically balancing the contributions of both; (2) Shallow stage: A multi-scale Gaussian module is used, and three different scales of Gaussian kernels are used to process the features in parallel to smooth the background texture and suppress high-frequency noise.

4. The method for impoverished space extraction based on edge multi-scale attention network according to claim 3, characterized in that, The specific process by which the Local Feature Extraction Attention LEFA module integrates structural information into semantic representation includes: Perform a structure-semantic pre-fusion step to align structural priors with content features, represented as: in, and These represent the input content features and the extracted structural features, respectively. This represents element-wise multiplication; it maps the adjusted features to intermediate feature representations. For use in attention calculations; The channel attention branch captures local channel interaction information through one-dimensional convolution to generate a channel weight map. The spatial attention branch uses large convolutional kernels to generate a spatial weight map. Ultimately, through residual connections and learnable parameters... Gain enhanced features: in, This represents the Sigmoid function. These are learnable parameters used for dynamically balancing the two branches; This is an attention enhancement feature for the final edge fusion.

5. A method for impoverished space extraction based on an edge multi-scale attention network according to claim 1 or 2, characterized in that, The step one, normalization preprocessing of image data, refers to uniformly adjusting the original large-format image to a resolution of 512×512 pixels, and mapping the image pixel values ​​using the ImageNet standard mean and variance.

6. A method for impoverished space extraction based on an edge multi-scale attention network according to claim 1 or 2, characterized in that, The composite loss function introduced in step five is a weighted combination of the binary cross-entropy loss and the Dice loss in a fixed weight ratio of 0.5:0.

5. Its overall loss function expression is as follows: in, Represents the composite loss function. This represents the binary cross-entropy loss, used for global pixel-level classification optimization; The Dice loss is used to maximize the cross-union ratio between the predicted mask and the ground truth label. Joint backpropagation is performed through the composite loss function to increase the optimization weight of the foreground target, which accounts for a very small proportion, and alleviate the class imbalance problem between the target region and the background.

7. A method for impoverished space extraction based on an edge multi-scale attention network according to claim 1 or 2, characterized in that, The optimization strategy for model training in step five is as follows: the AdamW optimizer is used to update parameters, and the weight decay is set to 0.01; the initial learning rate is set to 1e-4, and a warm-up strategy is implemented in the first 200 iterations to make the learning rate rise smoothly; then the platform learning rate scheduler is used, and when the validation set loss has not improved for 5 consecutive rounds, the learning rate is automatically reduced to 0.5 times the current value; an early stopping mechanism is introduced, using the Dice coefficient on the validation set as the monitoring indicator.

Images