Improved UNet remote sensing image segmentation network design method
Patent Information
- Application Number
- CN202511679893.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-17
- Publication Date
- 2026-03-24
AI Technical Summary
Existing remote sensing image segmentation algorithms struggle to accurately segment complex textures and small targets under high-resolution conditions. Traditional methods are ineffective and lack precise localization capabilities.
The classic UNet model is architecturally optimized by introducing the SE attention mechanism, TQConv module and LSKA-SPPF attention mechanism, improving the decoder part, and combining multiple attention mechanisms to enhance the model's ability to capture details.
It significantly improves the accuracy and robustness of remote sensing image segmentation, reduces the false negative and false positive rates, realizes an automated segmentation process for high-resolution remote sensing images, reduces manual intervention, and improves the segmentation accuracy for complex textures and small targets.
Smart Images
Figure CN121725237A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of image processing, in particular to an improved UNet remote sensing image segmentation network design method. BACKGROUND
[0002] Remote sensing image is a kind of earth observation data with wide coverage, diverse spectral information and variable target structure. Remote sensing technology has developed since the 1960s. It is usually collected by remote sensing satellites (such as Land Sat, Spot and MODIS series), unmanned aerial vehicles or airships and other platforms. Because remote sensing image has multiple spectral bands, such as visible light and infrared light, it provides rich information for representing ground features. Remote sensing image is the basis for the application of remote sensing technology and provides wide coverage and rich data support for many fields such as agriculture, city, environment, exploration and weather. Semantic segmentation of remote sensing image is the process of converting high-resolution remote sensing image into a map with semantic information. Through classification of each pixel, ground object targets are accurately extracted and identified. This technology is widely used in urban planning, land resource utilization, environmental protection, disaster monitoring and other fields to provide accurate and objective data support for related decision-making.
[0003] With the gradual expansion of the coverage of remote sensing image and the increasing number of objects involved, high-resolution remote sensing image has obvious ground feature details and high spatial resolution. The difficulty of extracting information from the image increases. Common remote sensing image algorithms include semantic segmentation, target recognition and detection, etc. Especially the semantic segmentation algorithm of remote sensing image is widely used in interpreting remote sensing image and natural image. Remote sensing image segmentation technology aims to classify remote sensing image at pixel level according to actual semantic information, so as to realize automatic identification and extraction of different ground object types. It divides the image into a series of regions with ground object category labels such as roads, farmland, villages and industrial areas, which is the processing basis for application researches such as urban planning, disaster monitoring and target recognition. Early methods for interpreting remote sensing image mainly use shallow semantic information such as color and texture, basic features of image, which have achieved good results in extracting low-level semantic information. However, with the improvement of image resolution, the challenge of segmentation task also gradually increases. Traditional image segmentation methods, such as rule-based segmentation and traditional machine learning methods, often rely on hand-designed features or thresholds. These rule-based methods do not perform well for complex features in high-resolution images.
[0004] In order to improve the segmentation accuracy and efficiency of remote sensing image, segmentation of high-resolution remote sensing image has gradually become a research hotspot. Some existing remote sensing segmentation systems try to realize the accuracy of segmentation process by restoring high resolution. Although it has been applied in some data sets, most systems still lack accurate positioning ability, resulting in unsatisfactory segmentation effect and unguaranteed accuracy. Summary of the Invention
[0005] To address the aforementioned technical problems, this invention provides an improved UNet remote sensing image segmentation network design method.
[0006] To achieve the above technical solution, the methods include: Collect remote sensing datasets and perform preprocessing operations; The classic UNet model underwent systematic architectural optimization and innovation. In terms of training strategy, the optimal hyperparameter configurations such as learning rate, batch size, and number of training epochs were determined through extensive experiments, laying the foundation for improving model performance. After training, the optimized model was obtained by comparing and evaluating the actual effects through accuracy, precision, average intersection-over-union ratio, and average pixel precision.
[0007] Furthermore, to address the core challenge of segmenting linear features such as fine roads and farmland boundaries in remote sensing images, a systematic architectural optimization and innovation was carried out on the classic UNet model. The improvements focused on the decoder portion of the classic UNet model, including: Specifically, after each skip connection and upsampled image fusion in the decoder, an SE attention mechanism is introduced to weight the feature differences of the merged feature maps to improve the model's segmentation accuracy for each category. Specifically, the TQConv module is used in the decoder to replace the convolution operation in the classic UNet model, thereby achieving feature capture for small targets and complex boundaries; Specifically, an LSKA-SPPF attention mechanism is introduced at the decoder output, enabling the model to capture more complex relationships between different parts of the input, thus providing better performance in tasks that capture details of small targets. First, the number of input feature channels is reduced in dimensionality using a 1×1 convolution of SPPF. Three max pooling operations are performed on the reduced features (kernel size 5×5, stride 1, padding 2), and the reduced features and the triple pooling features are concatenated in the channel dimension. Then, depthwise separable convolutions are used to extract horizontal and vertical features respectively. The two are added together and then compressed by a 1×1 convolution and activated by Sigmoid to generate attention weights. These weights are multiplied element-wise with the original features, and then normalized by groups, activated by SiLU, and mapped to the channel dimension by a 1×1 convolution to obtain the final features. The LSKA-SPPF attention mechanism reduces complexity by decomposing a large two-dimensional convolutional kernel into a one-dimensional convolution. Combined with multi-scale pooling to integrate global contextual information, it can enhance the model's ability to capture edge and similar features, suppress invalid noise, and increase the number of model parameters by almost no increase, thereby improving the boundary accuracy and multi-scale adaptability of remote sensing image segmentation.
[0008] Furthermore, in the TQConv module, the input feature map is... ,in Batch size For input channel number, and Let be the height and width of the feature map; the input feature map undergoes dual feature extraction through two parallel branches, A and B; firstly, each branch uses a 3×3 convolutional layer to extract two different sets of features from the input features respectively. Each set of features is then subjected to grouped normalization and ReLU activation to form a feature tensor with different channel dimensions, denoted as . and The expression is as follows: In the formula, This represents the convolution kernel of branch A; Represents the convolution kernel of branch B; ReLU represents the activation function; Indicates group normalization; and Different number of channels, different features and characteristics The number of channels is expressed as and The following relationship exists: In the formula, Indicates the number of output channels; Indicates the rank; if the number of output channels satisfies the square root requirement, then ; Extracted features and Reconstruct them into five-dimensional tensors containing rank dimensions. and This enables the reshaping of feature dimensions. The complexity of feature interaction depends on the rank, with each rank corresponding to a set of sub-features. For each sub-feature under a rank, features are computed at the height and width of each space via a broadcast mechanism. and The outer product along the channel dimension is performed, and the outer product results of all ranks are summed to form a tensor that integrates multiple sets of feature interaction information. The expression is as follows: In the formula, express The rank components express The rank components; The accumulated fusion tensor Remodeled into a two-dimensional channel form Then, the number of channels is adjusted to the target dimension through a 1×1 convolutional layer. After group normalization and ReLU activation, it is concatenated with the residuals of the original input after another 1×1 convolutional layer to output the final enhanced feature. The expression is as follows: In the formula, Indicates the input feature map; This represents the weight parameters of the convolutional layer.
[0009] Beneficial effects of the present invention This invention utilizes an improved UNet deep learning model, combined with a rich remote sensing image dataset and background data for training, significantly improving the accuracy and robustness of category segmentation. The model improves the segmentation accuracy of small feature pixels and is more sensitive to small targets against complex texture backgrounds. The system optimizes model inference and data transmission to meet the needs of remote sensing image segmentation, realizing an automated process from raw input to segmentation results, reducing manual intervention and lowering labor costs. Corresponding model attention mechanism algorithms are designed for features of different types and sizes, ensuring consistency and accuracy across categories, facilitating subsequent work, and exhibiting good scalability and portability.
[0010] This invention significantly improves the segmentation accuracy of complex textures and small targets in remote sensing by improving UNet; and by combining a multi-attention mechanism, it further reduces the false negative and false positive rates. Attached Figure Description
[0011] Figure 1 UNet architecture diagram designed for this invention; Figure 2 The TQConv structure diagram designed for this invention; Figure 3 This is a comparison image of remote sensing image segmentation according to the present invention, wherein, Figure 3 (a) is a remote sensing image. Figure 3 (b) is a schematic diagram of the segmentation results. Detailed Implementation
[0012] The present invention will be further described in detail below with reference to specific embodiments.
[0013] like Figure 1 As shown, an improved UNet remote sensing image segmentation network design method includes: By collecting remote sensing datasets and annotating them with Labelme, the datasets were divided into seven types of typical land features: bare land, vegetation, impermeable surfaces, roads, water bodies, agricultural land, and buildings. The sample size was expanded using various data augmentation techniques such as rotation, cropping, and brightness adjustment. The final total number of images was 4951. All experiments were conducted on the remote sensing datasets for training and testing, with a ratio of 8:1:1 between the training set, test set, and validation set.
[0014] This invention systematically optimizes and innovates the architecture of the classic UNet model for high-resolution remote sensing image semantic segmentation tasks. In terms of training strategy, the optimal hyperparameter configurations such as learning rate, batch size, and number of training epochs were determined through extensive experiments, laying the foundation for improving model performance. After training, the optimized model is obtained by comparing and evaluating the actual effects through accuracy, precision, average intersection-over-union ratio, and average pixel precision.
[0015] Furthermore, to address the core challenge of segmenting linear features such as fine roads and farmland boundaries in remote sensing images, a systematic architectural optimization and innovation was carried out on the classic UNet model. The improvements focused on the decoder portion of the classic UNet model, including: Specifically, after each skip connection and upsampled image fusion in the decoder, an SE attention mechanism is introduced to weight the differences in features of the merged feature map, thereby improving the model's segmentation accuracy for each category. The SE attention mechanism utilizes Squeeze and Excitation mechanisms to generate channel-adaptive attention weights, reweighting the concatenated features to highlight features in complex regions. First, global average pooling (GAP) is used to compress the spatial information of each channel of the input feature map into a single scalar to obtain the global statistical value of the channel. Then, two fully connected layers and ReLU activation are used to learn the dependencies between channels to generate channel weights. Finally, the original feature map is multiplied by the learned weights channel by channel to output the recalibrated features. The SE attention mechanism is embedded in the skip connection feature fusion of UNet, which can accurately strengthen the feature weights of key channels based on the differences in channel features such as color and texture of remote sensing images, significantly improving the recognition rate of complex categories. Specifically, the TQConv module is used in the decoder to replace the convolution operation in the classic UNet model, thereby achieving feature capture for small targets and complex boundaries. Efficient feature extraction and fusion are achieved through low-rank decomposition and a dual-branch structure: it first receives the input feature map, distributes it to two parallel branches, and extracts differentiated features on different channels through 3×3 convolution, group normalization, and ReLU activation. The features are then reshaped into a five-dimensional tensor with rank dimensions. The outer product of each rank sub-feature in the channel dimension is calculated through a broadcast mechanism and accumulated to form a tensor that fuses multiple sets of interactive information. Subsequently, the channel dimensions are adjusted through 1×1 convolution, and combined with group normalization, ReLU activation, and residual connections to the original input, the enhanced features are output. This module can strengthen high-order interaction relationships between features while reducing computational cost. Specifically, an LSKA-SPPF attention mechanism is introduced at the decoder output, enabling the model to capture more complex relationships between different parts of the input, thus providing better performance in tasks that capture details of small targets. First, the number of input feature channels is reduced in dimensionality using a 1×1 convolution of SPPF. Three max pooling operations are performed on the reduced features (kernel size 5×5, stride 1, padding 2), and the reduced features and the triple pooling features are concatenated in the channel dimension. Then, depthwise separable convolutions are used to extract horizontal and vertical features respectively. The two are added together and then compressed by a 1×1 convolution and activated by Sigmoid to generate attention weights. These weights are multiplied element-wise with the original features, and then normalized by groups, activated by SiLU, and mapped to the channel dimension by a 1×1 convolution to obtain the final features. The LSKA-SPPF attention mechanism reduces complexity by decomposing a large two-dimensional convolutional kernel into a one-dimensional convolution. Combined with multi-scale pooling to integrate global contextual information, it can enhance the model's ability to capture edge and similar features, suppress invalid noise, and increase the number of model parameters by almost no increase, thereby improving the boundary accuracy and multi-scale adaptability of remote sensing image segmentation.
[0016] like Figure 2 As shown, in the TQConv module, the input feature map is... ,in Batch size For input channel number, and Let be the height and width of the feature map; the input feature map undergoes dual feature extraction through two parallel branches, A and B; firstly, each branch uses a 3×3 convolutional layer to extract two different sets of features from the input features respectively. Each set of features is then subjected to grouped normalization and ReLU activation to form a feature tensor with different channel dimensions, denoted as . and The expression is as follows: In the formula, This represents the convolution kernel of branch A; Represents the convolution kernel of branch B; ReLU represents the activation function; Indicates group normalization; and Different number of channels, different features and characteristics The number of channels is expressed as and The following relationship exists: In the formula, Indicates the number of output channels; Indicates the rank; if the number of output channels satisfies the square root requirement, then ; Extracted features and Reconstruct them into five-dimensional tensors containing rank dimensions. and This enables the reshaping of feature dimensions. The complexity of feature interaction depends on the rank, with each rank corresponding to a set of sub-features. For each sub-feature under a rank, features are computed at the height and width of each space via a broadcast mechanism. and The outer product along the channel dimension is performed, and the outer product results of all ranks are summed to form a tensor that integrates multiple sets of feature interaction information. The expression is as follows: In the formula, express The rank components express The rank components; The accumulated fusion tensor Remodeled into a two-dimensional channel form Then, the number of channels is adjusted to the target dimension through a 1×1 convolutional layer. After group normalization and ReLU activation, it is concatenated with the residuals of the original input after another 1×1 convolutional layer to output the final enhanced feature. The expression is as follows: In the formula, Indicates the input feature map; This represents the weight parameters of the convolutional layer.
[0017] The TQConv module proposed in this invention implements tensor product operation of low-rank decomposition, which can efficiently capture complex interactions between features, thereby improving the model's recognition accuracy for difficult-to-segment small target categories and complex feature categories. The residual structure ensures stable gradient propagation, ensuring a balance between robustness and efficiency of the model in remote sensing image segmentation tasks.
[0018] To adapt to the significant scale differences of ground features in remote sensing images, this embodiment integrates the TQConv mechanism into each convolutional operation of the UNet decoder. This full-level deployment ensures that the model maintains its sensitivity to small ground features across different receptive fields. In particular, by performing multi-scale inference on the same image and normalizing and fusing the segmentation results from each scale, the low-rank decomposition characteristics of TQConv are fully utilized. Even with smaller-scale inputs, its tensor decomposition mechanism can effectively preserve the key features of ground features, avoiding the feature loss problem caused by scale compression in traditional methods.
[0019] like Figure 3 middle, Figure 3 (a) and Figure 3 As shown in (b), through system module optimization and parameter tuning (rank parameter set to 4), this embodiment achieves significant performance improvements on multiple remote sensing benchmark datasets. These improvements fully demonstrate the effectiveness and practicality of the TQConv module in remote sensing image semantic segmentation tasks, providing an innovative solution for fine segmentation of high-resolution remote sensing images. The innovation of this embodiment lies in the deep integration of advanced tensor decomposition attention mechanisms with classic segmentation networks. Through feature interaction enhancement and multi-scale optimization, it effectively solves the long-standing problem of small target segmentation in remote sensing images, laying a solid technical foundation for subsequent intelligent remote sensing interpretation applications.
[0020] It should be noted that the above are merely preferred embodiments of this application and do not limit the scope of patent protection of this application. Any equivalent structural or procedural changes made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.
Claims
1. An improved UNet remote sensing image segmentation network design method, characterized in that, The design method includes: Remote sensing datasets were collected and divided into satellite remote sensing datasets for seven typical land features: bare land, vegetation, impermeable surfaces, roads, water bodies, agricultural land, and buildings. Preprocessing operations were performed on the datasets. The preprocessed datasets were then input into an improved UNet model for image semantic segmentation. The improved UNet model uses the classic UNet model as the base model. In the decoder, after each skip connection and upsampled image fusion, an SE attention mechanism is introduced. In the decoder, the TQConv module replaces the convolution operation in the classic UNet model. An LSKA-SPPF attention mechanism is introduced at the output of the decoder.
2. The improved UNet remote sensing image segmentation network design method according to claim 1, characterized in that: In the TQConv module, after the input feature map, the image is assigned to two parallel branches A and B, and then subjected to 3×3 convolution and group normalization. After ReLU activation, the features are reshaped into a five-dimensional tensor with rank. The outer product of each rank sub-feature in the channel dimension is calculated through a broadcast mechanism and accumulated to form a tensor that integrates multiple sets of interactive information. Subsequently, the channel dimension is adjusted by 1×1 convolution and the groups are normalized. After ReLU activation, the residuals of the original input are connected to the output to enhance the features.
3. The improved UNet remote sensing image segmentation network design method according to claim 1, characterized in that: The LSKA-SPPF attention mechanism first reduces the dimensionality of the input feature channels using a 1×1 convolution of SPPF. Then, it performs triple max pooling on the reduced features, with the kernel size of the triple max pooling set to 5×5, stride of 1, and padding of 2. The reduced features and triple pooling features are then concatenated along the channel dimension. Next, depthwise separable convolutions are used to extract horizontal and vertical features respectively. The two are added together and then compressed by a 1×1 convolution and activated by Sigmoid to generate attention weights. These weights are then multiplied element-wise with the original features, and finally, group normalization, SiLU activation, and a 1×1 convolution are performed to map the channel dimension to obtain the final features.
4. The improved UNet remote sensing image segmentation network design method according to claim 2, characterized in that: In the two parallel branches A and B, each a 3×3 convolutional layer extracts two different sets of features from the input features. Each set of features is then subjected to grouped normalization and ReLU activation to form feature tensors with different channel dimensions, denoted as... and ,in and The height and width of the feature map, The expression for batch size is as follows: In the formula, , representing the input feature map, Input the number of channels; This represents the convolution kernel of branch A; Represents the convolution kernel of branch B; ReLU represents the activation function; Indicates group normalization; and Different number of channels, different features and characteristics The number of channels is expressed as follows: and The following relationship exists: In the formula, Indicates the number of output channels; Indicates the rank; if the number of output channels satisfies the square root requirement, then .
5. The improved UNet remote sensing image segmentation network design method according to claim 2, characterized in that: The feature is reshaped into a five-dimensional tensor with rank dimension. The outer product of each rank sub-feature in the channel dimension is calculated using a broadcast mechanism and then accumulated to form a tensor that integrates multiple sets of interactive information, denoted as... The expression is as follows: In the formula, express The rank components express The rank components; Indicates rank.
6. The improved UNet remote sensing image segmentation network design method according to claim 2, characterized in that: The channel dimensions are then adjusted via 1×1 convolution, grouped, normalized, and ReLU activated, and then concatenated with the residual of the original input to output enhanced features, denoted as... ,in and The height and width of the feature map, For batch size, The number of output channels is expressed as follows: In the formula, Indicates the input feature map, Input the number of channels; This represents the weight parameters of the convolutional layer.