Remote sensing image semantic segmentation method based on deep learning

Abstract

The invention discloses a remote sensing image semantic segmentation method based on deep learning, and relates to the technical field of image processing, and the method comprises the steps: carrying out the feature extraction of a target remote sensing image, and obtaining abstract semantic information features and edge information features; determining a first key feature corresponding to the target remote sensing image in the horizontal direction and a second key feature corresponding to the target remote sensing image in the vertical direction based on the abstract semantic information feature and the edge information feature; performing feature operation on the first key feature and the second key feature to obtain an output feature map corresponding to the target remote sensing image; and performing semantic segmentation on the output feature map to obtain a semantic segmentation result corresponding to the target remote sensing image. According to the invention, semantic segmentation is carried out on the output feature map obtained by carrying out feature operation on the first key feature corresponding to the target remote sensing image in the horizontal direction and the second key feature corresponding to the target remote sensing image in the vertical direction, so that edge details of the target remote sensing image can be more accurately captured from a semantic segmentation result.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a remote sensing image semantic segmentation method based on deep learning. Background Art

[0002] Remote sensing semantic segmentation assigns each pixel in a remote sensing image to a semantic category and labels it with different colors to form a segmentation map, thereby enabling the identification and extraction of different objects and regions on the ground. Remote sensing semantic segmentation has a wide range of applications in many fields, such as urban planning, land use, precision agriculture, environmental monitoring, and disaster assessment. Therefore, accurate semantic segmentation of remote sensing imagery is a key factor in determining the success and impact of Earth science research and its applications.

[0003] Currently, traditional semantic segmentation methods (such as those based on fully convolutional networks and fully connected fusion paths) can effectively capture semantic information in remote sensing images, but they suffer from the technical problem of blurred segmentation boundaries, resulting in low segmentation accuracy. Therefore, the industry is in urgent need of a semantic segmentation method that can accurately extract remote sensing image boundaries to improve segmentation accuracy. Summary of the Invention

[0004] The main purpose of this application is to provide a remote sensing image semantic segmentation method based on deep learning, aiming to solve the technical problem that the existing remote sensing image semantic segmentation methods have blurred segmentation boundaries and lead to low segmentation accuracy of remote sensing images.

[0005] To achieve the above objectives, the present application provides a remote sensing image semantic segmentation method based on deep learning, which includes the following steps:

[0006] Extract features from the target remote sensing image to obtain abstract semantic information features and edge information features;

[0007] Determining a first key feature corresponding to the target remote sensing image in the horizontal direction and a second key feature corresponding to the target remote sensing image in the vertical direction based on the abstract semantic information feature and the edge information feature;

[0008] Performing feature operations on the first key feature and the second key feature to obtain an output feature map corresponding to the target remote sensing image;

[0009] Perform semantic segmentation on the output feature map to obtain a semantic segmentation result corresponding to the target remote sensing image.

[0010] In one embodiment, the step of extracting features from the target remote sensing image to obtain abstract semantic information features and edge information features includes:

[0011] Extracting features of the target remote sensing image through a deep convolutional layer to obtain abstract semantic information features, wherein the abstract semantic information features include category features and attribute features of the object in the target remote sensing image;

[0012] The target remote sensing image is subjected to feature extraction through a shallow convolutional layer and a gated channel attention conversion unit to obtain edge information features, wherein the edge information features include contour features and texture features of the object in the target remote sensing image.

[0013] In one embodiment, the step of extracting features from the target remote sensing image through a shallow convolutional layer and a gated channel attention conversion unit to obtain edge information features includes:

[0014] Extracting features of the target remote sensing image through a shallow convolutional layer to obtain global features;

[0015] The global features are subjected to channel normalization and gate adaptation through a gated channel attention conversion unit to enhance key feature channels in the global features and suppress noise boundaries, thereby obtaining edge information features.

[0016] In one embodiment, the step of determining the first key feature corresponding to the target remote sensing image in the horizontal direction and the second key feature corresponding to the target remote sensing image in the vertical direction based on the abstract semantic information feature and the edge information feature includes:

[0017] Performing element-wise operations on the abstract semantic information features and the edge information features and then performing depth-wise separable convolution to obtain initial fused features;

[0018] Global average pooling is performed on the initial fusion features in the horizontal direction and the vertical direction respectively to obtain a first key feature corresponding to the target remote sensing image in the horizontal direction and a second key feature corresponding to the target remote sensing image in the vertical direction.

[0019] In one embodiment, the step of performing feature operation on the first key feature and the second key feature to obtain an output feature map corresponding to the target remote sensing image includes:

[0020] After performing two-dimensional convolution, batch normalization, and nonlinear activation on the first key feature and the second key feature, a third key feature and a fourth key feature are obtained;

[0021] Performing one-dimensional convolution transformation and sigmoid normalization on the third key feature and the fourth key feature to obtain attention maps corresponding to the target remote sensing image in the horizontal direction and the vertical direction;

[0022] The attention map is element-wise multiplied by the initial fusion feature to obtain an output feature map corresponding to the target remote sensing image.

[0023] In one embodiment, the step of performing semantic segmentation on the output feature map to obtain a semantic segmentation result corresponding to the target remote sensing image includes:

[0024] The output feature map is semantically segmented through a multi-scale cascaded dilated convolution module to obtain a semantic segmentation result corresponding to the target remote sensing image, and the dilated rate combinations of the multi-scale cascaded dilated convolution module are 4, 8, 12, and 16.

[0025] In addition, to achieve the above objectives, the present application also proposes a remote sensing image semantic segmentation device based on deep learning, the remote sensing image semantic segmentation device based on deep learning comprising:

[0026] Feature extraction module, used to extract features from target remote sensing images to obtain abstract semantic information features and edge information features;

[0027] a feature fusion module, configured to determine a first key feature corresponding to the target remote sensing image in the horizontal direction and a second key feature corresponding to the target remote sensing image in the vertical direction based on the abstract semantic information feature and the edge information feature;

[0028] a feature operation module, configured to perform feature operation on the first key feature and the second key feature to obtain an output feature map corresponding to the target remote sensing image;

[0029] The semantic segmentation module is used to perform semantic segmentation on the output feature map to obtain a semantic segmentation result corresponding to the target remote sensing image.

[0030] In addition, to achieve the above-mentioned purpose, the present application also proposes a remote sensing image semantic segmentation device based on deep learning, which includes: a memory, a processor, and a remote sensing image semantic segmentation program based on deep learning stored on the memory and runnable on the processor. The remote sensing image semantic segmentation program based on deep learning is configured to implement the steps of the remote sensing image semantic segmentation method based on deep learning as described above.

[0031] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and stores a remote sensing image semantic segmentation program based on deep learning. When the remote sensing image semantic segmentation program based on deep learning is executed by a processor, the steps of the remote sensing image semantic segmentation method based on deep learning as described above are implemented.

[0032] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer program product, which includes a remote sensing image semantic segmentation program based on deep learning. When the remote sensing image semantic segmentation program based on deep learning is executed by a processor, it implements the steps of the remote sensing image semantic segmentation method based on deep learning as described above.

[0033] The present application performs feature extraction on the target remote sensing image to obtain abstract semantic information features and edge information features; based on the abstract semantic information features and the edge information features, determines the first key feature corresponding to the target remote sensing image in the horizontal direction and the second key feature corresponding to the target remote sensing image in the vertical direction; performs feature operation on the first key feature and the second key feature to obtain an output feature map corresponding to the target remote sensing image; performs semantic segmentation on the output feature map to obtain a semantic segmentation result corresponding to the target remote sensing image. Compared with traditional remote sensing image semantic segmentation methods, the above-mentioned method of the present application determines the first key feature corresponding to the target remote sensing image in the horizontal direction and the second key feature corresponding to the vertical direction based on the abstract semantic information features and edge information features extracted from the target remote sensing image, and then performs semantic segmentation on the output feature map corresponding to the target remote sensing image obtained after feature operation based on the first key feature and the second key feature, thereby being able to more accurately capture the edge details of the target remote sensing image from the obtained semantic segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a schematic diagram of the structure of a remote sensing image semantic segmentation device based on deep learning in the hardware operating environment involved in the embodiment of the present application;

[0035] Figure 2 This is a flowchart of the first embodiment of the remote sensing image semantic segmentation method based on deep learning in this application;

[0036] Figure 3 This is a flowchart of the second embodiment of the remote sensing image semantic segmentation method based on deep learning of this application;

[0037] Figure 4 This is a flowchart of the third embodiment of the remote sensing image semantic segmentation method based on deep learning of this application;

[0038] Figure 5 This is a schematic diagram of the multi-scale cascaded dilated convolution module for the deep learning-based remote sensing image semantic segmentation method in this application;

[0039] Figure 6 This is a structural block diagram of the first embodiment of the remote sensing image semantic segmentation device based on deep learning in this application.

[0040] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0041] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0042] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a remote sensing image semantic segmentation device based on deep learning in the hardware operating environment involved in the embodiment of the present application.

[0043] like Figure 1 As shown, the deep learning-based remote sensing image semantic segmentation device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a wireless fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0044] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the remote sensing image semantic segmentation device based on deep learning, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0045] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a remote sensing image semantic segmentation program based on deep learning.

[0046] exist Figure 1In the remote sensing image semantic segmentation device based on deep learning shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the remote sensing image semantic segmentation device based on deep learning of this application can be set in the remote sensing image semantic segmentation device based on deep learning, and the remote sensing image semantic segmentation device based on deep learning calls the remote sensing image semantic segmentation program based on deep learning stored in the memory 1005 through the processor 1001, and executes the remote sensing image semantic segmentation method based on deep learning provided in the embodiment of this application.

[0047] The present invention provides a method for semantic segmentation of remote sensing images based on deep learning. Figure 2 , Figure 2 This is a flowchart of the first embodiment of the remote sensing image semantic segmentation method based on deep learning in this application.

[0048] In this embodiment, the remote sensing image semantic segmentation method based on deep learning includes the following steps:

[0049] Step S10: extracting features from the target remote sensing image to obtain abstract semantic information features and edge information features.

[0050] It should be noted that the execution entity of the method of this embodiment can be a terminal device with feature extraction, data processing, and program execution functions, such as a smartphone, computer, etc., or an electronic device with the same or similar functions, such as the above-mentioned deep learning-based remote sensing image semantic segmentation device. The following uses the deep learning-based remote sensing image semantic segmentation device as an example to illustrate this embodiment and the following embodiments.

[0051] It should also be noted that the aforementioned deep learning-based remote sensing image semantic segmentation device includes an improved UnetFormer model. The UNetFormer model is a model for efficient semantic segmentation that combines the features of UNet and Transformer. This model consists of two main components: a convolutional neural network-based encoder and a Transformer-based decoder. The encoder uses the lightweight ResNet18 as a feature extractor. ResNet18 is a classic convolutional neural network architecture with a low number of parameters and computational complexity, yet still effectively extracts image features. The decoder employs three global-local attention blocks to capture both global and local information. These global-local attention blocks enable the model to focus on both the entire image and local details, obtaining more comprehensive semantic information. Furthermore, the decoder includes a feature refinement head to enhance channel and spatial feature representation. Although the UNetFormer model effectively captures semantic information in images, its segmentation results fail to fully capture the shape of objects, resulting in blurred segmentation boundaries. Therefore, this embodiment improves upon the UNetFormer model to more accurately capture image edge details when performing semantic segmentation on target remote sensing images.

[0052] It is understood that the target remote sensing images mentioned above refer to image information of the Earth's surface and atmosphere acquired by satellites, aircraft, or other sensors. Target remote sensing images can be collected, detected, and recorded using sensors. These sensors are typically carried on platforms such as satellites and aircraft and are capable of remotely and non-contactly detecting the electromagnetic radiation and reflection characteristics of objects on the Earth's surface. Target remote sensing images can record a variety of information about target objects, such as shape, size, color, and texture. This information can reveal various attribute information, including the overall characteristics and distribution patterns of the target objects. Therefore, they are widely used in environmental monitoring, urban planning, agricultural management, and other fields.

[0053] It should be understood that the aforementioned abstract semantic information features refer to information contained in the target remote sensing image that can express the high-level meaning of the object or scene. This high-level information is often closely related to human understanding of images, such as the category features (including forests, mountains, farmland, etc.) and attribute features (including building density, green space ratio, canopy density, etc.) of the objects in the target remote sensing image. The aforementioned edge information features refer to the features possessed by the edges of the objects or scenes in the target remote sensing image. These features are often related to the shape, outline, and boundary of the objects, and are important bases for tasks such as image segmentation, target recognition, and scene understanding. Examples include contour features and texture features in the target remote sensing image.

[0054] Step S20: determining a first key feature corresponding to the target remote sensing image in the horizontal direction and a second key feature corresponding to the target remote sensing image in the vertical direction based on the abstract semantic information feature and the edge information feature.

[0055] It should be noted that the first key feature mentioned above refers to the feature that the target remote sensing image contains edge details in the horizontal direction, and the second key feature mentioned above refers to the feature that the target remote sensing image contains edge details in the vertical direction.

[0056] It should be understood that by performing element operations (such as convolution, normalization, etc.) on the above-mentioned abstract semantic information features and edge information features, the first key feature corresponding to the above-mentioned target remote sensing image in the horizontal direction and the second key feature corresponding to the above-mentioned target remote sensing image in the vertical direction can be obtained.

[0057] Step S30: performing feature operations on the first key feature and the second key feature to obtain an output feature map corresponding to the target remote sensing image.

[0058] In a specific implementation, the output feature map corresponding to the target remote sensing image can be obtained by performing feature operations including but not limited to convolution, normalization, and nonlinear activation on the first and second key features. In particular, the output feature map can include information such as object categories (e.g., forests, mountains, farmland, etc.), object attributes (e.g., building density, green space ratio, canopy density, etc.), object contours, and object textures in the real scene corresponding to the target remote sensing image, which is not limited in this embodiment.

[0059] Step S40: performing semantic segmentation on the output feature map to obtain a semantic segmentation result corresponding to the target remote sensing image.

[0060] In a specific implementation, the above-mentioned output feature map can be semantically segmented through an encoder-decoder structure to obtain the semantic segmentation result corresponding to the target remote sensing image. More specifically, the encoder-decoder structure is the basic framework in the image semantic segmentation task. The encoder performs dimensionality reduction and feature extraction on the input image through convolution operations, while the decoder is responsible for mapping these features back to the original resolution and generating pixel-level prediction results. This structure achieves efficient and accurate image semantic segmentation while reducing computational complexity and the number of parameters. It should be noted that, in this embodiment, the above-mentioned method of performing semantic segmentation on the output feature map through the encoder-decoder structure is only used for illustration and not for limitation. Other methods of performing semantic segmentation on the feature map are also applicable to this embodiment and will not be elaborated here.

[0061] This embodiment performs feature extraction on the target remote sensing image to obtain abstract semantic information features and edge information features; based on the abstract semantic information features and the edge information features, a first key feature corresponding to the target remote sensing image in the horizontal direction and a second key feature corresponding to the target remote sensing image in the vertical direction are determined; feature operations are performed on the first key feature and the second key feature to obtain an output feature map corresponding to the target remote sensing image; and semantic segmentation is performed on the output feature map to obtain a semantic segmentation result corresponding to the target remote sensing image. Compared to traditional remote sensing image semantic segmentation methods, the method of this embodiment determines the first key feature corresponding to the target remote sensing image in the horizontal direction and the second key feature corresponding to the vertical direction based on the abstract semantic information features and edge information features extracted from the target remote sensing image, and then performs semantic segmentation on the output feature map corresponding to the target remote sensing image obtained after feature operations based on the first key feature and the second key feature, thereby being able to more accurately capture the edge details of the target remote sensing image from the obtained semantic segmentation results.

[0062] refer to Figure 3 , Figure 3 This is a flow chart of the second embodiment of the remote sensing image semantic segmentation method based on deep learning in this application.

[0063] In a feasible implementation, step S10 may include:

[0064] Step S101: extracting features of a target remote sensing image through a deep convolutional layer to obtain abstract semantic information features, where the abstract semantic information features include category features and attribute features of objects in the target remote sensing image.

[0065] Step S102: extracting features of the target remote sensing image through a shallow convolutional layer and a gated channel attention conversion unit to obtain edge information features, where the edge information features include contour features and texture features of the object in the target remote sensing image.

[0066] In its implementation, this embodiment introduces a Bottleneck Attention Module (BAM) to extract edge information of the target. However, due to the high noise content in low-level features, the aforementioned gated channel attention transformation (GCT) is also introduced to enhance important feature channels and suppress noisy edges. GCT primarily consists of three components: global context embedding, channel normalization, and gate adaptation. GCT embeds global context information to capture the global characteristics of each channel, thereby avoiding local semantic ambiguity and enabling shallow-level features to better capture common properties. Through normalization, GCT establishes a competitive relationship between channels, strengthening channels with larger responses while suppressing channels with smaller feedback. This normalization method establishes a competitive relationship between neurons or channels, thereby improving model performance. GCT also implements channel selection through gating weights and biases: when a channel's gating weight is positively activated, GCT encourages that channel to compete with other channels; when the gating weight is negatively activated, GCT encourages that channel to cooperate with other channels. This mechanism effectively enhances important feature channels (i.e., the aforementioned edge information features) and suppresses noisy edges.

[0067] In a feasible implementation, step S102 may include:

[0068] Step S1021: extracting features from the target remote sensing image through a shallow convolutional layer to obtain global features.

[0069] Step S1022: performing channel normalization and gate adaptation on the global features through a gated channel attention conversion unit to enhance key feature channels in the global features and suppress noise boundaries to obtain edge information features.

[0070] In a specific implementation, the above global features can be input into the gated channel attention conversion unit to adaptively adjust the channel weights to enhance important feature channels, suppress noise boundaries, and enhance contextual relevance. Next, the weighted features are fused through two 3x3 convolution operations. Finally, the fused features are normalized through a 1x1 convolution and a sigmoid function to obtain the above edge information features. In this way, the boundary perception module can extract features with edge information from different levels of the encoder and enhance important features through GCT to obtain more accurate boundary predictions, which helps improve the accuracy of remote sensing image segmentation tasks.

[0071] This embodiment performs feature extraction on the target remote sensing image through a deep convolutional layer to obtain abstract semantic information features, which include category features and attribute features of the object in the target remote sensing image; performs feature extraction on the target remote sensing image through a shallow convolutional layer to obtain global features; performs channel normalization and gated adaptation on the global features through a gated channel attention conversion unit to enhance the key feature channels in the global features and suppress noise boundaries to obtain edge information features, which include contour features and texture features of the object in the target remote sensing image. Compared with traditional remote sensing image semantic segmentation methods, the above method in this embodiment extracts key features from the target remote sensing image into global features by introducing a boundary perception module, and enhances these key features through a gated channel attention conversion unit to obtain more accurate boundary predictions, which helps to improve the semantic segmentation accuracy of the target remote sensing image.

[0072] refer to Figure 4 , Figure 4 This is a flowchart of the third embodiment of the remote sensing image semantic segmentation method based on deep learning in this application.

[0073] In a feasible implementation, step S20 may include:

[0074] Step S201: performing element-wise operations on the abstract semantic information features and the edge information features and then performing depthwise separable convolution to obtain initial fusion features.

[0075] In a specific implementation, the above-mentioned abstract semantic information features can be downsampled and then element-wise multiplied with the edge information features. The resulting product is passed through a skip connection and a 3×3 depth-separable convolution to obtain the above-mentioned initial fusion features.

[0076] Step S202: performing global average pooling in the horizontal direction and the vertical direction on the initial fusion features to obtain a first key feature corresponding to the target remote sensing image in the horizontal direction and a second key feature corresponding to the target remote sensing image in the vertical direction.

[0077] In the specific implementation, this embodiment proposes an improved boundary-guided fusion module (BFM), which realizes a more effective fusion of the above-mentioned abstract semantic information features and the above-mentioned edge information features by adopting depthwise separable convolution and spatial information encoding improvement strategy. This embodiment adopts 3×3 depthwise separable convolution to replace the traditional 3×3 convolution. The depthwise separable convolution decomposes the convolution into depthwise convolution and pointwise convolution, which significantly reduces the computational complexity and the number of parameters while maintaining the feature extraction capability. In order to more effectively fuse the abstract semantic information features and edge information features, BFM introduces Coordinate Attention (an attention mechanism for computer vision tasks) to decompose the channel attention into two one-dimensional feature encoding processes. Coordinate Attention can aggregate features in the horizontal and vertical directions respectively. This method not only retains the dependency between channels, but also embeds position information, so that the model can more accurately capture the position information of the target area. Specifically, Coordinate Attention can achieve its function in two steps: first, it aggregates global information along one direction (such as horizontally or vertically), and then performs the same operation in the other direction. This bidirectional aggregation process enables the model to capture richer spatial information and long-distance dependencies. In order to further enhance the ability of BFM in edge feature extraction, this embodiment also proposes an improved spatial information encoding strategy, which introduces an additional spatial encoding layer to further encode and represent the input features in spatial dimensions, so that the model can better understand the spatial relationship between different positions in the image. This explicit encoding of spatial information helps the model to more accurately locate edge positions and retain more edge detail information when fusing abstract semantic information features and edge information features.

[0078] In a feasible implementation, step S30 may include:

[0079] Step S301: performing two-dimensional convolution, batch normalization, and nonlinear activation on the first key feature and the second key feature to obtain a third key feature and a fourth key feature.

[0080] Step S302: performing one-dimensional convolution transformation and sigmoid normalization on the third key feature and the fourth key feature to obtain attention maps corresponding to the target remote sensing image in the horizontal direction and the vertical direction.

[0081] Step S303: performing element-wise multiplication of the attention map and the initial fusion feature to obtain an output feature map corresponding to the target remote sensing image.

[0082] In a specific implementation, the first and second key features can be subjected to two-dimensional convolution, batch normalization, and nonlinear activation to obtain the third and fourth key features. The third and fourth key features are then subjected to two-dimensional convolution and sigmoid normalization to generate attention maps corresponding to the target remote sensing image in the horizontal and vertical directions. Finally, the attention map is element-wise multiplied with the initial fused features to obtain the output feature map corresponding to the target remote sensing image.

[0083] In a feasible implementation, step S40 may include:

[0084] Step S401: semantically segmenting the output feature map through a multi-scale cascaded dilated convolution module to obtain a semantic segmentation result corresponding to the target remote sensing image, wherein the dilated rate combinations of the multi-scale cascaded dilated convolution module are 4, 8, 12, and 16.

[0085] In a specific implementation, in the traditional DeepLabv3+ (a deep learning model), the dilated convolutions used in the ASPP (Atrous Spatial Pyramid Pooling) module have dilated ratios of 6, 12, and 18. However, as the backbone network extracts features, the resolution of the feature map gradually decreases, resulting in the inability of the 6, 12, and 18 combinations to effectively extract features from multi-resolution feature maps. In addition, since a smaller dilated ratio is not set, the network's ability to segment small objects is poor, resulting in weak performance when processing segmentation objects of different sizes. In order to more effectively extract features from multi-resolution feature maps and improve the segmentation capability of segmentation objects of different sizes, this embodiment changes the dilated convolution ratio combination to 4, 8, 12, and 16. To reduce computational costs, this embodiment replaces standard convolution with depthwise separable dilated convolution. To further improve feature utilization efficiency, multi-level fused feature information is obtained through fusion, that is, the input features are pooled and then spliced with the multi-level fused feature information. The improved ASPP is called Multi-Scale Cascaded-Atrous Spatial Pyramid Pooling (MSC-ASPP), as shown in Figure 5 As shown. Figure 5In the figure, DCconv represents depth-wise separable dilated convolution, GConv represents grouped convolution, Image Pooling represents image pooling, "1×1" represents the size of the convolution kernel is 1×1, "3×3" represents the size of the convolution kernel is 3×3, r represents the dilation rate, and add represents the element-by-element addition operation.

[0086] In this embodiment, element-wise operations are performed on the abstract semantic information features and the edge information features, followed by depthwise separable convolution to obtain initial fused features; global average pooling is performed on the initial fused features in the horizontal and vertical directions respectively to obtain a first key feature corresponding to the target remote sensing image in the horizontal direction and a second key feature corresponding to the target remote sensing image in the vertical direction; two-dimensional convolution, batch normalization, and nonlinear activation are performed on the first key feature and the second key feature to obtain a third key feature and a fourth key feature; one-dimensional convolution transformation and sigmoid normalization are performed on the third key feature and the fourth key feature to obtain an attention map corresponding to the target remote sensing image in the horizontal and vertical directions; the attention map is element-wise multiplied with the initial fused features to obtain an output feature map corresponding to the target remote sensing image; semantic segmentation is performed on the output feature map through a multi-scale cascaded dilated convolution module to obtain a semantic segmentation result corresponding to the target remote sensing image, and the dilation rate combinations of the multi-scale cascaded dilated convolution module are 4, 8, 12, and 16. Compared with the traditional remote sensing image semantic segmentation method, the above method in this embodiment realizes a more effective fusion of abstract semantic information features and edge information features through the edge-guided aggregation module, thereby being able to more accurately locate the edge position in the target remote sensing image to retain more edge detail information; at the same time, the output feature map is semantically segmented through a multi-scale cascaded void convolution module with void ratio combinations of 4, 8, 12, and 16, thereby more effectively extracting the features of the multi-resolution feature map in the target remote sensing image and improving the segmentation ability of targets of different sizes.

[0087] In addition, an embodiment of the present application also proposes a storage medium, on which a remote sensing image semantic segmentation program based on deep learning is stored. When the remote sensing image semantic segmentation program based on deep learning is executed by a processor, the steps of the remote sensing image semantic segmentation method based on deep learning as described above are implemented.

[0088] Reference Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the remote sensing image semantic segmentation device based on deep learning in this application.

[0089] like Figure 6 As shown, the remote sensing image semantic segmentation device based on deep learning proposed in the embodiment of the present application includes:

[0090] The feature extraction module 601 is used to extract features from the target remote sensing image to obtain abstract semantic information features and edge information features;

[0091] A feature fusion module 602 is configured to determine a first key feature corresponding to the target remote sensing image in the horizontal direction and a second key feature corresponding to the target remote sensing image in the vertical direction based on the abstract semantic information feature and the edge information feature;

[0092] A feature operation module 603 is configured to perform feature operation on the first key feature and the second key feature to obtain an output feature map corresponding to the target remote sensing image;

[0093] The semantic segmentation module 604 is used to perform semantic segmentation on the output feature map to obtain a semantic segmentation result corresponding to the target remote sensing image.

[0094] This embodiment performs feature extraction on the target remote sensing image to obtain abstract semantic information features and edge information features; based on the abstract semantic information features and the edge information features, a first key feature corresponding to the target remote sensing image in the horizontal direction and a second key feature corresponding to the target remote sensing image in the vertical direction are determined; feature operations are performed on the first key feature and the second key feature to obtain an output feature map corresponding to the target remote sensing image; and semantic segmentation is performed on the output feature map to obtain a semantic segmentation result corresponding to the target remote sensing image. Compared to traditional remote sensing image semantic segmentation methods, the method of this embodiment determines the first key feature corresponding to the target remote sensing image in the horizontal direction and the second key feature corresponding to the vertical direction based on the abstract semantic information features and edge information features extracted from the target remote sensing image, and then performs semantic segmentation on the output feature map corresponding to the target remote sensing image obtained after feature operations based on the first key feature and the second key feature, thereby being able to more accurately capture the edge details of the target remote sensing image from the obtained semantic segmentation results.

[0095] Based on the first embodiment of the remote sensing image semantic segmentation device based on deep learning mentioned above in the present application, a second embodiment of the remote sensing image semantic segmentation device based on deep learning in the present application is proposed.

[0096] In this embodiment, the feature extraction module 601 is further used to perform feature extraction on the target remote sensing image through a deep convolutional layer to obtain abstract semantic information features, and the abstract semantic information features include category features and attribute features of the object in the target remote sensing image; and perform feature extraction on the target remote sensing image through a shallow convolutional layer and a gated channel attention conversion unit to obtain edge information features, and the edge information features include contour features and texture features of the object in the target remote sensing image.

[0097] Furthermore, the feature extraction module 601 is also used to extract features of the target remote sensing image through a shallow convolutional layer to obtain global features; and perform channel normalization and gate adaptation on the global features through a gated channel attention conversion unit to enhance the key feature channels in the global features and suppress noise boundaries to obtain edge information features.

[0098] Furthermore, the feature fusion module 602 is also used to perform element-level operations on the abstract semantic information features and the edge information features and then perform depth-separable convolution to obtain initial fusion features; and perform global average pooling in the horizontal and vertical directions on the initial fusion features to obtain the first key feature corresponding to the target remote sensing image in the horizontal direction and the second key feature corresponding to the target remote sensing image in the vertical direction.

[0099] Furthermore, the feature operation module 603 is also used to perform two-dimensional convolution, batch normalization and nonlinear activation on the first key feature and the second key feature to obtain a third key feature and a fourth key feature; perform one-dimensional convolution transformation and sigmoid normalization on the third key feature and the fourth key feature to obtain the attention map corresponding to the target remote sensing image in the horizontal and vertical directions; and perform element-wise multiplication of the attention map with the initial fusion feature to obtain an output feature map corresponding to the target remote sensing image.

[0100] Furthermore, the semantic segmentation module 604 is also used to perform semantic segmentation on the output feature map through a multi-scale cascaded dilated convolution module to obtain a semantic segmentation result corresponding to the target remote sensing image, and the dilated rate combinations of the multi-scale cascaded dilated convolution module are 4, 8, 12, and 16.

[0101] Other embodiments or specific implementation methods of the remote sensing image semantic segmentation device based on deep learning in this application can refer to the above-mentioned method embodiments and will not be repeated here.

[0102] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0103] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0104] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0105] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A remote sensing image semantic segmentation method based on deep learning, characterized in that: The method comprises the following steps: Extract features from the target remote sensing image to obtain abstract semantic information features and edge information features; Determining a first key feature corresponding to the target remote sensing image in the horizontal direction and a second key feature corresponding to the target remote sensing image in the vertical direction based on the abstract semantic information feature and the edge information feature; Performing feature operations on the first key feature and the second key feature to obtain an output feature map corresponding to the target remote sensing image; Perform semantic segmentation on the output feature map to obtain a semantic segmentation result corresponding to the target remote sensing image.

2. The remote sensing image semantic segmentation method based on deep learning according to claim 1, characterized in that: The step of extracting features from the target remote sensing image to obtain abstract semantic information features and edge information features includes: Extracting features of the target remote sensing image through a deep convolutional layer to obtain abstract semantic information features, wherein the abstract semantic information features include category features and attribute features of the object in the target remote sensing image; The target remote sensing image is subjected to feature extraction through a shallow convolutional layer and a gated channel attention conversion unit to obtain edge information features, wherein the edge information features include contour features and texture features of the object in the target remote sensing image.

3. The remote sensing image semantic segmentation method based on deep learning according to claim 2, characterized in that: The step of extracting features from the target remote sensing image through a shallow convolutional layer and a gated channel attention conversion unit to obtain edge information features includes: Extracting features of the target remote sensing image through a shallow convolutional layer to obtain global features; The global features are subjected to channel normalization and gate adaptation through a gated channel attention conversion unit to enhance key feature channels in the global features and suppress noise boundaries, thereby obtaining edge information features.

4. The remote sensing image semantic segmentation method based on deep learning according to claim 1, characterized in that: The step of determining a first key feature corresponding to the target remote sensing image in the horizontal direction and a second key feature corresponding to the target remote sensing image in the vertical direction based on the abstract semantic information feature and the edge information feature includes: Performing element-wise operations on the abstract semantic information features and the edge information features and then performing depth-wise separable convolution to obtain initial fused features; Global average pooling is performed on the initial fusion features in the horizontal direction and the vertical direction respectively to obtain a first key feature corresponding to the target remote sensing image in the horizontal direction and a second key feature corresponding to the target remote sensing image in the vertical direction.

5. The remote sensing image semantic segmentation method based on deep learning according to claim 4, characterized in that: The step of performing feature operation on the first key feature and the second key feature to obtain an output feature map corresponding to the target remote sensing image includes: After performing two-dimensional convolution, batch normalization, and nonlinear activation on the first key feature and the second key feature, a third key feature and a fourth key feature are obtained; Performing one-dimensional convolution transformation and sigmoid normalization on the third key feature and the fourth key feature to obtain attention maps corresponding to the target remote sensing image in the horizontal direction and the vertical direction; The attention map is element-wise multiplied by the initial fusion feature to obtain an output feature map corresponding to the target remote sensing image.

6. The remote sensing image semantic segmentation method based on deep learning according to claim 1, characterized in that: The step of performing semantic segmentation on the output feature map to obtain a semantic segmentation result corresponding to the target remote sensing image includes: The output feature map is semantically segmented through a multi-scale cascaded dilated convolution module to obtain a semantic segmentation result corresponding to the target remote sensing image, and the dilated rate combinations of the multi-scale cascaded dilated convolution module are 4, 8, 12, and 16.

7. A remote sensing image semantic segmentation device based on deep learning, characterized in that: The remote sensing image semantic segmentation device based on deep learning includes: Feature extraction module, used to extract features from target remote sensing images to obtain abstract semantic information features and edge information features; a feature fusion module, configured to determine a first key feature corresponding to the target remote sensing image in the horizontal direction and a second key feature corresponding to the target remote sensing image in the vertical direction based on the abstract semantic information feature and the edge information feature; a feature operation module, configured to perform feature operation on the first key feature and the second key feature to obtain an output feature map corresponding to the target remote sensing image; The semantic segmentation module is used to perform semantic segmentation on the output feature map to obtain a semantic segmentation result corresponding to the target remote sensing image.

8. A remote sensing image semantic segmentation device based on deep learning, characterized in that: The device includes: a memory, a processor, and a deep learning-based remote sensing image semantic segmentation program stored in the memory and runnable on the processor, wherein the deep learning-based remote sensing image semantic segmentation program is configured to implement the steps of the deep learning-based remote sensing image semantic segmentation method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a remote sensing image semantic segmentation program based on deep learning is stored on the storage medium. When the remote sensing image semantic segmentation program based on deep learning is executed by a processor, the steps of the remote sensing image semantic segmentation method based on deep learning as described in any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product includes a remote sensing image semantic segmentation program based on deep learning, and when the remote sensing image semantic segmentation program based on deep learning is executed by a processor, the steps of the remote sensing image semantic segmentation method based on deep learning as described in any one of claims 1 to 6 are implemented.

Images