Remote sensing image segmentation method based on Gabor transformation and orthogonal attention

Abstract

The invention discloses a remote sensing image segmentation method based on Gabor transformation and orthogonal attention, and the method comprises the steps: carrying out the feature extraction of a preprocessed target remote sensing image through the combination of Gabor transformation and a deep learning model, and obtaining an initial feature; processing the initial features based on an orthogonal channel attention mechanism to obtain channel attention; performing element-by-element multiplication on the channel attention and the initial feature to obtain a weighted feature map; performing convolution on the initial features through a dynamic separable convolution layer to obtain global information; extracting global information by adopting a global average pooling layer, and generating a weight map through a second Sigmoid activation function; performing full-dimensional dynamic convolution on the initial features, and multiplying the initial features with the weight map to generate optimized features; fusing the weighted feature map and the optimized feature to obtain an enhanced feature map; and converting the enhanced feature map into an image segmentation result. The method can improve the precision and robustness of remote sensing image segmentation in a complex scene.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and more specifically, to a remote sensing image segmentation method based on Gabor transform and orthogonal attention. Background Art

[0002] With the rapid development of remote sensing technology, remote sensing images are increasingly widely used in fields such as environmental monitoring, resource exploration, urban management, and disaster warning. In these applications, image segmentation, as a basic technology, plays a very important role. The purpose of remote sensing image segmentation is to effectively distinguish the target objects in the image from the background or other targets, so as to provide accurate basic data for subsequent analysis and processing.

[0003] However, due to the fact that remote sensing images are often affected by various factors, such as atmospheric scattering, illumination changes, complex terrain, etc., the difference between the target objects and the background in the image is not significant, resulting in the difficulty of traditional image segmentation methods in effectively dealing with complex scenes, especially in cases where the boundaries of the targets are not clear, the texture similarity is high, and the background noise is large. Although existing image segmentation methods have achieved relatively remarkable results in some scenes, they still face many challenges, such as how to improve the segmentation accuracy, how to enhance the robustness of the model, etc.

[0004] Therefore, how to improve the accuracy and robustness of remote sensing image segmentation in complex scenes is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] In view of the above problems, the present invention provides a remote sensing image segmentation method based on Gabor transform and orthogonal attention, so as to at least solve some of the technical problems mentioned in the above background art.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] The present invention provides a remote sensing image segmentation method based on Gabor transform and orthogonal attention, including the following steps:

[0008] Combining Gabor transform and a deep learning model, extract features from the preprocessed target remote sensing image to obtain initial features;

[0009] Based on the orthogonal channel attention mechanism, process the initial features to obtain channel attention; multiply the channel attention and the initial features element by element to obtain a weighted feature map;

[0010] Convolve the initial features through a dynamic separable convolutional layer to obtain global information; use a global average pooling layer to extract the global information, and generate a weight map through a second Sigmoid activation function; multiply the initial features after full-dimensional dynamic convolution by the weight map to generate optimized features;

[0011] Fuse the weighted feature map and the optimized features to obtain an enhanced feature map;

[0012] Convert the enhanced feature map into an image segmentation result.

[0013] Furthermore, the preprocessing includes denoising, enhancement, and color correction.

[0014] Furthermore, combine Gabor transform and a deep learning model to extract features from the preprocessed target remote sensing image to obtain initial features, specifically including:

[0015] Extract features from the preprocessed target remote sensing image based on Gabor transform and a deep learning model respectively to obtain corresponding Gabor features and depth features;

[0016] Fuse the Gabor features and the depth features to obtain initial features.

[0017] Furthermore, process the initial features based on an orthogonal channel attention mechanism to obtain channel attention, specifically including:

[0018] Process the initial features through an initial filter to obtain a filtered feature map;

[0019] Orthogonalize the filtered feature map using the Gram-Schmidt orthogonalization algorithm to obtain orthogonal feature vectors;

[0020] Pass the orthogonal feature vectors through a fully connected layer and a first activation function in sequence to obtain channel attention.

[0021] Furthermore, the conversion of the enhanced feature map into an image segmentation result specifically includes:

[0022] Introduce a diffusion model to decode the enhanced feature map to obtain a denoised feature map;

[0023] Input the denoised feature map into a decoder for image segmentation to obtain a prediction map with the same size as the target remote sensing image.

[0024] Furthermore, the decoder consists of multiple transposed convolutional layers and upsampling layers.

[0025] Furthermore, the cross-entropy loss function and the Dice coefficient loss are used as the total loss function for the segmentation task.

[0026] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a remote sensing image segmentation method based on Gabor transform and orthogonal attention, which has the following beneficial effects:

[0027] By fusing the features extracted by Gabor transform and deep learning, the present invention not only retains the texture information of the image, but also effectively enhances the semantic features, thereby improving the segmentation accuracy of remote sensing images.

[0028] The present invention uses an orthogonal filter and the Gram-Schmidt orthogonalization algorithm to optimize the feature representation, which can effectively remove redundant information and strengthen the key features of the image, thereby improving the performance of the image segmentation model, especially in the segmentation tasks of complex backgrounds and multi-scale targets.

[0029] The technical solutions of the present invention will be further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other accompanying drawings can be obtained according to the provided accompanying drawings without creative efforts.

[0031] Figure 1 It is a schematic diagram of the remote sensing image segmentation method based on Gabor transform and orthogonal attention provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0033] The embodiment of the present invention discloses a remote sensing image segmentation method based on Gabor transform and orthogonal attention. Refer to Figure 1 as shown, and includes the following steps:

[0034] S1. Combine Gabor transform and a deep learning model to extract features from the preprocessed target remote sensing image to obtain initial features;

[0035] S2. Process the initial features based on the ortho-channel attention mechanism to obtain channel attention; multiply the channel attention with the initial features element-wise to obtain a weighted feature map;

[0036] S3. Convolve the initial features through a dynamic separable convolutional layer to obtain global information; use a global average pooling layer to extract the global information and generate a weight map through a second Sigmoid activation function; multiply the initial features after full-dimensional dynamic convolution with the weight map to generate optimized features;

[0037] S4. Fuse the weighted feature map and the optimized features to obtain an enhanced feature map;

[0038] S5. Convert the enhanced feature map into an image segmentation result.

[0039] This method combines the local feature extraction ability of the Gabor transform and the global information aggregation ability of the ortho-channel attention mechanism, aiming to overcome the deficiencies of traditional methods and improve the accuracy and robustness of remote sensing image segmentation, especially in complex scenarios.

[0040] The above notations S1 - S5 are only for convenience of subsequent description and do not limit the execution order of the above steps. Next, each of the above steps will be described in detail.

[0041] In the above step S1, it specifically includes the following steps:

[0042] (1) Preprocess the obtained target remote sensing image; where the preprocessing includes denoising, enhancement, color correction, etc., to ensure that the target remote sensing image has high quality and effective information through preprocessing. The obtained target remote sensing image can be in single-channel (grayscale) or multi-channel (such as RGB, infrared, etc.) format, and the size can be adjusted and cropped according to specific application requirements.

[0043] (2) Combine the Gabor transform and a deep learning model to extract features from the preprocessed target remote sensing image to obtain the initial feature X o ; Specifically:

[0044] 1) Respectively extract features from the preprocessed target remote sensing image based on the Gabor transform and the deep learning model to obtain the corresponding Gabor features and deep features; including:

[0045] ① Extract features from the preprocessed target remote sensing image based on the Gabor transform to obtain the corresponding Gabor features; Specifically:

[0046] As a classic filtering method widely used in image processing, the Gabor transform can effectively extract the frequency-domain features of an image, efficiently capture the local structural features of the image, and capture the texture information of the image at different scales and directions. Remote sensing images often contain rich texture information, which is crucial for target recognition and segmentation. Assuming the input image is I(x, y), the output of the Gabor transform is multiple filtered image feature maps G θ，λ (x, y), which can be expressed as:

[0047]

[0048] where x and y represent the size of the remote sensing image; σ represents the control filter scale; λ represents the wavelength; θ represents the direction angle of the filter; by selecting appropriate σ, λ, and θ, the texture features of the remote sensing image at different scales and directions can be extracted.

[0049] ② Feature extraction is performed on the preprocessed target remote sensing image based on a deep learning model to obtain corresponding deep features; specifically:

[0050] The present invention will also use a deep learning model (such as a convolutional neural network) to extract features from the image. The deep learning model can gradually extract the semantic features of the image from low level to high level through multiple convolutional layers, pooling layers, and fully connected layers. Assuming the output feature of the deep learning model is F DL (I), where I is the input image I(x, y), and F DL (I) is the high-level feature extracted by the network. It is expressed as:

[0051]

[0052] where, represents the deep learning model, which can automatically learn the high-level semantic features of the image.

[0053] 2) Fuse the Gabor features and the deep features to obtain the initial feature X o ; It is expressed as:

[0054] X o = f(G θ，λ (x, y), F DL (I))

[0055] where f(·) is the feature fusion function, which can be concatenation, weighted average, or other adaptive fusion strategies. The fused initial feature X o retains both the fine texture information in the image and contains deep semantic features, providing rich information for subsequent image classification, segmentation, or other tasks. This initial feature X o(Its dimension is C×H×W, where C is the number of channels, and H and W represent the height and width of the image respectively) contains the spatial and spectral information of the image.

[0056] In the embodiments of the present invention, by combining the Gabor transform with a deep learning model, multi-scale local feature extraction of remote sensing images can be performed, thereby providing a more refined feature representation for the segmentation task.

[0057] In the above step S2, in order to effectively extract and compress the information in the initial feature X o while maintaining the key spatial and spectral features of the image, the embodiments of the present invention perform the following operations:

[0058] (1) Process the initial feature X o based on the orthogonal channel attention mechanism to obtain channel attention;

[0059] The orthogonal channel attention mechanism can enhance the features of important regions in the image and suppress the interference of irrelevant information by modeling the attention weights in different directions. By introducing the orthogonal attention mechanism, the embodiments of the present invention can adaptively adjust the attention degree to the target and background regions in the remote sensing image, thereby improving the model's recognition ability for different targets. In addition, the orthogonal channel attention mechanism can also effectively process the complex background and noise in the image and improve the robustness of segmentation.

[0060] In the embodiments of the present invention, specifically:

[0061] 1) Process the initial feature X o through an initial filter to obtain a filtered feature map; the weight matrix of this initial filter is randomly initialized and gradually optimized through training to adapt to the features of the remote sensing image. Specifically:

[0062] Let the initial orthogonal filter be W, and process the initial feature X o to generate a new filtered feature map whose dimension is still C×H×W; expressed as:

[0063]

[0064] 2) Use the Gram - Schmidt orthogonalization algorithm to orthogonalize the filtered feature map to obtain the compressed orthogonal feature vector Z o ; its dimension is C, representing the compressed representation of each channel in the image. This orthogonal feature vector Z o can effectively improve the resolution and accuracy in the remote sensing image segmentation task by removing redundant information and strengthening key features. Specifically expressed as:

[0065]

[0066] Among them, v k is the k-th component of the original input feature vector. In this process, the initial vector is regarded as {v1, v2,..., v C}. Among them, u k is the k-th orthogonal vector, which is calculated by the following formula:

[0067]

[0068] The finally obtained orthogonal feature vector Z o is a C-dimensional feature vector, where each dimension corresponds to the compressed representation of a channel:

[0069] Z o = [u1, u2,..., u C

[0070] In this way, the redundant information in the initial feature X o can be eliminated, and the features of each channel are optimized, thus providing a more concise and efficient feature representation for subsequent image segmentation, object detection, and classification tasks.

[0071] In the remote sensing image segmentation task, remote sensing images usually have complex backgrounds and irregular target areas (such as buildings, roads, vegetation, etc.), and the spatial information and spectral information of these areas are often intertwined. In the embodiments of the present invention, through the combination of an orthogonal filter and the Gram-Schmidt orthogonalization algorithm, distinguishable features can be effectively extracted, and the key discriminative features are retained by compressing the orthogonal feature vector Z o so as to improve the performance of the image segmentation model.

[0072] 3) Pass the orthogonal feature vector through a fully connected layer and a first activation function in sequence to obtain channel attention; specifically:

[0073] The obtained orthogonal feature vector Z o contains the high-dimensional features of the remote sensing image. On this basis, in order to further optimize the feature representation, the orthogonal feature vector Z o is passed through a fully connected layer (Fully ConnectedLayer), and the channel attention A o is obtained through the Sigmoid activation function.

[0074] The channel attention A o ​is a tensor with the shape of C×1×1, used to assign a weight to each channel. These weights can help the model focus on more important feature channels in the remote sensing image during the segmentation process, thereby improving the segmentation accuracy.

[0075] (2) Multiply the channel attention A o element-wise with the initial feature X o to obtain the weighted feature map X; this weighted feature map X contains enhanced key feature information. In this way, the model can strengthen the attention to key regions in the remote sensing image while suppressing the influence of unimportant regions.

[0076] In step S3 above, in the remote sensing image segmentation task, the shapes, scales, and backgrounds of the targets in the image are very complex. Therefore, more flexible and efficient convolution operations are needed to extract detailed features. To further enhance the feature expression ability, the following operations are performed in the embodiments of the present invention:

[0077] (1) Convolve the initial feature X o through a depthwise separable convolutional layer (DWConv) to obtain global information;

[0078] (2) Use a global average pooling layer (GAP) to extract global information and generate a weight map W through a second Sigmoid activation function; this weight map W reflects the importance of each channel in feature fusion;

[0079] (3) After passing the initial feature X o through an omni-dimensional dynamic convolution (ODConv), multiply it with the weight map W to obtain an optimized feature representation, denoted as the optimized feature Expressed as:

[0080]

[0081] where, represents the matrix multiplication operation; GAP represents global average pooling; ReLU represents the ReLU activation function; DWConv represents the depthwise separable convolutional layer. This dynamic convolution method enables the network to more flexibly process multi-scale and complex structure information in the remote sensing image. In this step, the optimized feature o generated after optimizing the initial feature X can more accurately reflect the local and global information of the remote sensing image.

[0082] In step S4 above, fuse the weighted feature map X and the optimized feature to obtain the enhanced feature map X + ; specifically:

[0083] In the remote sensing image segmentation task, feature fusion is crucial for improving the performance of the model. By fusing features from different sources, the complex information of the image can be better represented. In the embodiments of the present invention, the weighted feature map X and the optimized features are fused to integrate the information of each feature map.

[0084] Through the weighted fusion operation, the model can effectively combine features at different levels, making the final segmentation result more accurate. The ultimate goal of this step is to obtain the enhanced feature map X by fusing features from different sources + , for use in subsequent segmentation tasks. Finally, the obtained enhanced feature map X + will be used as the input of the subsequent image segmentation network to generate the final segmentation result.

[0085] In step S5 above, the obtained enhanced feature map X + is used as the input. Combining the characteristics of the segmentation task, it is further refined and optimized to finally achieve the accurate segmentation of remote sensing images. The goal of this step is to convert the enhanced feature map X + into the segmentation result of the image through the decoding process and optimize the segmentation accuracy. Specifically:

[0086] (1) Diffusion model decoding. To improve the accuracy and robustness of the segmentation result, first, a diffusion model is used to decode the enhanced feature map X + . The diffusion model is a generative model that gradually adds noise and learns the denoising process. In the image segmentation task, the diffusion model makes the detailed information in the image better restored by introducing noise and denoising the noise, and makes the model more stable when dealing with complex backgrounds and target shapes.

[0087] The diffusion model decoding process can be represented by the following steps:

[0088] 1) Add noise: Add noise to the enhanced feature map X + to simulate the complexity and background noise in the image and generate a noisy feature map.

[0089] 2) Denoise: The diffusion model gradually restores the key structural information in the feature map X + through a denoising process to obtain a clearer feature representation.

[0090] By introducing the denoising process of the diffusion model, it can help the model restore more accurate segmentation boundaries and structural information in the complex background of remote sensing images.

[0091] (2) After the diffusion model is decoded, the obtained denoised feature map will be passed to the decoder for image segmentation. The task of the decoder is to map the feature map back to the spatial distribution and restore the final segmentation result of the image. The decoder usually consists of multiple deconvolution layers and upsampling layers, which can gradually restore the spatial information in the feature map.

[0092] The decoder will enhance the feature map and decode it into a prediction map with the same size as the target remote sensing image where each pixel value represents the class label or probability at a certain position in the image.

[0093] (3) Apply the segmentation loss function. To optimize the performance of the segmentation model, a suitable loss function needs to be defined to evaluate the difference between the segmentation result output by the model and the ground truth label. In this step, the cross-entropy loss function and the Dice coefficient loss are used as the total loss functions for the segmentation task; among them:

[0094] 1) Cross-Entropy Loss: Measures the difference between the prediction result and the ground truth label, and is suitable for multi-class segmentation tasks.

[0095] The formula of the cross-entropy loss function is expressed as:

[0096]

[0097] where N is the number of classes, y n is the actual label, and is the class probability predicted by the model.

[0098] 2) Dice Coefficient Loss: This loss function is used to evaluate the overlap degree between the predicted region and the ground truth label region, and is especially suitable for segmentation tasks with imbalanced classes.

[0099] The total loss function is the weighted sum of the cross-entropy loss and the Dice loss, and is usually defined as:

[0100] L = αL CE + βL Dice

[0101] where α and β are weight coefficients, which determine the influence degree of the two loss functions in the total loss.

[0102] In the embodiments of the present invention, the training process of the model optimizes the network parameters by minimizing the total loss function L. The training process typically uses a gradient descent algorithm (such as the Adam optimizer) to update the model weights and gradually improve the segmentation performance. Through training, the network can automatically learn how to extract key features from the input image and generate accurate segmentation results in the complex background of remote sensing images.

[0103] The optimized segmentation model will output the final segmentation map S, which contains the accurate segmentation boundaries of the target regions in the remote sensing image. In practical applications, the output segmentation map can be directly used for further target recognition, regional analysis, or other remote sensing data processing tasks. The segmentation result is the classification label of each pixel point obtained after threshold processing. Usually, by setting an appropriate threshold, the probability map is converted into a binary segmentation map S, thus achieving the final image segmentation.

[0104] In summary, the embodiments of the present invention provide a remote sensing image segmentation method based on Gabor transform and orthogonal attention. By combining the Gabor transform and the orthogonal channel attention mechanism, it can effectively improve the segmentation accuracy of complex targets in remote sensing images, especially in scenarios where the target boundaries are unclear, the texture similarity is high, and the background is complex. The introduction of the orthogonal channel attention mechanism enables the model to adaptively focus on the important regions in the image and effectively suppress the influence of noise and background, thereby enhancing the robustness of the model. This method can not only be applied to the conventional segmentation tasks of remote sensing images but also process complex remote sensing image data in multiple fields including land use, urban construction, environmental monitoring, disaster warning, etc.

[0105] The local features extracted by the Gabor transform in the present invention can effectively capture the detailed information of the target object, while the global features obtained by the orthogonal attention mechanism can provide more extensive context information. The combination of the two enables the model to comprehensively analyze the image at different scales and angles, and thus achieve high-precision segmentation.

[0106] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other.

[0107] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image segmentation method based on Gabor transform and orthogonal attention, characterized in that It includes the following steps: Combining the Gabor transform and a deep learning model, extract features from the preprocessed target remote sensing image to obtain initial features; Based on the orthogonal channel attention mechanism, process the initial features to obtain channel attention; multiply the channel attention and the initial features element-wise to obtain a weighted feature map; Convolve the initial features through a dynamic separable convolutional layer to obtain global information; use a global average pooling layer to extract the global information, and generate a weight map through a second Sigmoid activation function; after passing the initial features through a full-dimensional dynamic convolution, multiply them with the weight map to generate optimized features; Fuse the weighted feature map and the optimized features to obtain an enhanced feature map; Convert the enhanced feature map into an image segmentation result.

2. The remote sensing image segmentation method based on Gabor transform and orthogonal attention according to claim 1, characterized in that The preprocessing includes denoising, enhancement, and color correction.

3. The remote sensing image segmentation method based on Gabor transform and orthogonal attention according to claim 1, characterized in that The combining of the Gabor transform and the deep learning model to extract features from the preprocessed target remote sensing image to obtain initial features specifically includes: Based on the Gabor transform and the deep learning model respectively, extract features from the preprocessed target remote sensing image to obtain corresponding Gabor features and depth features; Fuse the Gabor features and the depth features to obtain initial features.

4. The remote sensing image segmentation method based on Gabor transform and orthogonal attention according to claim 1, characterized in that, Based on the orthogonal channel attention mechanism, process the initial features to obtain channel attention, specifically including: Process the initial features through an initial filter to obtain a filtered feature map; Use the Gram-Schmidt orthogonalization algorithm to orthogonalize the filtered feature map to obtain orthogonal feature vectors; Pass the orthogonal feature vectors through a fully connected layer and a first activation function in sequence to obtain channel attention.

5. The remote sensing image segmentation method based on Gabor transform and orthogonal attention according to claim 1, characterized in that, The converting of the enhanced feature map into an image segmentation result specifically includes: Introduce a diffusion model to decode the enhanced feature map to obtain a denoised feature map; Input the denoised feature map into a decoder for image segmentation to obtain a prediction map with the same size as the target remote sensing image.

6. The remote sensing image segmentation method based on Gabor transform and orthogonal attention according to claim 5, characterized in that, The decoder consists of multiple transposed convolutional layers and upsampling layers.

7. The remote sensing image segmentation method based on Gabor transform and orthogonal attention according to claim 1, characterized in that Adopt the cross-entropy loss function and the Dice coefficient loss as the total loss function for the segmentation task.

Images