Remote sensing image segmentation method based on Gabor transform and orthogonal attention

Abstract

The present invention discloses a remote sensing image segmentation method based on Gabor transform and orthogonal attention, comprising: combining Gabor transform and deep learning model to extract features from pre-processed target remote sensing images to obtain initial features; processing the initial features based on the orthogonal channel attention mechanism to obtain channel attention; element-wise multiplication of the channel attention with the initial features to obtain a weighted feature map; convolving the initial features with a dynamic separable convolution layer to obtain global information; extracting global information using a global average pooling layer and generating a weight map through a second sigmoid activation function; subjecting the initial features to full-dimensional dynamic convolution and multiplying them with the weight map to generate optimized features; fusing the weighted feature map with the optimized features to obtain an enhanced feature map; and converting the enhanced feature map into an image segmentation result. This method can improve the accuracy and robustness of remote sensing image segmentation in complex scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and more particularly to a remote sensing image segmentation method based on Gabor transformation and orthogonal attention. Background Art

[0002] With the rapid development of remote sensing technology, remote sensing images are increasingly being used in fields such as environmental monitoring, resource exploration, urban management, and disaster warning. Image segmentation, as a foundational technology, plays a crucial role in these applications. The goal of remote sensing image segmentation is to effectively distinguish the target object from the background or other objects in an image, thereby providing accurate basic data for subsequent analysis and processing.

[0003] However, because remote sensing images are often affected by various factors, such as atmospheric scattering, illumination variations, and complex terrain, the difference between the target object and the background in the image is not significant. This makes traditional image segmentation methods difficult to effectively handle complex scenes, especially when the target boundaries are unclear, texture similarity is high, and background noise is high. Although existing image segmentation methods have achieved relatively significant results in some scenarios, they still face many challenges, such as how to improve segmentation accuracy and enhance model robustness.

[0004] Therefore, how to improve the accuracy and robustness of remote sensing image segmentation in complex scenes is an urgent problem that technicians in this field need to solve. Summary of the Invention

[0005] In view of the above problems, the present invention provides a remote sensing image segmentation method based on Gabor transform and orthogonal attention, so as to at least solve some of the technical problems mentioned in the above background technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The present invention provides a remote sensing image segmentation method based on Gabor transform and orthogonal attention, comprising the following steps:

[0008] Combining Gabor transform and deep learning model, feature extraction is performed on the preprocessed target remote sensing image to obtain initial features;

[0009] Processing the initial features based on the orthogonal channel attention mechanism to obtain channel attention; performing element-wise multiplication of the channel attention and the initial features to obtain a weighted feature map;

[0010] Convolving the initial features through a dynamic separable convolution layer to obtain global information; extracting the global information through a global average pooling layer, and generating a weight map through a second Sigmoid activation function; subjecting the initial features to full-dimensional dynamic convolution and multiplying them with the weight map to generate optimized features;

[0011] Fusing the weighted feature map and the optimized feature map to obtain an enhanced feature map;

[0012] The enhanced feature map is converted into an image segmentation result.

[0013] Furthermore, the preprocessing includes denoising, enhancement and color correction.

[0014] Furthermore, the combination of Gabor transform and deep learning model is used to extract features from the pre-processed target remote sensing image to obtain initial features, specifically including:

[0015] Based on Gabor transform and deep learning model respectively, feature extraction is performed on the preprocessed target remote sensing image to obtain the corresponding Gabor features and depth features;

[0016] The Gabor feature and the depth feature are fused to obtain an initial feature.

[0017] Furthermore, the initial features are processed based on the orthogonal channel attention mechanism to obtain channel attention, specifically including:

[0018] Processing the initial features through an initial filter to obtain a filtered feature map;

[0019] Orthogonalizing the filtering feature map using a Gram-Schmidt orthogonalization algorithm to obtain an orthogonal feature vector;

[0020] The orthogonal feature vector is sequentially passed through the fully connected layer and the first activation function to obtain channel attention.

[0021] Furthermore, converting the enhanced feature map into an image segmentation result specifically includes:

[0022] Introducing a diffusion model to decode the enhanced feature map to obtain a denoised feature map;

[0023] The denoised feature map is input into a decoder for image segmentation to obtain a prediction map with the same size as the target remote sensing image.

[0024] Furthermore, the decoder is composed of multiple deconvolution layers and upsampling layers.

[0025] Furthermore, the cross entropy loss function and Dice coefficient loss are used as the total loss function for the segmentation task.

[0026] It can be seen from the above technical solution that, compared with the prior art, the present invention discloses a remote sensing image segmentation method based on Gabor transform and orthogonal attention, which has the following beneficial effects:

[0027] By fusing Gabor transform and features extracted by deep learning, the present invention not only retains the texture information of the image, but also effectively enhances the semantic features, thereby improving the segmentation accuracy of remote sensing images.

[0028] The present invention adopts orthogonal filters and the Gram-Schmidt orthogonalization algorithm to optimize feature representation, which can effectively remove redundant information and enhance the key features of the image, thereby improving the performance of the image segmentation model, especially in the segmentation tasks of complex backgrounds and multi-scale targets.

[0029] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0031] Figure 1 Schematic diagram of a remote sensing image segmentation method based on Gabor transform and orthogonal attention provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0033] The embodiment of the present invention discloses a remote sensing image segmentation method based on Gabor transform and orthogonal attention, see Figure 1 As shown, the following steps are included:

[0034] S1. Combine Gabor transform and deep learning model to extract features of the pre-processed target remote sensing image to obtain initial features;

[0035] S2. Process the initial features based on the orthogonal channel attention mechanism to obtain channel attention; multiply the channel attention with the initial features element by element to obtain a weighted feature map;

[0036] S3, convolving the initial features through a dynamic separable convolution layer to obtain global information; using a global average pooling layer to extract global information, and generating a weight map through a second Sigmoid activation function; after the initial features undergo full-dimensional dynamic convolution, multiplying them with the weight map to generate optimized features;

[0037] S4, fusing the weighted feature map and the optimized feature to obtain an enhanced feature map;

[0038] S5. Convert the enhanced feature map into image segmentation results.

[0039] This method combines the local feature extraction capability of Gabor transform with the global information aggregation capability of orthogonal channel attention mechanism, aiming to overcome the shortcomings of traditional methods and improve the accuracy and robustness of remote sensing image segmentation, especially in complex scenes.

[0040] The above-mentioned marks S1-S5 are only for the convenience of subsequent description and do not limit the execution order of the above-mentioned steps. Next, each of the above-mentioned steps will be described in detail.

[0041] In the above step S1, the following steps are specifically included:

[0042] (1) Preprocess the acquired target remote sensing image; preprocessing includes denoising, enhancement, and color correction, ensuring that the target remote sensing image has high quality and effective information. The acquired target remote sensing image can be in single-channel (grayscale) or multi-channel (such as RGB, infrared, etc.) format, and the size can be adjusted and cropped according to specific application requirements.

[0043] (2) Combine Gabor transform and deep learning model to extract features of the preprocessed target remote sensing image and obtain the initial feature X o ;Specifically:

[0044] 1) Based on Gabor transform and deep learning model, feature extraction is performed on the pre-processed target remote sensing image to obtain corresponding Gabor features and deep features; including:

[0045] ① Based on Gabor transform, feature extraction is performed on the pre-processed target remote sensing image to obtain the corresponding Gabor features; specifically:

[0046] As a classic filtering method widely used in image processing, Gabor transform can effectively extract the frequency domain features of the image, effectively capture the local structural features of the image, and capture the texture information of the image at different scales and directions. Remote sensing images often contain rich texture information, which is crucial for target recognition and segmentation. Assuming that the input image is I(x,y), the output of Gabor transform is multiple filtered image feature maps G θ，λ (x, y), can be expressed as:

[0047]

[0048] Among them, x and y represent the size of the remote sensing image; σ represents the scale of the control filter; λ represents the wavelength; θ represents the direction angle of the filter; by selecting appropriate σ, λ, and θ, texture features of different scales and directions in the remote sensing image can be extracted.

[0049] ② Based on the deep learning model, feature extraction is performed on the pre-processed target remote sensing image to obtain the corresponding deep features; specifically:

[0050] The present invention also uses a deep learning model (such as a convolutional neural network) to extract features from images. The deep learning model can gradually extract the semantic features of the image from low to high levels through multiple convolutional layers, pooling layers, and fully connected layers. Assume that the output feature of the deep learning model is F DL (I), where I is the input image (x, y), and F DL (I) is the high-level feature extracted by the network. It is expressed as:

[0051]

[0052] in, Represents a deep learning model that can automatically learn high-level semantic features of images.

[0053] 2) Fuse Gabor features and depth features to obtain the initial feature X o ; expressed as:

[0054] X o =f(G θ，λ (x, y), F DL (I))

[0055] Among them, f(·) is the feature fusion function, which can be splicing, weighted average or other adaptive fusion strategies. The initial feature X after fusion o It not only retains the fine texture information in the image, but also contains deep semantic features, providing rich information for subsequent image classification, segmentation or other tasks. o(Its dimensions are C×H×W, where C is the number of channels, H and W represent the height and width of the image, respectively) and contain the spatial and spectral information of the image.

[0056] In an embodiment of the present invention, by combining Gabor transform with a deep learning model, multi-scale local feature extraction can be performed on remote sensing images, thereby providing a more refined feature representation for segmentation tasks.

[0057] In the above step S2, in order to effectively extract and compress the initial feature X o In order to preserve the key spatial and spectral features of the image, the embodiment of the present invention performs the following operations:

[0058] (1) Based on the orthogonal channel attention mechanism, the initial feature X o Processing to gain channel attention;

[0059] By modeling attention weights in different directions, the orthogonal channel attention mechanism can enhance the features of important image regions while suppressing interference from irrelevant information. By introducing the orthogonal attention mechanism, embodiments of the present invention can adaptively adjust the focus on target and background areas in remote sensing images, thereby improving the model's ability to recognize different targets. Furthermore, the orthogonal channel attention mechanism can effectively handle complex backgrounds and noise in images, improving segmentation robustness.

[0060] In the embodiment of the present invention, specifically:

[0061] 1) The initial feature X is filtered by the initial filter o Processing is performed to obtain a filter feature map; the weight matrix of the initial filter is randomly initialized and gradually optimized through training to adapt to the characteristics of the remote sensing image. Specifically:

[0062] Assume that the initial orthogonal filter is W, and the initial feature X o Processing to generate a new filter feature map Its dimensions are still C×H×W; it can be expressed as:

[0063]

[0064] 2) Use Gram-Schmidt orthogonalization algorithm to filter feature map Orthogonalize to obtain the compressed orthogonal eigenvector Z o ; Its dimension is C, which represents the compressed representation of each channel in the image. The orthogonal eigenvector Z o By removing redundant information and strengthening key features, the resolution and accuracy of remote sensing image segmentation tasks can be effectively improved. Specifically, it can be expressed as:

[0065]

[0066] Among them, v k is the kth component of the original input feature vector (in this process, the initial vector is considered as {v1, v2, ..., v C}. k is the kth orthogonal vector and is calculated as follows:

[0067]

[0068] The final orthogonal eigenvector Z o is a C-dimensional feature vector, where each dimension corresponds to the compressed representation of a channel:

[0069] Z o =[u1, u2, ..., u C ]

[0070] In this way, the initial feature X can be eliminated o The redundant information in the image is eliminated, and the features of each channel are optimized, thus providing a more concise and efficient feature representation for subsequent image segmentation, target detection and classification tasks.

[0071] In the remote sensing image segmentation task, remote sensing images usually have complex backgrounds and irregular target areas (such as buildings, roads, vegetation, etc.), and the spatial information and spectral information of these areas are often intertwined. In the embodiment of the present invention, by combining the orthogonal filter and the Gram-Schmidt orthogonalization algorithm, it is possible to effectively extract the features with recognition, and by compressing the orthogonal feature vector Z o Preserve key discriminative features to improve the performance of image segmentation models.

[0072] 3) Pass the orthogonal feature vector through the fully connected layer and the first activation function in sequence to obtain channel attention; specifically:

[0073] The orthogonal eigenvector Z obtained above o Contains the high-dimensional features of remote sensing images. On this basis, in order to further optimize the representation of features, the orthogonal feature vector Z o After a fully connected layer (Fully ConnectedLayer), the channel attention A is obtained through the Sigmoid activation function o .

[0074] Channel Attention A oIt is a tensor of shape C×1×1 that is used to assign a weight to each channel. These weights can help the model focus on the more important feature channels in the remote sensing image during the segmentation process, thereby improving segmentation accuracy.

[0075] (2) Channel attention A o With the initial feature X o Perform element-by-element multiplication to obtain a weighted feature map X, which contains enhanced key feature information. In this way, the model can strengthen the focus on key areas in remote sensing images while suppressing the influence of unimportant areas.

[0076] In step S3 above, in the remote sensing image segmentation task, the shape, scale, and background of the target in the image are very complex. Therefore, a more flexible and efficient convolution operation is needed to extract detailed features. To further enhance the expressiveness of features, the following operations are performed in the embodiment of the present invention:

[0077] (1) The initial feature X is processed by the dynamic separable convolution layer (DWConv) o Perform convolution to obtain global information;

[0078] (2) A global average pooling layer (GAP) is used to extract global information and a weight map W is generated through a second Sigmoid activation function; the weight map W reflects the importance of each channel in feature fusion;

[0079] (3) The initial feature X o After full-dimensional dynamic convolution (ODConv), it is multiplied with the weight map W to obtain the optimized feature representation, which is recorded as the optimized feature Expressed as:

[0080]

[0081] in, Represents matrix multiplication operation; GAP represents global average pooling; ReLU represents ReLU activation function; DWConv represents dynamic separable convolution layer. This dynamic convolution method enables the network to more flexibly process multi-scale and complex structural information in remote sensing images. In this step, the initial feature X o Optimized features generated after optimization It can more accurately reflect the local and global information of remote sensing images.

[0082] In the above step S4, the weighted feature map X and the optimized feature Perform fusion to obtain enhanced feature map X + ;Specifically:

[0083] In remote sensing image segmentation tasks, feature fusion is crucial to improving the performance of the model. By fusing features from different sources, the complex information of the image can be better expressed. In this embodiment of the present invention, the weighted feature map X and the optimized feature map X are combined to form a fusion matrix. Fusion can integrate the information of each feature map.

[0084] Through the weighted fusion operation, the model can effectively combine features at different levels, making the final segmentation result more accurate. The ultimate goal of this step is to obtain an enhanced feature map X by fusing features from different sources. + , for subsequent segmentation tasks. Finally, the enhanced feature map X + It will be used as the input of the subsequent image segmentation network to generate the final segmentation result.

[0085] In the above step S5, the enhanced feature map X obtained above is + As input, combined with the characteristics of the segmentation task, further refinement and optimization are performed to ultimately achieve accurate segmentation of remote sensing images. The goal of this step is to enhance the feature map X through the decoding process. + Convert it into image segmentation results and optimize the segmentation accuracy. Specifically:

[0086] (1) Diffusion model decoding. In order to improve the accuracy and robustness of the segmentation results, the diffusion model is first used to enhance the feature map X + The diffusion model is a generative model that gradually adds noise and learns the denoising process. In image segmentation tasks, the diffusion model, by introducing and denoising noise, can better recover image details and make the model more stable when dealing with complex backgrounds and object shapes.

[0087] The diffusion model decoding process can be represented as the following steps:

[0088] 1) Add noise: Enhance the feature map X + Noise is added to simulate the complexity and background noise in the image and generate a noisy feature map.

[0089] 2) Denoising: The diffusion model gradually restores the feature map X through a denoising process + The key structural information in the image can be obtained to obtain a clearer feature representation.

[0090] By introducing the denoising process of the diffusion model, the model can be helped to restore more accurate segmentation boundaries and structural information in the complex background of remote sensing images.

[0091] (2) After the diffusion model is decoded, the resulting denoised feature map is passed to the decoder for image segmentation. The decoder's task is to map the feature map back to its spatial distribution and recover the final image segmentation result. The decoder typically consists of multiple deconvolution layers and upsampling layers, which can gradually recover the spatial information in the feature map.

[0092] The decoder will enhance the feature map Decoded into a prediction image with the same size as the target remote sensing image Each pixel value represents the class label or probability of a certain position in the image.

[0093] (3) Apply the segmentation loss function. In order to optimize the performance of the segmentation model, it is necessary to define a suitable loss function to evaluate the difference between the segmentation results output by the model and the true label. In this step, the cross-entropy loss function (Cross-Entropy Loss) and the Dice coefficient loss (Dice Loss) are used as the total loss function of the segmentation task; where:

[0094] 1) Cross-Entropy Loss: measures the difference between the predicted result and the true label, suitable for multi-category segmentation tasks.

[0095] The cross entropy loss function formula is expressed as:

[0096]

[0097] Where N is the number of categories, y n is the actual label, The class probabilities predicted by the model.

[0098] 2) Dice coefficient loss (Dice Loss): This loss function is used to evaluate the degree of overlap between the predicted area and the true label area, and is particularly suitable for segmentation tasks of unbalanced categories.

[0099] The total loss function is the weighted sum of cross entropy loss and Dice loss, usually defined as:

[0100] L=αL CE +βL Dice

[0101] Among them, α and β are weight coefficients that determine the influence of the two loss functions on the total loss.

[0102] In embodiments of the present invention, the model training process optimizes network parameters by minimizing a total loss function, L. This training process typically employs a gradient descent algorithm (such as the Adam optimizer) to update model weights and gradually improve segmentation performance. Through training, the network automatically learns how to extract key features from the input image and generate accurate segmentation results within the complex background of remote sensing images.

[0103] The optimized segmentation model outputs a final segmentation map S, which accurately defines the segmentation boundaries of the target region in the remote sensing image. In practical applications, this output segmentation map can be directly used for further target recognition, regional analysis, or other remote sensing data processing tasks. The segmentation result is then thresholded to obtain a classification label for each pixel. Typically, by setting an appropriate threshold, the probability map is converted into a binary segmentation map S, achieving final image segmentation.

[0104] In summary, the embodiments of the present invention provide a remote sensing image segmentation method based on Gabor transform and orthogonal attention. By combining Gabor transform with the orthogonal channel attention mechanism, the segmentation accuracy of complex targets in remote sensing images can be effectively improved, especially in scenes with unclear target boundaries, high texture similarity, and complex backgrounds. The introduction of the orthogonal channel attention mechanism enables the model to adaptively focus on important areas in the image and effectively suppress the influence of noise and background, thereby enhancing the robustness of the model. This method can not only be applied to conventional segmentation tasks of remote sensing images, but can also process complex remote sensing image data in multiple fields including land use, urban construction, environmental monitoring, disaster warning, etc.

[0105] The local features extracted by the Gabor transform effectively capture detailed information about the target object, while the global features obtained by the orthogonal attention mechanism provide broader contextual information. The combination of the two enables the model to comprehensively analyze images at different scales and angles, achieving high-precision segmentation.

[0106] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0107] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image segmentation method based on Gabor transform and orthogonal attention, characterized in that: The steps include: Combining Gabor transform and deep learning model, feature extraction is performed on the preprocessed target remote sensing image to obtain initial features; Processing the initial features based on the orthogonal channel attention mechanism to obtain channel attention; performing element-wise multiplication of the channel attention and the initial features to obtain a weighted feature map; Convolving the initial features through a dynamic separable convolution layer to obtain global information; extracting the global information through a global average pooling layer, and generating a weight map through a second Sigmoid activation function; subjecting the initial features to full-dimensional dynamic convolution and multiplying them with the weight map to generate optimized features; Fusing the weighted feature map and the optimized feature map to obtain an enhanced feature map; Converting the enhanced feature map into an image segmentation result; The converting the enhanced feature map into an image segmentation result specifically includes: Introducing a diffusion model to decode the enhanced feature map to obtain a denoised feature map; The denoised feature map is input into a decoder for image segmentation to obtain a prediction map with the same size as the target remote sensing image.

2. The remote sensing image segmentation method based on Gabor transform and orthogonal attention according to claim 1, characterized in that: The preprocessing includes denoising, enhancement and color correction.

3. The remote sensing image segmentation method based on Gabor transform and orthogonal attention according to claim 1, characterized in that: The Gabor transform and the deep learning model are combined to extract features from the pre-processed target remote sensing image to obtain initial features, specifically including: Based on Gabor transform and deep learning model respectively, feature extraction is performed on the preprocessed target remote sensing image to obtain the corresponding Gabor features and depth features; The Gabor feature and the depth feature are fused to obtain an initial feature.

4. The remote sensing image segmentation method based on Gabor transform and orthogonal attention according to claim 1, characterized in that: The initial features are processed based on the orthogonal channel attention mechanism to obtain channel attention, specifically including: Processing the initial features through an initial filter to obtain a filtered feature map; Orthogonalizing the filtering feature map using a Gram-Schmidt orthogonalization algorithm to obtain an orthogonal feature vector; The orthogonal feature vector is sequentially passed through the fully connected layer and the first activation function to obtain channel attention.

5. The remote sensing image segmentation method based on Gabor transform and orthogonal attention according to claim 1, characterized in that: The decoder consists of multiple deconvolution layers and upsampling layers.

6. The remote sensing image segmentation method based on Gabor transform and orthogonal attention according to claim 1, characterized in that: The cross entropy loss function and Dice coefficient loss are used as the total loss function for the segmentation task.

Images