Remote sensing image land class segmentation processing method based on improved SAM model

Abstract

The application discloses a remote sensing image land class segmentation processing method based on an improved SAM model, which comprises the following steps: S1, data preprocessing; S2, SAM model construction; S3, prompt generation and preliminary segmentation; S4, semantic classification and end-to-end training; and S5, post-processing optimization. On one hand, the application effectively solves problems such as boundary blur, over-segmentation, spectral aliasing and the like, the average intersection over union of land class segmentation can reach more than 85%, the F1 score of land class boundary is improved by about 20%, and the land class segmentation result with a semantic label can be output; on the other hand, the memory occupation and inference time of the model are greatly reduced, the zero sample characteristic of the SAM model is retained, the model does not need to be retrained for different scenes, the adaptability and generalization ability of the model are significantly improved, in addition, the end-to-end training strategy can complete model fine tuning in combination with a small amount of labeled data, and the data labeling cost and model training period are greatly reduced.

Description

Technical Field

[0001] This invention belongs to the field of digital image processing technology, specifically relating to a method for land cover segmentation processing of remote sensing images based on an improved SAM model. Background Technology

[0002] The rapid development of remote sensing technology has promoted the large-scale application of high-resolution remote sensing imagery in fields such as land resource management, urban construction, and emergency disaster reduction. As a core task of remote sensing data processing, land cover classification in remote sensing imagery aims to accurately assign pixels or regions in the image to the corresponding land cover categories, providing accurate spatial data support for subsequent decision analysis. Its classification accuracy and processing efficiency directly determine the application value of remote sensing data.

[0003] Traditional methods for land cover segmentation in remote sensing images mainly include maximum likelihood classification, support vector machine (SVM), and random forest. These methods have simple computational logic and low deployment threshold, but they are not robust to complex scenes and cannot effectively solve the problems of spectral aliasing and geometric distortion that are common in remote sensing images. The segmentation accuracy is difficult to meet the actual application requirements of 0.5m-level high-resolution remote sensing images.

[0004] In recent years, deep learning methods have become the mainstream technology for remote sensing image segmentation. Classic models such as U-Net, DeepLab, and MaskR-CNN extract multi-scale features of images through convolutional neural networks (CNNs), significantly improving segmentation accuracy compared to traditional methods. However, these deep learning methods have inherent drawbacks: first, they are highly dependent on labeled data, requiring large-scale, high-quality pixel-level labeled samples for model training, resulting in high labeling costs and long cycles; second, their generalization ability is limited, exhibiting poor adaptability to remote sensing images of different resolutions and spectral bands, leading to high costs for retraining and optimization, and making it difficult to meet the large-scale processing needs of remote sensing images across multiple platforms.

[0005] Currently, the SAM zero-shot segmentation model on the market breaks through the dependence of traditional segmentation models on labeled data. It can segment any object based on lightweight cues such as points, boxes, and masks, and has strong generalization and human-computer interaction, providing a new technical approach for land cover segmentation in remote sensing images. However, the original SAM model is designed for natural images, and its direct application to land cover segmentation in remote sensing images has obvious limitations. Current related technologies mostly focus on the single optimization of specific network architectures. For example, the "A Remote Sensing Image Segmentation Processing Method Based on Deep Learning" disclosed in patent CN114049338A only improves the convolutional neural network and does not fully explore the zero-shot advantage of the SAM model, nor does it specifically adapt the SAM model to the characteristics of remote sensing images such as multispectral, large scale, and strong geometric features. Therefore, it has the following shortcomings: 1. Poor adaptability to multispectral channels of remote sensing images, unable to fully extract feature information of remote sensing-specific bands such as near-infrared, and low processing efficiency for large-scale remote sensing images of 5000×5000 pixels, with a single image inference time exceeding 3 seconds and memory usage exceeding 16G. The segmentation speed and memory usage cannot meet the real-time requirements of large-scale remote sensing monitoring. 2. Technical issues such as poor multispectral channel adaptability, low efficiency in large-scale image processing, blurred boundary segmentation, and lack of semantic classification capabilities result in low accuracy of the model in identifying land use regions as a whole, with the average intersection-over-union ratio (mIoU, a core indicator for measuring segmentation accuracy) being below 80%. 3. The Transformer architecture focuses on capturing the global contextual features of the image, ignoring the geometric features of remote sensing images (such as the sharpness of land cover edges and the diversity of surface textures). During the segmentation process, it is prone to problems such as blurred boundaries and oversegmentation. The F1 score of land cover boundaries is only about 65%, which makes it impossible to achieve fine segmentation of land cover boundaries. 4. It lacks a semantic supervision mechanism for land use types, and can only output the target mask. It cannot directly output the land category label, and requires an additional post-processing step to complete the matching of the mask and the land category, which increases the operational complexity and reduces the overall segmentation efficiency.

[0006] In summary, no existing technology has emerged that can balance segmentation accuracy, processing efficiency, and generalization ability in remote sensing image land cover segmentation. Improving the SAM model to adapt it to the characteristics of remote sensing images, solving its problems such as insufficient utilization of multispectral channels, blurred boundary segmentation, and lack of semantic classification ability, while retaining its zero-sample advantage, has become a key technical challenge that urgently needs to be addressed in this field. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide an improved method for land cover segmentation processing of remote sensing images based on an improved SAM model.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for land cover segmentation processing of remote sensing images based on an improved SAM model includes the following steps: S1, Data Preprocessing Eliminate radiometric errors, atmospheric interference, and geometric distortions in the original remote sensing image imaging process to generate multispectral images; S2 and SAM model construction First, the multispectral band features of the input image are adaptively weighted and fused using a channel attention mechanism. Then, a geometric attention layer is embedded in the Transformer layer, using edge features extracted by Canny edge detection as priors and incorporating them into the Transformer's attention calculation. The image encoder front end introduces a multispectral fusion module to reduce the dimensionality of the feature maps of each channel of the input image. Then, an adaptive weight for each channel is calculated using a Softmax channel attention mechanism. Based on the weights, features from different bands such as visible light and infrared are fused to generate a unified multispectral fused feature map. The Transformer layer is divided into an extraction segment and a learning segment. The extraction segment is used to extract low-level features, textures, edges, and local structures. The learning segment is used to learn high-level semantics and global context features. The feature map output from the Transformer learning segment is linearly transformed to obtain a key and a value, where Key = W_K × F; Value = W_V × F, where W_K and W_V are the key and value projection matrices, and F is the output feature of the Transformer learning segment. S3. Prompt Generation and Preliminary Segmentation Segmentation prompts are generated through user interaction or automatic detection algorithms. The SAM model completes preliminary segmentation based on the segmentation prompts and outputs a binary mask of land use. S4. Semantic Classification and End-to-End Training Based on the extraction of land category regional features using binary masks, land category semantic classification is completed through a lightweight classification head. Then, an end-to-end training strategy is adopted, and the SAM model is fine-tuned by combining mask loss and classification loss. S5, Post-processing optimization The segmentation results are smoothed by using conditional random fields or morphological operations, and the final land class segmentation results with semantic labels are output.

[0009] Preferably, in step S2, the Canny operator is used to perform edge detection on the preprocessed remote sensing image, outputting a binarized edge feature map E. This edge feature map is then linearly transformed to obtain the attention query Query, where Query = W_Q × E, and W_Q is the query projection matrix; E is the Canny edge feature map. Through the above calculation, the Canny edge prior information is integrated into the Transformer attention mechanism, enhancing the model's ability to capture land cover boundaries in the remote sensing image. Furthermore, the geometric attention calculation uses scaled dot product attention, as shown in the following formula: Attention(Q,K,V) = Softmax((Q×K^T) / √d_k)×V, where d_k is the feature dimension of the Key; √d_k is the scaling factor.

[0010] In some specific implementations, the specific calculation process includes the following: 1) Perform global average pooling on the reduced-dimensional multispectral band feature map in the spatial dimension to obtain the global feature description vector of each channel; 2) Obtain the initial weight coefficients of each channel by performing a nonlinear transformation on the global feature description vector; 3) Softmax normalize the initial weight coefficients to obtain channel adaptive weights that are non-negative and sum to 1; And the calculation formula is as follows: w k =Softmax(f k )=exp(f k ) / Σ(exp(fᵢ)) where, w k For the adaptive weights of the k-th band channel, f k This represents the feature value of the k-th channel after global pooling and transformation. Based on the aforementioned weights, features from different bands such as visible light and infrared are weighted and fused to generate a unified multispectral fusion feature map. The fusion method involves adaptively multiplying the corresponding band feature maps channel by channel using adaptive weights, and then concatenating or accumulating them element by element according to the channel dimension. The final output is a multispectral fusion feature map with unified dimensions and enhanced features.

[0011] According to a specific embodiment and preferred aspect of the present invention, in step S1, the input raw remote sensing image is sequentially subjected to radiometric calibration, atmospheric correction and geometric correction to eliminate radiometric errors, atmospheric interference and geometric distortion in the image imaging process, generate a standardized multispectral image, and at the same time calculate the NDVI normalized vegetation index, and integrate the NDVI normalized vegetation index and GLCM gray-level co-occurrence matrix texture features as auxiliary channels into the multispectral image.

[0012] Preferably, radiometric calibration utilizes calibration coefficients provided by satellites / UAVs to perform linear / polynomial conversions on each pixel of each image band, obtaining standardized radiometric data. Atmospheric correction employs tools such as ENVI / ArcGIS, based on an atmospheric correction model, inputting parameters such as imaging time, observation angle, and atmospheric conditions to remove the influence of aerosols and water vapor, obtaining a true surface reflectance image. Geometric correction uses a high-precision DEM or reference map as a benchmark, selecting ground control points (GCPs), and performing geometric correction and resampling through a polynomial / rational function model to ensure that image coordinates are consistent with real geographic coordinates, guaranteeing the accuracy of ground feature boundaries and spatial positions. In short, radiometric calibration is used to convert the raw DN values ​​of remote sensing images into physically meaningful radiance / apparent reflectance values, eliminating errors caused by sensor gain and offset. Its implementation involves using calibration coefficients provided by satellites / UAVs to perform linear / polynomial conversions on each pixel of each image band, obtaining standardized radiometric data, ensuring numerical consistency across different time phases and different sensor images. Atmospheric correction is used to eliminate radiation distortion caused by atmospheric scattering and absorption in remote sensing signals, restoring the true reflectance of the land surface. It is implemented using tools such as ENVI / ArcGIS, based on atmospheric correction models (e.g., FLAASH, 6S), inputting parameters such as imaging time, observation angle, and atmospheric conditions to remove the influence of aerosols and water vapor, obtaining an image of the true reflectance of the land surface and improving the accuracy of land cover spectral characteristics. Geometric correction is used to eliminate geometric positional offsets in remote sensing images caused by terrain undulations, sensor attitude, and Earth curvature, achieving precise geospatial matching. It is implemented using a high-precision DEM or reference map as a benchmark, selecting ground control points (GCPs), and performing geometric correction and resampling through a polynomial / rational function model to ensure that the image coordinates are consistent with the actual geographic coordinates, guaranteeing the accuracy of feature boundaries and spatial positions.

[0013] In some specific implementations, the NDV calculation formula is: NDVI=(NIR−R) / (NIR+R), where NIR is the reflectance value of the near-infrared band; R is the reflectance value of the red band (red light band). The Normalized Difference Vegetation Index (NDVI) is calculated pixel-by-pixel according to the above formula; the result range is -1 to +1, with positive values ​​representing vegetation, and higher values ​​indicating more abundant vegetation. In this application, the NDVI serves as an auxiliary feature channel integrated into the multispectral image, enhancing the distinction between land types such as cultivated land and forest land, and improving the model's segmentation accuracy.

[0014] According to another specific embodiment and preferred aspect of the present invention, in step S4, after the preliminary segmentation module, a lightweight classification head is introduced, and a pre-trained ResNet model is used as the classification backbone network. The land use regional features are extracted from the land use binary mask through global average pooling, and the regional features are input into the fully connected layer to complete the semantic classification of land use types such as cultivated land, forest land, water body, buildings, and bare land. And / or, adopt an end-to-end training strategy, and fine-tune the improved SAM model with a small amount of remotely sensed image annotation data. Use the SAM model mask loss as the segmentation loss and the cross-entropy loss as the classification loss. Design the total loss function as L = L_mask + αL_class, where α is the loss weight and its value range is 0.4 to 0.6. Jointly optimize the model through the total loss function.

[0015] According to another specific implementation and preferred aspect of the present invention, the ResNet model includes an initial convolutional layer, multiple residual block groups, a global average pooling layer, and a fully connected layer. Each residual block group adopts a double-layer convolutional structure, and the residual calculation is H(x) = F(x) + x, where F(x) is the convolutional mapping residual function and x is the input shortcut feature; the preliminary segmentation mask feature F_mask is obtained through global average pooling to get F_gap; the input ResNet model obtains the deep semantic feature F_res; then it is mapped to F_fc = W_fc・F_res + b_fc through the fully connected layer; finally, the class probability P = Softmax(F_fc) is output through Softmax.

[0016] In addition, in step S5, when using the conditional random field, boundary optimization is achieved by minimizing the energy function, and the energy function is: E(X) = Σ_iψ_u(x_i) + Σ_i<jψ_p(x_i,x_j), where the binary potential function uses a Gaussian kernel, the smoothing parameter σ takes values from 3 to 5, and the number of iterations is 5 to 10 times to eliminate the segmentation error caused by image noise, improve the refinement degree of the segmentation result, and output the final land class segmentation result of the remotely sensed image with semantic labels.

[0017] At the same time in step S5, when using morphological operations, boundary smoothing is achieved through closing operation A・B = (A⊕B)⊖B to eliminate noise and burrs, make the segmentation boundary more regular and smooth, and improve the refinement degree of the result.

[0018] In addition, the above-mentioned land class segmentation processing method of remotely sensed images based on the improved SAM model includes remotely sensed images, a data preprocessing module, an improved SAM model, a semantic classification integration module, and a post-processing optimization module. The data preprocessing module includes four sub-modules: radiometric calibration, atmospheric correction, geometric correction, and auxiliary channel feature calculation; the improved SAM model includes a multispectral fusion module, a geometric attention layer, an image encoder, a prompt encoder, and a mask decoder; the post-processing optimization module uses CRF or morphological operations to achieve boundary smoothing.

[0019] Due to the implementation of the above technical solutions, the present invention has the following advantages compared with the prior art: In existing remote sensing image land cover segmentation processes, firstly, the multispectral channels of remote sensing images are poorly adapted, failing to fully extract feature information from remote sensing-specific bands such as near-infrared. Furthermore, processing efficiency for large-scale remote sensing images (5000×5000 pixels) is low, with single-image inference time exceeding 3 seconds and memory usage exceeding 16GB. Both segmentation speed and memory usage cannot meet the real-time requirements of large-scale remote sensing monitoring. Secondly, technical issues such as poor multispectral channel adaptability, low processing efficiency for large-scale images, blurred boundary segmentation, and lack of semantic classification capabilities result in low accuracy in the model's overall identification of land cover regions, meaning the average accuracy is low. The intersection-over-union ratio (mIoU, a core metric for segmentation accuracy) is below 80%. Furthermore, the Transformer architecture focuses on capturing global contextual features of the image, neglecting the geometric features of remote sensing imagery (such as land cover edge sharpness and surface texture diversity). This easily leads to problems like blurred boundaries and oversegmentation during segmentation, with an F1 score of only around 65% for land cover boundaries, failing to achieve fine-grained segmentation. Finally, it lacks a semantic supervision mechanism for land use types, only outputting target masks and not directly outputting land cover category labels. Additional post-processing steps are needed to match the mask with the land cover type, increasing the complexity of the process. The present invention addresses the shortcomings of existing methods, such as increased complexity and reduced overall segmentation efficiency. It is based on an improved SAM algorithm for remote sensing image land cover segmentation, cleverly resolving these deficiencies. This method first eliminates radiometric errors, atmospheric interference, and geometric distortions in the original remote sensing image imaging process, generating a multispectral image. Secondly, based on a ViT-based image encoder, cue encoder, and mask decoder, a multispectral fusion module is introduced at the front end of the image encoder. Through a channel attention mechanism, the multispectral band features of the input image are adaptively weighted and fused. Simultaneously, a geometric attention layer is embedded in the Transformer layer of the image encoder, using edge features extracted by Canny edge detection as priors, and incorporating Transformer attention calculations to form an improved SAM model. Thirdly, segmentation cue is generated through user interaction or automatic detection algorithms. The SAM model completes preliminary segmentation based on these cue, outputting a binary land cover mask. Next, land cover region features are extracted based on the binary mask, and a lightweight classification head completes semantic classification of land cover. Finally, an end-to-end training strategy is employed, combining mask loss and classification loss to fine-tune the SAM model.Finally, the segmentation results are smoothed using conditional random fields or morphological operations to output the final land use segmentation results with semantic labels. Therefore, this invention, on the one hand, fully leverages the spectral and geometric features of remote sensing images through the design of the multispectral fusion module and geometric attention layer, effectively solving problems such as blurred boundaries, over-segmentation, and spectral aliasing. Furthermore, the SAM model used achieves an average intersection-over-union (mIoU) ratio of over 85% for land use segmentation in remote sensing images, improves the F1 score of land use boundaries by 20%, and can output land use segmentation results with semantic labels. On the other hand, the block processing strategy for large-scale remote sensing images and the lightweight optimization of the model significantly reduce the model's memory usage and inference time, while retaining the zero-shot characteristic of the SAM model, eliminating the need to retrain the model for different scenarios. This significantly improves model adaptability and generalization ability. In addition, the end-to-end training strategy combined with a small amount of labeled data allows for model fine-tuning, greatly reducing data labeling costs and model training cycles. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the land cover segmentation system for remote sensing images based on the improved SAM algorithm in this embodiment; Figure 2 This is a schematic diagram of the improved SAM model in this embodiment; Figure 3 This is a comparison of the experimental results of this embodiment (visual comparison of Embodiment 1, original remote sensing image, original SAM segmentation result, and U-Net segmentation result). Detailed Implementation

[0021] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0022] like Figure 1 and Figure 2 As shown, the remote sensing image land cover segmentation system based on the improved SAM algorithm involved in this embodiment includes a remote sensing image 1, a data preprocessing module 2, an improved SAM model 8, a semantic classification integration module, and a post-processing optimization module. The data preprocessing module 2 includes radiometric calibration 3, atmospheric correction 4, geometric correction 5, and auxiliary channel feature calculation 6. The improved SAM model 8 includes a multispectral fusion module 9, a geometric attention layer 10, an image encoder 11, a cue encoder 12, and a mask decoder 13. The post-processing optimization module uses CRF or morphological operations to achieve boundary smoothing. The segmentation process is described in [reference needed]. Figure 1 .

[0023] In this example, the remote sensing image land cover segmentation processing method based on the improved SAM algorithm includes the following steps: S1, Data Preprocessing Radiometric calibration, atmospheric correction, and geometric correction are performed sequentially on the input raw remote sensing images to eliminate radiometric errors, atmospheric interference, and geometric distortions during the image imaging process, generating a standardized multispectral image. At the same time, the NDVI normalized vegetation index is calculated, and the NDVI normalized vegetation index and GLCM gray-level co-occurrence matrix texture features are integrated into the multispectral image as auxiliary channels. S2 and SAM model construction First, the multispectral band features of the input image are adaptively weighted and fused using a channel attention mechanism. Then, a geometric attention layer is embedded in the Transformer layer, using edge features extracted by Canny edge detection as priors and incorporating them into the Transformer's attention calculation. The image encoder front end introduces a multispectral fusion module to reduce the dimensionality of the feature maps of each channel of the input image. Then, an adaptive weight for each channel is calculated using a Softmax channel attention mechanism. Based on the weights, features from different bands such as visible light and infrared are fused to generate a unified multispectral fused feature map. The Transformer layer is divided into an extraction segment and a learning segment. The extraction segment is used to extract low-level features, textures, edges, and local structures. The learning segment is used to learn high-level semantics and global context features. The feature map output from the Transformer learning segment is linearly transformed to obtain a key and a value, where Key = W_K × F; Value = W_V × F, where W_K and W_V are the key and value projection matrices, and F is the output feature of the Transformer learning segment. S3. Prompt Generation and Preliminary Segmentation Segmentation prompts are generated through user interaction or automatic detection algorithms. The SAM model completes preliminary segmentation based on the segmentation prompts and outputs a binary mask of land use. S4. Semantic Classification and End-to-End Training Based on the extraction of land category regional features using binary masks, land category semantic classification is completed through a lightweight classification head. Then, an end-to-end training strategy is adopted, and the SAM model is fine-tuned by combining mask loss and classification loss. S5, Post-processing optimization The segmentation results are smoothed by using conditional random fields or morphological operations, and the final land class segmentation results with semantic labels are output.

[0024] Specifically, in step S1, radiometric calibration utilizes calibration coefficients provided by satellites / UAVs to perform linear / polynomial conversion on each pixel of each band of the image, obtaining standardized radiometric data. Atmospheric correction employs tools such as ENVI / ArcGIS, based on an atmospheric correction model, inputting parameters such as imaging time, observation angle, and atmospheric conditions to remove the influence of aerosols and water vapor, obtaining a true surface reflectance image. Geometric correction uses a high-precision DEM or reference map as a benchmark, selects ground control points (GCPs), and performs geometric correction and resampling through a polynomial / rational function model to ensure that the image coordinates are consistent with the actual geographic coordinates, guaranteeing the accuracy of ground feature boundaries and spatial positions. In short, radiometric calibration is used to convert the raw DN values ​​of remote sensing images into physically meaningful radiance / apparent reflectance values, eliminating errors caused by sensor gain and offset. Its implementation involves using calibration coefficients provided by satellites / UAVs to perform linear / polynomial conversion on each pixel of each band of the image, obtaining standardized radiometric data, and ensuring numerical consistency across different time phases and different sensor images. Atmospheric correction is used to eliminate radiation distortion caused by atmospheric scattering and absorption in remote sensing signals, restoring the true reflectance of the land surface. It is implemented using tools such as ENVI / ArcGIS, based on atmospheric correction models (e.g., FLAASH, 6S), inputting parameters such as imaging time, observation angle, and atmospheric conditions to remove the influence of aerosols and water vapor, obtaining an image of the true reflectance of the land surface and improving the accuracy of land cover spectral characteristics. Geometric correction is used to eliminate geometric positional offsets in remote sensing images caused by terrain undulations, sensor attitude, and Earth curvature, achieving precise geospatial matching. It is implemented using a high-precision DEM or reference map as a benchmark, selecting ground control points (GCPs), and performing geometric correction and resampling through a polynomial / rational function model to ensure that the image coordinates are consistent with the actual geographic coordinates, guaranteeing the accuracy of feature boundaries and spatial positions.

[0025] In some specific implementations, the NDV calculation formula is: NDVI=(NIR−R) / (NIR+R), where NIR is the reflectance value of the near-infrared band; R is the reflectance value of the red band (red light band). The Normalized Difference Vegetation Index (NDVI) is calculated pixel-by-pixel according to the above formula; the result range is -1 to +1, with positive values ​​representing vegetation, and higher values ​​indicating more abundant vegetation. In this application, the NDVI serves as an auxiliary feature channel integrated into the multispectral image, enhancing the distinction between land types such as cultivated land and forest land, and improving the model's segmentation accuracy.

[0026] In step S2, the Canny operator is used to perform edge detection on the preprocessed remote sensing image, outputting a binarized edge feature map E. This edge feature map is then linearly transformed to obtain the attention query Query, where Query = W_Q × E, and W_Q is the query projection matrix; E is the Canny edge feature map. Through the above calculation, the Canny edge prior information is integrated into the Transformer attention mechanism, enhancing the model's ability to capture land cover boundaries in the remote sensing image. Furthermore, geometric attention is calculated using scaled dot product attention, as follows: Attention(Q,K,V) = Softmax((Q×K^T) / √d_k)×V, where d_k is the feature dimension of the Key; √d_k is the scaling factor.

[0027] The specific calculation process includes the following: 1) Perform global average pooling on the reduced-dimensional multispectral band feature map in the spatial dimension to obtain the global feature description vector of each channel; 2) Obtain the initial weight coefficients of each channel by performing a nonlinear transformation on the global feature description vector; 3) Softmax normalize the initial weight coefficients to obtain channel adaptive weights that are non-negative and sum to 1; And the calculation formula is as follows: w k =Softmax(f k )=exp(f k ) / Σ(exp(fᵢ)) where, w k For the adaptive weights of the k-th band channel, f k The feature value of the k-th channel after global pooling and transformation is given. Based on the above weights, the features of different bands such as visible light and infrared are weighted and fused to generate a unified multispectral fusion feature map. The fusion method is to perform channel-by-channel weighted multiplication of the corresponding band feature maps with adaptive weights for each channel, and then stitch them together or accumulate them element-by-element according to the channel dimension. Finally, a multispectral fusion feature map with unified dimension and enhanced features is output. In step S4, a lightweight classification head is introduced after the preliminary segmentation module. Using the pre-trained ResNet model as the classification backbone network, the land cover region features are extracted from the land cover binary mask through global average pooling. The region features are input into the fully connected layer to complete the semantic classification of land use types such as cultivated land, forest land, water body, building, and bare land. An end-to-end training strategy is adopted, and the improved SAM model is fine-tuned with a small amount of remote sensing image annotation data. Taking the SAM model mask loss as the segmentation loss and the cross-entropy loss as the classification loss, the total loss function is designed as L = L_mask + αL_class, where α is the loss weight with a value range of 0.4 to 0.6. The model is jointly optimized through the total loss function. Specifically, the ResNet model includes an initial convolutional layer, multiple residual block groups, a global average pooling layer, and a fully connected layer. Each residual block group adopts a double-layer convolutional structure, and the residual calculation is H(x) = F(x) + x, where F(x) is the convolutional mapping residual function and x is the input shortcut feature. The preliminary segmentation mask feature F_mask is obtained through global average pooling to get F_gap. The input ResNet model obtains the deep semantic feature F_res. Then it is mapped through the fully connected layer to F_fc = W_fc・F_res + b_fc. Finally, the land cover probability P = Softmax(F_fc) is output through Softmax. In step S5, the segmentation result is boundary-smoothed through conditional random field or morphological operation to output the final land cover segmentation result with semantic labels. When using the conditional random field, the boundary is optimized by minimizing the energy function, and the energy function is: E(X) = Σiψ_u(x_i) + Σi<jψ_p(x_i,x_j), where the binary potential function uses a Gaussian kernel, the smoothing parameter σ takes a value of 3 to 5, and the number of iterations is 5 to 10 times to eliminate the segmentation error caused by image noise and improve the refinement degree of the segmentation result, and the final land cover segmentation result of the remote sensing image with semantic labels is output. When using morphological operation, the boundary is smoothed through closing operation A・B = (A⊕B)⊖B to eliminate noise and burrs, making the segmentation boundary more regular and smooth and improving the refinement degree of the result.

[0028] The experimental environment of this invention is as follows: The hardware uses NVIDIA A100 GPU (80G), Intel Xeon Gold 6330 CPU, and 256G memory; the software uses Python 3.9, PyTorch 2.0 deep learning framework, OpenCV 4.7.0 image processing library, and ENVI 5.6 remote sensing image processing software; the experimental data uses the Gaofen-2 high-resolution satellite remote sensing dataset and the actually collected UAV aerial remote sensing dataset, and the land cover classification targets are five categories of cultivated land, forest land, water body, building, and bare land.

[0029] Example 1 illustrates land use segmentation using high-resolution satellite remote sensing imagery. This case focuses on Gaofen-2 high-resolution satellite remote sensing imagery with a resolution of 0.5m, a pixel size of 5000×5000, and 4-channel multispectral resolution. It achieves automated segmentation of five land use categories: cultivated land, forest land, water bodies, buildings, and bare land. The specific steps are as follows: S1. Data Preprocessing: The original Gaofen-2 satellite image was radiometrically calibrated, atmospherically corrected, and geometrically corrected using ENVI 5.6 software to eliminate radiometric errors, atmospheric interference, and geometric distortions. The NDVI normalized vegetation index was calculated and used as the 5th channel. The GLCM gray-level co-occurrence matrix texture features were extracted using a 5×5 window and used as the 6th channel to obtain a 6-channel normalized multispectral image. The 6-channel image was then divided into blocks and normalized to 1024×1024 image blocks to fit the model input size. S2. Improved SAM model construction: ViT-Large is selected as the backbone network of the image encoder. A multispectral fusion module is built before encoding: 1×1 convolution is applied to the 6-channel feature maps to reduce the number of channels to 64. The adaptive weights of each channel are calculated through the Softmax channel attention mechanism. The weights are then fused into a unified feature map of 64 channels. The geometric attention layer is placed after the 6th layer of the Transformer. The Canny operator is used to extract the edge feature map of the image. This edge feature map is used as the attention query and attention is calculated with the key and value output by the 6th layer of the Transformer to enhance edge feature extraction. S3. Hint Generation: The K-means clustering algorithm is used to perform cluster analysis on the preprocessed 1024×1024 image blocks, and the center points of five land types, namely cultivated land, forest land, water bodies, buildings, and bare land, are automatically generated as segmentation hints. At the same time, users can manually select the land type of interest through human-computer interaction to generate custom bounding box hints, realizing interactive segmentation. S4. Semantic Classification and End-to-End Training: The improved SAM model receives segmentation prompts to complete initial segmentation and outputs binary masks for five land classes. A lightweight classification head extracts land class regional features from the binary masks through global average pooling and inputs these regional features into two fully connected layers to complete the semantic classification of the five land classes. The model training uses cross-entropy loss as the classification loss, combined with the original SAM mask loss, and designs the total loss function as L = L_mask + 0.5L_class. 10% of the labeled samples in the Gaofen-2 dataset are selected to complete the model fine-tuning. The learning rate is set to 1e-5, the batch size is 8, and the number of training epochs is 50. S5. Post-processing optimization: Apply Conditional Random Field (CRF) to optimize the segmentation results by setting the CRF parameter σ=3 and the number of iterations to 5. Smooth the segmentation boundary to eliminate segmentation errors caused by image noise and output the final land use segmentation results with semantic labels.

[0030] Specifically, experimental verification was performed on the Gaofen-2 dataset. The experimental results in this case show that, compared with the original SAM, the average intersection-union ratio (mIoU) of the method of this invention is improved by 15% to 87%, the F1 score of land class boundaries is improved by 20% to 85%, the segmentation time of a single 5000×5000 pixel image is shortened by 30% to only 0.8s, and the memory usage is reduced to 7G.

[0031] Example 2: Land Use Segmentation in UAV Aerial Remote Sensing Imagery. This case study focuses on land use segmentation of UAV aerial remote sensing imagery with RGB+near-infrared 4 channels, a pixel size of 2000×2000, and a resolution of 1m. UAV aerial imagery is characterized by numerous small-scale land use targets and rich surface textures. This example, based on Example 1, optimizes the segmentation to address these characteristics. The specific steps are as follows: S1. Data preprocessing: As in Example 1, after completing radiometric calibration, atmospheric correction, and geometric correction, the NDVI normalized vegetation index is calculated as the 5th channel, and the GLCM gray-level co-occurrence matrix texture features are extracted using a 3×3 window as the 6th channel, and the image is divided into 1024×1024 image blocks. S2. Improved SAM model construction: Consistent with Example 1, the architecture of ViT-Large backbone network + multispectral fusion module + geometric attention layer is adopted. S3. Hint Generation: Based on the automatic hints from K-means clustering in Example 1, a multi-scale hint generation strategy is added to generate land cover bounding boxes and center point hints at three scales: 128×128, 256×256, and 512×512, thereby improving the model's ability to identify and segment small-scale land cover targets. S4. Semantic classification and end-to-end training: Consistent with Example 1, the total loss function is set to L=L_mask+0.5L_class, and 10% of the labeled samples in the drone aerial photography dataset are selected to complete the model fine-tuning; S5. Post-processing optimization: Apply Conditional Random Field (CRF) for boundary optimization, set CRF parameter σ=4, and iterate 8 times to adapt to the texture features of UAV images.

[0032] The experimental results in this case show that the method achieves an average intersection-union ratio (mIoU) of 86% and an F1 score of 84% for land cover segmentation in UAV aerial images. The inference time for a single 2000×2000 pixel image is only 0.3s. It can accurately identify small-scale buildings, farmland patches, small water bodies and other targets in the images. The segmentation effect is significantly better than the original SAM and U-Net models.

[0033] Example 3: This case studies land cover segmentation on 2m resolution satellite remote sensing imagery (3-channel RGB) to verify the generalization ability of the method of the present invention. No retraining of the model is required; only the model trained in Example 1 is fine-tuned. The fine-tuning sample size is 5% of the dataset, and the remaining steps are the same as in Example 1. The experimental results show that the method achieves a segmentation mIoU of 83% and a boundary F1 score of 82% for 2m resolution remote sensing imagery, proving that the method of the present invention has strong generalization ability and can be adapted to remote sensing imagery with different resolutions and different spectral channels.

[0034] In summary, the core innovation of the above embodiments lies in solving the technical problems of the prior art through the following specific technical features, thereby achieving high-precision and high-efficiency segmentation of land cover types in remote sensing images: A multispectral fusion module was designed, which achieves adaptive weight allocation and fusion of multispectral channel features of remote sensing images through 1×1 convolutional dimensionality reduction and Softmax channel attention mechanism. This fully explores the feature information of remote sensing-specific bands such as near-infrared, and solves the problems of poor adaptability of the original SAM to multispectral remote sensing images and insufficient utilization of band features, thereby improving the feature utilization rate of multispectral channels by 30%. This invention proposes a geometric attention layer, which is embedded after the 6th layer of the Transformer (the Transformer structure consists of N layers, usually N is 12 or 24 layers. The first 6 layers mainly complete the extraction of low-level features, textures, edges, and local structures. After the 6th layer, it enters the learning stage of high-level semantics and global context features. The reason for embedding the geometric attention layer after the 6th layer is that the 6th layer has completed the full extraction of geometric features such as remote sensing image edges, contours, and land cover shapes. Introducing edge priors at this position can accurately inject geometric structure information into the global feature modeling process, which can not destroy the low-level feature extraction, but also significantly enhance the model's ability to perceive land cover boundaries, avoiding boundary blurring and oversegmentation. At the same time, edge features from Canny edge detection are used as priors to integrate into the attention calculation of the Transformer, which strengthens the model's ability to capture geometric features of remote sensing images, effectively solving the technical problems of blurred and oversegmented segmentation boundaries, and improving the F1 score of land cover boundaries by 20%). This approach achieves an organic combination of zero-shot characteristics and semantic supervision. While retaining the strong generalization ability of the SAM model, it integrates a lightweight classification head with ResNet as the backbone and adopts an end-to-end training strategy to jointly optimize the mask loss and classification loss. This allows the model to directly output land class segmentation results with semantic labels without additional post-processing, improving operational efficiency by 40%. A 1024×1024 block processing strategy was designed for large-scale remote sensing images. Combined with model lightweight optimization, the memory usage of the model was significantly reduced while ensuring segmentation accuracy. This resulted in an inference time of less than 1 second and a memory usage of less than 8G for a single 5000×5000 pixel high-resolution remote sensing image, making it suitable for large-scale remote sensing image processing needs. The design of the multispectral fusion module and geometric attention layer fully explores the spectral and geometric features of remote sensing images, effectively solving problems such as blurred boundaries, over-segmentation, and spectral aliasing. The model achieves an average intersection-union ratio (mIoU) of over 85% for land cover segmentation in remote sensing images, and improves the F1 score of land cover boundaries by 20%, thus realizing refined segmentation of land cover. The model can directly output land category segmentation results with semantic labels without additional post-processing steps. It can be seamlessly integrated into GIS geographic information systems and ENVI / ERDAS remote sensing image processing platforms, and can also be deployed on servers and UAV-borne edge devices. It provides accurate and efficient technical support for land resource monitoring, urban planning, disaster assessment, ecological protection and other fields, and has significant engineering application value. At the same time, the end-to-end training strategy can complete model fine-tuning with a small amount of labeled data, which greatly reduces data labeling costs and model training cycle. Compared with traditional deep learning segmentation methods, the training cost is reduced by more than 60%, making it easier to achieve engineering implementation.

[0035] In summary, the segmentation methods based on the above embodiments are compared with the original SAM model and U-Net segmentation, and the results are shown in Table 1.

[0036] Table 1 Case mIoU (%) Boundary F1 score (%) Inference time (s) Original SAM model 72 65 1.1 U-Net model 67 57 1.5 Example 1 87 85 0.8 Example 2 86 84 0.3 Example 3 88 86 0.6 At the same time, combined Figure 3 As shown in Table 1, the process from the original image to the original SAM, then to U-Net, and finally to this invention, with annotations for mIoU, F1, and time, yields the following advantages: 1) Segmentation accuracy is significantly improved. The mIoU and boundary F1 scores of all embodiments are significantly higher than those of the original SAM model and U-Net model. The highest mIoU reaches 88% (Example 3), which is 16 percentage points higher than the original SAM's 72%; the highest boundary F1 score reaches 86% (Example 3), which is 21 percentage points higher than the original SAM's 65%, indicating that the segmentation accuracy and boundary detail processing capabilities are significantly enhanced.

[0037] 2) The inference speed is significantly accelerated. The inference time of each embodiment is shorter. Embodiment 2 requires only 0.3 seconds, which is about 73% faster than the original SAM (1.1 seconds) and 80% faster than U-Net (1.5 seconds). The embodiments achieve more efficient computation while improving accuracy.

[0038] 3) Comprehensive performance optimization: The implementation method achieves a better balance between accuracy and speed, resulting in significantly higher accuracy and faster processing, with overall performance superior to traditional methods.

[0039] The present invention has been described in detail above, with the aim of enabling those skilled in the art to understand and implement the invention. However, this description should not be construed as limiting the scope of protection of the invention. All equivalent changes or modifications made in accordance with the spirit and essence of the invention should be included within the scope of protection of the invention.

Claims

1. A method for land cover segmentation processing of remote sensing images based on an improved SAM model, comprising the following steps: S1, Data Preprocessing Eliminate radiometric errors, atmospheric interference, and geometric distortions in the original remote sensing image imaging process to generate multispectral images; S2 and SAM model construction First, the multispectral band features of the input image are adaptively weighted and fused using a channel attention mechanism; then, a geometric attention layer is embedded in the Transformer layer, using the edge features extracted by Canny edge detection as a priori, and incorporating the attention calculation of the Transformer. S3. Prompt Generation and Preliminary Segmentation Segmentation prompts are generated through user interaction or automatic detection algorithms. The SAM model completes preliminary segmentation based on the segmentation prompts and outputs a binary mask of land use. S4. Semantic Classification and End-to-End Training Based on the extraction of land category regional features using binary masks, land category semantic classification is completed through a lightweight classification head. Then, an end-to-end training strategy is adopted, and the SAM model is fine-tuned by combining mask loss and classification loss. S5, Post-processing optimization The segmentation results are smoothed by applying conditional random fields or morphological operations, and the final land class segmentation result with semantic labels is output. Its characteristic is that... In step S2, a multispectral fusion module is introduced at the front end of the image encoder to perform dimensionality reduction on the feature maps of each channel of the input image. Then, the adaptive weights of each channel are calculated through the Softmax channel attention mechanism. Based on the weights, the features of different bands such as visible light and infrared are fused to generate a unified multispectral fusion feature map. The Transformer layer is divided into an extraction segment and a learning segment. The extraction segment is used to extract low-level features, textures, edges, and local structures. The learning segment is used to learn high-level semantics and global context features. The feature maps output by the Transformer learning segment are transformed linearly to obtain the key and value, Key = W_K × F; Value = W_V × F, where W_K and W_V are the key and value projection matrices, and F is the output feature of the Transformer learning segment.

2. The method for land cover segmentation processing of remote sensing images based on the improved SAM model according to claim 1, characterized in that, In step S2, the Canny operator is used to perform edge detection on the preprocessed remote sensing image, and a binarized edge feature map E is output. The edge feature map is then linearly transformed to obtain the attention query Query, where Query = W_Q × E, and W_Q is the query projection matrix; E is the Canny edge feature map.

3. The method for land cover segmentation processing of remote sensing images based on the improved SAM model according to claim 2, characterized in that, Geometric attention is calculated using scaled dot product attention, as shown in the following formula: Attention(Q,K,V)=Softmax((Q×K^T) / √d_k)×V, where: d_k is the feature dimension of the key; √d_k is the scaling factor.

4. The method for land cover segmentation processing of remote sensing images based on the improved SAM model according to claim 3, characterized in that, The specific calculation process includes the following: 1) Perform global average pooling on the spatial dimension of the reduced multispectral band feature map to obtain the global feature description vector of each channel; 2) Obtain the initial weight coefficients for each channel by performing a nonlinear transformation on the global feature description vector; 3) Softmax normalize the initial weight coefficients to obtain channel adaptive weights that are non-negative and sum to 1; And the calculation formula is as follows: w k =Softmax(f k )=exp(f k ) / Σ(exp(fᵢ)) where, w k For the adaptive weights of the k-th band channel, f k The eigenvalues ​​of the k-th channel after global pooling and transformation are denoted as .

5. The method for land cover segmentation processing of remote sensing images based on the improved SAM model according to claim 1, characterized in that, In step S1, the input original remote sensing image is successively subjected to radiometric calibration, atmospheric correction, and geometric correction to eliminate the radiometric error, atmospheric interference, and geometric distortion during the image imaging process, generating a standardized multispectral image. At the same time, the NDVI (Normalized Difference Vegetation Index) is calculated, and the NDVI and the texture features of the GLCM (Gray-Level Co-occurrence Matrix) are incorporated into the multispectral image as auxiliary channels.

6. The method for land cover segmentation processing of remote sensing images based on the improved SAM model according to claim 5, characterized in that, Radiometric calibration uses the calibration coefficients provided by the satellite / drone to perform linear / polynomial conversion for each pixel in each band of the image to obtain standardized radiometric data. Atmospheric correction uses tools such as ENVI / ArcGIS, based on the atmospheric correction model, inputs parameters such as imaging time, observation angle, and atmospheric conditions, and removes the influence of aerosols and water vapor to obtain the true surface reflectance image. Geometric correction is based on a high-precision DEM or reference base map, selects ground control points (GCPs), and performs geometric correction and resampling through a polynomial / rational function model to make the image coordinates consistent with the true geographic coordinates and ensure the accuracy of the ground object boundaries and spatial positions.

7. The method for land cover segmentation processing of remote sensing images based on the improved SAM model according to claim 5 or 6, characterized in that, NDV calculation formula: NDVI = (NIR - R) / (NIR + R), where NIR is the reflectance value of the near-infrared band; R is the reflectance value of the red band (red light band).

8. The method for land cover segmentation processing of remote sensing images based on the improved SAM model according to claim 1, characterized in that, In step S4, a lightweight classification head is introduced after the preliminary segmentation module. Using the pre-trained ResNet model as the classification backbone network, the regional features of land classes are extracted from the binary mask of land classes through global average pooling, and the regional features are input into the fully connected layer to complete the semantic classification of land use types such as cultivated land, forest land, water bodies, buildings, and bare land. And / or, adopt an end-to-end training strategy, and fine-tune the improved SAM model by combining a small amount of remote sensing image annotation data. Use the SAM model mask loss as the segmentation loss and the cross-entropy loss as the classification loss, and design the total loss function as L = L_mask + αL_class, where α is the loss weight and its value range is 0.4 - 0.

6.

9. The method for land cover segmentation processing of remote sensing images based on the improved SAM model according to claim 8, characterized in that, The ResNet model includes an initial convolutional layer, multiple residual block groups, a global average pooling layer, and a fully connected layer. Each residual block group adopts a double-layer convolutional structure, and the residual calculation is H(x) = F(x) + x, where F(x) is the convolutional mapping residual function and x is the input shortcut feature; the preliminary segmentation mask feature F_mask is obtained through global average pooling as F_gap; the depth semantic feature F_res is obtained by inputting into the ResNet model; then it is mapped through the fully connected layer as F_fc = W_fc・F_res + b_fc; finally, the land class probability P = Softmax(F_fc) is output through Softmax.

10. The method for land cover segmentation processing of remote sensing images based on the improved SAM model according to claim 1, characterized in that, In step S5, when using the conditional random field, boundary optimization is achieved by minimizing the energy function. The energy function is: E(X) = Σiψ u(x_i) + Σi<jψ p(x_i,x_j), where the binary potential function uses a Gaussian kernel, the smoothing parameter σ takes values from 3 to 5, and the number of iterations is 5 to 10 times; when using morphological operations, boundary smoothing is achieved through the closing operation A・B = (A⊕B)⊖B.

Images