Remote sensing image segmentation method and system based on cross-normal-form feature fusion and alignment

Abstract

The invention discloses a remote sensing image segmentation method and system based on cross-normal-form feature fusion and alignment, and the method comprises the steps: carrying out the preprocessing of an input remote sensing image, and extracting an initial feature; inputting a cross-normal-form feature fusion and alignment network, and fusing multi-modal and cross-scale remote sensing image structure information through sparse channel enhancement and space alignment and space pixel refining and channel alignment to obtain a first-stage fusion feature; inputting a multi-stage cross-paradigm enhanced feature extraction network, fusing local details and global context information through multi-level information interaction and a dynamic gating mechanism, and gradually extracting a joint feature map of semantic and spatial structure collaborative expression; a final semantic segmentation result is generated through the segmentation head, and composite loss is calculated based on a real label; according to the method, the multi-stage feature extraction network and a cross-normal-form feature alignment mechanism are constructed, local textures, spatial contexts and multi-modal information are effectively fused, and the segmentation performance is enhanced while the calculation efficiency is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, and more particularly to a remote sensing image segmentation method and system based on cross-paradigm feature fusion and alignment. BACKGROUND

[0002] Remote sensing image segmentation, as a core technology in the fields of geographic information systems, urban planning, agricultural monitoring, and disaster assessment, has received extensive attention in recent years. The goal of remote sensing image segmentation is to semantically classify the pixels in high-resolution remote sensing images and accurately assign each pixel to the corresponding land cover category, such as buildings, water bodies, roads, farmland, and forest land.

[0003] With the continuous improvement of remote sensing data quality and the diversification of remote sensing imaging methods, how to achieve high-precision, multi-scale, and structure-aware image segmentation in complex scenes has become one of the key challenges in current research.

[0004] Most existing mainstream methods are based on convolutional neural networks (CNN) or Transformer structures, which extract multi-level semantic information from images to complete pixel-level classification of land covers. However, traditional CNNs have limited receptive fields, making it difficult to capture target information with long-range dependencies in remote sensing images. While standard Transformers have strong global modeling capabilities, they lack the ability to express local details, and they face difficulties in aligning structures and redundant calculations when dealing with multi-scale targets or different modal remote sensing images (such as optical and SAR fusion).

[0005] In addition, remote sensing images have the following unique challenges: uneven distribution of target categories, fuzzy boundaries, large proportion of small targets, and existence of multi-scale overlapping regions. These factors collectively limit the accuracy of segmentation models, especially in terms of edge details and minority class detection. SUMMARY

[0006] In view of the above, the present application provides a remote sensing image segmentation method and system based on cross-paradigm feature fusion and alignment, aiming to improve the segmentation accuracy of complex land cover structures in remote sensing images, especially in challenging scenarios with large scale differences, fuzzy boundaries, and uneven distribution of categories, to achieve robust perception and accurate identification of fine-grained target regions.

[0007] To achieve the above purpose, the present application adopts the following technical solutions:

[0008] A remote sensing image segmentation method based on cross-paradigm feature fusion and alignment, comprising the following steps:

[0009] S1. Preprocess the input remote sensing image and input the initial convolutional layer to extract low-level texture and edge features, obtaining initial features;

[0010] S2. input the initial features into the constructed cross-paradigm feature fusion and alignment network, enhance through sparse channels and spatial alignment, refine spatial pixels and align channels, fuse multi-modal and cross-scale remote sensing image structure information, and obtain first-stage fusion features;

[0011] S3. input the first-stage fusion features into the constructed multi-stage cross-paradigm enhanced feature extraction network, fuse local details and global context information through multi-level information interaction and dynamic gating mechanism, and gradually extract joint feature maps with collaborative expression of semantics and spatial structure;

[0012] S3. generate the final semantic segmentation result according to the joint feature maps through a segmentation head, and calculate a composite loss based on real labels to optimize model parameters.

[0013] Preferably, the specific content of step S2 includes:

[0014] S21. perform linear transformation on the input features through a convolutional layer to extract preliminary semantic features;

[0015] S22. strengthen the response of key regions through a sparse attention mechanism and a spatial alignment strategy for the preliminary semantic features, and obtain sparse channel features and sparse channel spatial weights;

[0016] S23. enhance the spatial structure perception ability of the preliminary semantic features in units of pixels, and selectively align the channels, spatial enhancement features and spatial alignment weights;

[0017] S24. respectively weight fuse the sparse channel features and the spatial enhancement features, and output the fusion result through linear transformation.

[0018] Preferably, the specific content of step S22 includes:

[0019] S221. input the preliminary semantic features into four parallel sub-branches with different but complementary functions respectively, form a multi-view feature perception mechanism, generate a query matrix Q, a key matrix K, a sparse attention index ρ, and a value matrix V;

[0020] S222. construct a sparse attention atlas, reshape Q and K, perform dot product to obtain an initial attention matrix, filter key attention scores according to the index ρ, obtain a sparse attention mapping, and use Softmax activation to act on V to obtain sparse channel features. The sparse channel features are passed through convolution, GELU activation, convolution again, and Sigmoid to obtain channel spatial fusion weights.

[0021] Preferably, the specific content of step S23 includes:

[0022] S231. generate spatial alignment weights:

[0023]

[0024] S232. Generate spatial enhancement features by combining the global average pooling with weight-enhanced spatial responses:

[0025]

[0026] S233. Generate spatial alignment weights based on the spatial enhancement features:

[0027]

[0028] Preferably, the specific content of step S3 includes:

[0029] S31. Construct a multi-scale information flow gating network to perform a normalization operation on the first-stage fusion features, then pass the features through two parallel sub-paths to form a lightweight gating mechanism to obtain fusion features, and then perform a 1x1 convolution to restore the channel, and perform a residual connection with the features after the normalization operation to obtain first-stage low-level enhancement features;

[0030] S32. The first-stage low-level enhancement features are sequentially passed through two groups of stacked cross-paradigm feature fusion and alignment networks and multi-scale information flow gating networks to obtain second-stage enhancement features and third-stage enhancement features, and after a splicing operation, further input a group of stacked cross-paradigm feature fusion and alignment networks and multi-scale information flow gating networks to obtain high-order abstract features;

[0031] S33. Splice the low-level enhancement features generated in the first stage and the high-order abstract features generated in the last stage to form the final joint feature map.

[0032] Preferably, the specific content of step S31, through two parallel sub-paths to form a lightweight gating mechanism to obtain fusion features includes:

[0033] The features after the normalization operation are input into the two parallel sub-paths constructed, path A passes through a 1x1 convolution to compress the channel dimension, then passes through a 3x3 depth separable convolution to extract local context information, path B passes through a 1x1 convolution to compress the channel dimension and a 3x3 depth separable convolution and a GELU nonlinear activation to model nonlinear response, the outputs of path A and path B are multiplied point by point to form fusion features.

[0034] Preferably, in step S3, the composite loss includes a pixel-level cross-entropy loss, a boundary perception loss, and a class balance loss;

[0035]

[0036] wherein λ1, λ2, and λ3 are weighting coefficients, a pixel-level cross-entropy loss, a boundary-aware loss, a class-balanced loss.

[0037] A remote sensing image segmentation system based on cross-paradigm feature fusion and alignment, based on the remote sensing image segmentation method based on cross-paradigm feature fusion and alignment, comprising: an image acquisition module, an initial feature extraction module, a first stage feature fusion module, a multi-stage cross-paradigm enhanced feature extraction module, and a prediction and optimization module.

[0038] The image acquisition module is used for acquiring remote sensing images and performing preprocessing

[0039] The initial feature extraction module is used for extracting low-level texture and edge features by an initial convolutional layer from the preprocessed remote sensing images to obtain initial features.

[0040] The first stage feature fusion module is used for inputting the initial features into a constructed cross-paradigm feature fusion and alignment network to fuse multi-modal and cross-scale remote sensing image structure information to obtain first stage fusion features, and the cross-paradigm feature fusion and alignment network comprises a sparse channel enhancement and spatial alignment module and a spatial pixel refinement and channel alignment module.

[0041] The multi-stage cross-paradigm enhanced feature extraction module is used for inputting the first stage fusion features into a constructed multi-stage cross-paradigm enhanced feature extraction network to fuse local details and global context information through multi-level information interaction and a dynamic gating mechanism to gradually extract joint feature maps with collaborative expression of semantics and spatial structure.

[0042] The prediction and optimization module is used for generating a final semantic segmentation result from the joint feature maps through a segmentation head and calculating a composite loss based on real labels to optimize model parameters.

[0043] A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the remote sensing image segmentation method based on cross-paradigm feature fusion and alignment.

[0044] A processing terminal comprising a memory and a processor, the memory storing a computer program executable on the processor, and the processor implementing the remote sensing image segmentation method based on cross-paradigm feature fusion and alignment when executing the computer program.

[0045] According to the above technical solution, compared with the prior art, the remote sensing image segmentation method and system based on cross-paradigm feature fusion and alignment have the following beneficial effects:

[0046] (1)Propose a multi-stage cross-paradigm enhanced feature extraction network, through progressive information fusion, multi-scale gating mechanism and dynamic interaction of shallow and deep features, significantly enhance the model's expression ability for details in complex scenes (such as building edges, road skeletons, water boundaries, etc.), Compared with traditional single-scale or static fusion models, the accuracy and robustness are significantly improved;

[0047] (2)Construct a cross-paradigm feature fusion and alignment network, introduce sparse attention channel modeling and spatial structure alignment mechanism, combined with boundary perception supervision and class balance loss design, not only improve the model's recognition ability for target edges and small sample classes, but also effectively alleviate the performance bottleneck caused by "inter-class imbalance" and "scale inconsistency" in remote sensing images, improve the overall segmentation consistency and generalization ability. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only a part of the embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0049] Figure 1 A remote sensing image segmentation method based on cross-paradigm feature fusion and alignment provided by the present application is shown in the figure.

[0050] Figure 2 The cross-paradigm feature fusion and alignment network provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0052] The embodiments of the present application disclose a remote sensing image segmentation method based on cross-paradigm feature fusion and alignment, as shown in Figure 1 , comprising the following steps:

[0053] S1. Preprocess the input remote sensing image , and input the initial convolution layer to extract low-level texture and edge features, and obtain the initial feature X0=Conv(I), wherein Conv represents the convolution layer;

[0054] S2. Input the initial feature X0 into the constructed cross-paradigm feature fusion and alignment network, enhance through sparse channel and spatial alignment, and refine spatial pixels and align channels, fuse multi-modal and cross-scale remote sensing image structure information, and obtain first-stage fusion features

[0055] S3. Input the first-stage fusion features into the constructed multi-stage cross-paradigm enhanced feature extraction network, fuse local details and global context information through multi-level information interaction and dynamic gating mechanism, and gradually extract joint feature maps with collaborative expression of semantics and spatial structure.

[0056] S3. Generate the final semantic segmentation result according to the joint feature map through the segmentation head, and calculate the composite loss based on the real label to optimize the model parameters.

[0057] In order to further implement the above technical solutions, such as Figure 2 The specific content of step S2 includes:

[0058] S21. Linearly transform the input features through the convolution layer to extract preliminary semantic features.

[0059] Let the input feature be Where H, W and C are the height, width and channel number of the feature map, and the preliminary semantic representation is

[0060] S22. Strengthen the response of key areas through sparse attention mechanism and spatial alignment strategy for the preliminary semantic features, and obtain sparse channel features and sparse channel spatial weights.

[0061] S23. Enhance the spatial structure perception ability of the preliminary semantic features in pixel units, and selectively align the channels, spatial enhancement features and spatial alignment weights.

[0062] S24. Weighted fusion is performed on the sparse channel features and spatial enhancement features respectively, and the fusion result is output through linear transformation.

[0063] In order to further implement the above technical solutions, the specific content of step S22 includes:

[0064] S221. Send the preliminary semantic features into four parallel sub-branches with different but complementary functions respectively, form a multi-view feature perception mechanism, generate a query matrix Q, a key matrix K, a sparse attention index p and a value matrix V.

[0065] Specifically:

[0066] The first branch is used to generate a query matrix Q, which is obtained by concatenating a standard convolution operation and a depth separable convolution operation, effectively reducing the computational complexity while maintaining spatial perception. The core purpose is to extract potential attention query vectors from the input features to capture structural information with significant differences in the image The query matrix Q plays a role in focusing on the target area in the attention mechanism and is the key starting point for matching relationships.

[0067] The second branch is used to generate a key matrix K, which has the same structure as the first branch and also uses convolution and depth separable convolution. The purpose is to form key vectors that are semantically aligned with query vectors to capture similar response positions in the entire image In remote sensing images, similar object classes may appear in different spatial positions, so this branch helps to achieve spatial matching at the semantic level.

[0068] The third branch is used to generate a sparse attention index ρ, which is obtained by layer-by-layer compression and nonlinear mapping operations. It adaptively learns the distribution probability of important attention channels to eliminate low-value attention connections. The entire process is represented as:

[0069]

[0070] where Norm(·) is the normalization operation, σ represents the Sigmoid activation function, Flatten is the flattening operation, and GAP represents global average pooling. This structure effectively captures the importance scores across channels and plays a "channel gating" role in the sparse attention mechanism, focusing calculations on high-contribution channels and improving computational efficiency and model discrimination.

[0071] The fourth branch is used to generate a value matrix V, which also uses a cascade of convolution and depth separable convolution. The purpose is to generate value vectors for attention matrix weighting, which contains spatial semantics and contextual features of the image:

[0072]

[0073] In the cross-region feature fusion process, V serves as the carrier of feature responses, and its information expression quality directly affects the representation effect after attention enhancement.

[0074] S222. Construct a sparse attention map, reshape Q and K, and perform dot product to obtain an initial attention matrix According to the index ρ, filter the key attention scores to obtain a sparse attention map and after using Softmax activation, it acts on V to obtain sparse channel features The sparse channel features Through convolution, GELU activation, re-convolution and Sigmoid, a channel space fusion weight is obtained

[0075] To further implement the above technical solution, the specific content of step S23 includes:

[0076] S231. Generating a spatial alignment weight:

[0077]

[0078] S232. Enhancing the spatial response by combining the weight through global average pooling to generate a spatial enhancement feature:

[0079]

[0080] S233. Generating a spatial alignment weight based on the spatial enhancement feature:

[0081]

[0082] In this embodiment, the fusion result output by step S24 is:

[0083]

[0084] Wherein, ⊙ represents point-by-point multiplication, represents feature addition.

[0085] In this embodiment, the cross-paradigm feature fusion and alignment network is to better fuse the feature representations of different scales and different modalities (such as optical and SAR) in remote sensing images, and improve the segmentation capability for fine-grained structures and semantic regions.

[0086] To further implement the above technical solution, the specific content of step S3 includes:

[0087] S31. Constructing a multi-scale information flow gating network to perform normalization operation on the first stage fusion feature Wherein, Norm represents the normalization operation, then a lightweight gating mechanism is formed through two parallel sub-paths to obtain the fusion feature X, and then 1×1 convolution is performed for channel restoration, and the normalized feature is connected in residual to obtain the first stage low-level enhancement feature This operation improves the stability and semantic consistency of feature transmission;

[0088] ​​S32. Multi-stage alternating fusion: the first stage low-level enhanced features are sequentially input into a set of stacked cross-paradigm feature fusion and alignment networks and multi-scale information flow gating networks to obtain second stage enhanced features and third stage enhanced features, and after a concatenation operation, the features are further input into a set of stacked cross-paradigm feature fusion and alignment networks and multi-scale information flow gating networks to obtain high-level abstract features;

[0089] In this embodiment, the multi-stage alternating stacked fusion structure progressively enhances the features layer by layer. Specifically, the enhanced features output by the previous stage are first input into a cross-paradigm feature fusion and alignment network again to extract more complete context semantics, and then the features are input into a multi-scale information flow gating network to strengthen the adaptive perception of spatial scale changes. After the first round of fusion, the intermediate features obtained are again subjected to the above processing procedure to form second round enhanced features. After two rounds of stacking, the features extracted in the two stages are concatenated to form composite feature representations with multi-scale responses. The concatenated features are further input into a set of cross-paradigm feature fusion modules and information flow gating modules to extract higher-level abstract features, realizing cross-level and cross-semantic deep fusion.

[0090] S33. Global feature aggregation and output: after the multi-stage enhanced feature extraction is completed, the low-level enhanced features generated in the first stage and the high-level abstract features generated in the last stage are concatenated to form a final joint feature map. The joint feature map retains both the texture details of the shallow layer and the semantic expression of the deep layer, and has rich spatial context information.

[0091] In this embodiment, this bottom-up feature aggregation method can effectively improve the segmentation accuracy of complex scenes in remote sensing images, such as urban buildings, road networks, water body boundaries, and farmland regions. It performs better in robustness and expression when dealing with images with large differences in ground object scales and high boundary ambiguity.

[0092] To further implement the above technical solution, step S31, a lightweight gating mechanism is constructed through two parallel sub-paths to obtain the specific content of the fused features, including:

[0093] After the normalization operation, the features input into the two parallel sub-paths. Path A compresses the channel dimension through 1x1 convolution, and then extracts local context information through 3x3 depth separable convolution. Path B compresses the channel dimension through 1x1 convolution, and then models the nonlinear response through 3x3 depth separable convolution and GELU nonlinear activation. The outputs of path A and path B are multiplied point by point to form the fused features.

[0094] In this embodiment, the fused features obtained in step S31 are represented as:

[0095] X+ = DSConv(Conv(X n )) GELU(DSConv(Conv(X n ))

[0096] where, ⊙ represents an element-wise multiplication operation, and DSConv represents a depth separable convolution operation; the two parallel sub-paths constitute a lightweight gating mechanism, which enhances the selective response capability of the model to multi-scale heterogeneous targets.

[0097] To further implement the above technical solution, in step S3, the composite loss includes a pixel-level cross-entropy loss, a boundary perception loss, and a class balance loss.

[0098]

[0099] wherein λ1, λ2, and λ3 are weighting coefficients, is a pixel-level cross-entropy loss, is a boundary perception loss, is a class balance loss.

[0100] In this embodiment, in order to map the feature map to a semantic class probability map, the segmentation head is a lightweight segmentation head, which is composed of a series of convolution layers, up-sampling layers, and a Softmax layer, represents the final output prediction map of the model, wherein H and W are image sizes, and C is the number of ground object classes, and P i represents the prediction probability of the pixel belonging to each class;

[0101] The pixel-level cross-entropy loss is specifically: in order to achieve basic classification accuracy, a conventional multi-class cross-entropy loss function is used to supervise the prediction probability map P and the real label map G pixel by pixel, and let wherein each pixel value is the real ground object class number, and the cross-entropy loss is:

[0102]

[0103] wherein N = H x W, is an indicator function, and if the real class of the i-th pixel is c, then this term takes 1;

[0104] The boundary perception loss and the class balance loss are introduced to further improve the recognition ability of the model to small target boundaries and slender structures (such as roads and rivers), and to solve the class imbalance problem in remote sensing images, and are specifically:

[0105] Boundary perception loss To enhance the sensitivity of the model to the edge structure of the ground object, a boundary loss mechanism is introduced, specifically: the edge extraction operator (such as Sobel or Laplacian) is used on the real label graph G and the prediction graph P to generate the boundary mask E G and E P , and the binary cross-entropy loss between them is calculated:

[0106]

[0107] Wherein, E G = EdgeDetect(G), indicating that the edge detection is performed on the real label graph; E P = EdgeDetect(argmax(P)), indicating that the edge detection is performed on the prediction class graph; the loss term effectively improves the prediction accuracy of the model at the pixel boundary, especially for small targets and regions with obvious structure changes;

[0108] Class balance loss In view of the phenomenon that the class distribution is seriously uneven in the remote sensing image, FocalLoss or Dice Loss is introduced to strengthen the learning ability of difficult classification samples or small class regions. Taking Dice Loss as an example:

[0109]

[0110] Wherein, represents the probability that the ith pixel is predicted as its real class; the numerator is the overlapping part of the prediction and the label, and the denominator is the sum of the total amount of prediction and the total amount of real; Dice Loss can effectively balance the influence of foreground and background, and is especially suitable for small target detection scenes (such as narrow roads, islands, aircraft, etc.);

[0111] The final composite loss function is constructed to constrain the model performance from multiple angles, and the typical values of the weighting coefficients are λ1=1.0, λ2=0.5 and λ3=0.5; the three loss terms can optimize the model performance from the overall accuracy, boundary clarity and class balance; this composite mechanism is especially suitable for multi-target, multi-class and boundary complex remote sensing image tasks.

[0112] In the embodiment, the application proposes a multi-stage cross-paradigm enhanced fusion remote sensing image segmentation system, a cross-paradigm feature extraction structure combining the advantages of Transformer and CNN is constructed, through sparse attention mechanism, spatial alignment strategy, multi-scale information flow gating network, and boundary perception and class balance supervision mechanism, the accurate modeling of complex ground objects in remote sensing images is realized;

[0113] Specifically, a multi-stage cross-paradigm enhanced feature extraction network is constructed, a multi-level and alternating information enhancement mechanism is used to fuse local texture and global context, and high-quality feature maps that cooperatively express semantics and spatial structure are gradually extracted; In this process, the dynamic gating mechanism can adjust the features according to the feature scale and semantic intensity, effectively dealing with the expression deviation of heterogeneous targets.

[0114] A cross-paradigm feature fusion and alignment network is introduced, a four-branch structure is designed to model the query, key, value matrix and channel importance index, and the spatial reconstruction and channel compression strategy is used to improve the alignment ability of multi-modal information in space and the fusion effect in semantics; At the same time, combined with the spatial pixel refining and channel alignment module, the spatial structure perception enhancement and channel level response difference calibration are realized, which effectively enhances the sensitivity of the model to complex texture regions and structure boundaries.

[0115] In view of the small target detection difficulty and class imbalance problem in remote sensing images, a composite supervision mechanism including cross-entropy loss, boundary perception loss and class balance loss is designed; Among them, the boundary loss significantly improves the response ability of the model to the pixel edge region, and the class balance loss (such as Dice or FocalLoss) enhances the robustness of the model in the detection of minority classes; The segmentation head part adopts a lightweight structure, and the final semantic segmentation result is output by combining the Softmax probability mapping, to ensure the effective balance between model accuracy and computational efficiency.

[0116] In summary, the present application provides a systematic solution to the precision bottleneck, boundary ambiguity and class imbalance problems in remote sensing image segmentation by integrating multi-stage progressive feature extraction, cross-paradigm fusion and alignment, spatial channel joint regulation, and composite supervision optimization, which has wide practical application prospects and research value.

[0117] A remote sensing image segmentation system based on cross-paradigm feature fusion and alignment, based on a remote sensing image segmentation method based on cross-paradigm feature fusion and alignment, comprising: an image acquisition module, an initial feature extraction module, a first stage feature fusion module, a multi-stage cross-paradigm enhanced feature extraction module, and a prediction and optimization module;

[0118] The image acquisition module is used for acquiring remote sensing images and pre-processing

[0119] The initial feature extraction module is used for extracting low-level texture and edge features from the pre-processed remote sensing image through an initial convolutional layer to obtain initial features.

[0120] The first-stage feature fusion module is configured to input the initial features into a constructed cross-paradigm feature fusion and alignment network, fuse multi-modal and cross-scale remote sensing image structure information, and obtain first-stage fusion features.

[0121] The multi-stage cross-paradigm enhanced feature extraction module is configured to input the first-stage fusion features into a constructed multi-stage cross-paradigm enhanced feature extraction network, fuse local details and global context information through multi-level information interaction and a dynamic gating mechanism, and gradually extract joint feature maps that cooperatively express semantics and spatial structures.

[0122] The prediction and optimization module is configured to generate a final semantic segmentation result through a segmentation head according to the joint feature maps, and calculate a composite loss based on real labels to optimize model parameters.

[0123] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement a remote sensing image segmentation method based on cross-paradigm feature fusion and alignment.

[0124] A processing terminal includes a memory and a processor, and the memory stores a computer program executable on the processor. The processor executes the computer program to implement a remote sensing image segmentation method based on cross-paradigm feature fusion and alignment.

[0125] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0126] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image segmentation method based on cross-paradigm feature fusion and alignment, characterized in that: The following steps are involved: S1. Preprocess the input remote sensing image and input it into the initial convolution layer to extract low-level texture and edge features to obtain initial features; S2. Input the initial features into the constructed cross-paradigm feature fusion and alignment network. Through sparse channel enhancement and spatial alignment, as well as spatial pixel refinement and channel alignment, the multimodal and cross-scale remote sensing image structural information is integrated to obtain the first-stage fused features. S3. The fused features from the first stage are fed into the constructed multi-stage cross-paradigm enhanced feature extraction network. Through multi-level information interaction and a dynamic gating mechanism, local details are integrated with global context information to gradually extract a joint feature map that synergistically expresses semantics and spatial structure. S3. Generate the final semantic segmentation result through the segmentation head based on the joint feature map, and calculate the composite loss based on the true label to optimize the model parameters.

2. A remote sensing image segmentation method based on cross-paradigm feature fusion and alignment according to claim 1, characterized in that: The specific contents of step S2 include: S21. Perform linear transformation on the input features through the convolution layer to extract preliminary semantic features; S22. The preliminary semantic features are strengthened with key region responses through sparse attention mechanism and spatial alignment strategy to obtain sparse channel features and sparse channel spatial weights. S23. Enhance spatial structure perception capabilities of preliminary semantic features in pixel units, and selectively align channels, spatial enhancement features, and spatial alignment weights; S24. Perform weighted fusion on the sparse channel features and the spatial enhancement features respectively, and output the fusion results through linear transformation.

3. The remote sensing image segmentation method based on cross-paradigm feature fusion and alignment according to claim 2, characterized in that: The specific contents of step S22 include: S221. Send the preliminary semantic features to four functionally different but complementary parallel sub-branches respectively to form a multi-view feature perception mechanism, generating the query matrix Q, key matrix K, sparse attention index ρ and value matrix V; S222. Construct a sparse attention map, reshape Q and K, perform dot product to obtain the initial attention matrix, filter the key attention scores according to the index ρ, obtain a sparse attention map, and use Softmax activation on V to obtain sparse channel features. The sparse channel features are passed through convolution, GELU activation, reconvolution and Sigmoid to obtain channel space fusion weights.

4. The remote sensing image segmentation method based on cross-paradigm feature fusion and alignment according to claim 2, characterized in that: The specific contents of step S23 include: S231. Generate spatial alignment weights: S232. Generate spatial enhancement features by combining global average pooling with weighted spatial response: S233. Generate spatial alignment weights based on spatial enhancement features:

5. The remote sensing image segmentation method based on cross-paradigm feature fusion and alignment according to claim 1, characterized in that: The specific contents of step S3 include: S31. Construct a multi-scale information flow gating network to normalize the first-stage fused features. Then, a lightweight gating mechanism is formed through two parallel subpaths to obtain the fused features. The fused features are then restored through 1×1 convolution and residually connected with the normalized features to obtain the first-stage low-level enhanced features. S32. The low-level enhanced features from the first stage are sequentially passed through two stacked cross-paradigm feature fusion and alignment networks and a multi-scale information flow gating network to obtain the second-stage enhanced features and the third-stage enhanced features. After concatenation, these features are further input into a stacked cross-paradigm feature fusion and alignment network and a multi-scale information flow gating network to obtain high-level abstract features. S33. Concatenate the low-level enhanced features generated in the first stage with the high-level abstract features generated in the last stage to form the final joint feature map.

6. The remote sensing image segmentation method based on cross-paradigm feature fusion and alignment according to claim 5, characterized in that: Step S31, forming a lightweight gating mechanism through two parallel sub-paths to obtain fusion features, specifically includes: Two parallel sub-paths are constructed based on the feature input after normalization. Path A compresses the channel dimension through 1×1 convolution and then extracts local context information through 3×3 depth-separable convolution. Path B compresses the channel dimension through 1×1 convolution and 3×3 depth-separable convolution and GELU nonlinear activation to model nonlinear responses. The outputs of path A and path B are multiplied point by point to form a fused feature.

7. The remote sensing image segmentation method based on cross-paradigm feature fusion and alignment according to claim 1, characterized in that: In step S3, the composite loss includes pixel-level cross entropy loss, boundary-aware loss, and category balance loss; Among them, λ1, λ2, and λ3 are weighting coefficients. is the pixel-level cross entropy loss, is the boundary-aware loss, is the class balancing loss.

8. A remote sensing image segmentation system based on cross-paradigm feature fusion and alignment, characterized in that: A remote sensing image segmentation method based on cross-paradigm feature fusion and alignment according to any one of claims 1 to 7, comprising: an image acquisition module, an initial feature extraction module, a first-stage feature fusion module, a multi-stage cross-paradigm enhanced feature extraction module, and a prediction and optimization module; Image acquisition module, used to collect remote sensing images and perform preprocessing The initial feature extraction module is used to extract low-level texture and edge features from the preprocessed remote sensing image through the initial convolution layer to obtain the initial features; The first-stage feature fusion module is used to input the initial features into the constructed cross-paradigm feature fusion and alignment network, fuse the multimodal and cross-scale remote sensing image structural information, and obtain the first-stage fused features. The cross-paradigm feature fusion and alignment network includes a sparse channel enhancement and spatial alignment module and a spatial pixel refinement and channel alignment module; The multi-stage cross-paradigm enhanced feature extraction module is used to input the fused features of the first stage into the constructed multi-stage cross-paradigm enhanced feature extraction network. Through multi-level information interaction and dynamic gating mechanism, it integrates local details and global context information, and gradually extracts a joint feature map that collaboratively expresses semantics and spatial structure; The prediction and optimization module is used to generate the final semantic segmentation result through the segmentation head based on the joint feature map, and calculate the composite loss based on the true label to optimize the model parameters.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the remote sensing image segmentation method based on cross-paradigm feature fusion and alignment according to any one of claims 1 to 7 is implemented.

10. A processing terminal comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the remote sensing image segmentation method based on cross-paradigm feature fusion and alignment as described in any one of claims 1 to 7 is implemented.

Images