Remote sensing image reasoning segmentation method based on scene perception guide network

Abstract

The invention discloses a remote sensing image reasoning segmentation method based on a scene perception guide network, and the method is characterized in that the method comprises the following steps: 1, constructing a multi-scene reasoning segmentation remote sensing data set; step 2, constructing a scene perception guiding network based on an encoder-decoder structure, and processing the data set constructed in the step 1 to obtain a segmentation mask; 3, performing network training by adopting a self-adaptive scene perception loss function; according to the cross-scene generalization performance evaluation method, an efficient remote sensing image reasoning segmentation model is constructed through a scene perception guide mechanism and context-rich adaptive feature transformation, and the adaptation of semantic expressions in different scenes is guided by introducing scene cognition, so that the robustness of the remote sensing image reasoning segmentation model is improved. The problem that feature representation is inconsistent when a traditional method faces a complex geographical environment is effectively avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing and deep learning, in particular to a remote sensing image inference segmentation method based on a scene perception guided network. BACKGROUND

[0002] Remote sensing image inference segmentation is a method of segmenting specified targets in remote sensing images under the guidance of natural language expression. Due to its flexibility and user-friendliness, remote sensing image inference segmentation has become a promising direction for automatic remote sensing image interpretation. However, remote sensing images have the characteristics of wide coverage and large data volume. Typical remote sensing images often present complex geographical scene compositions within a single coverage area, including rural settlements, urban built-up areas, and urban-rural transition zones, and the distribution of land cover types in different geographical units shows significant heterogeneity.

[0003] Currently, remote sensing information extraction based on remote sensing image inference segmentation is still in its early stages. Existing research mainly focuses on improving the accuracy of small targets, adapting to the recognition of rotating targets, and aligning images and text. For example, some research has studied remote sensing image inference segmentation from a multi-scale perspective, proposing a language-guided cross-scale enhancement module to solve the small target segmentation problem in aerial images. Other research has addressed the challenge of directional diversity from a rotationally adaptive perspective, developing a network with complex interaction mechanisms and adaptive rotation convolution to handle targets at various angles. Still other research has focused on alignment angles, decomposing representations into target and location components to achieve more fine-grained language representations, thereby enabling better cross-modal matching. While current remote sensing image inference segmentation methods employ various strategies to improve the accuracy of remote sensing image information extraction within a single context and small image coverage area, real-world remote sensing images present fundamentally different challenges, as they typically cover a wide range of coverage areas and span multiple different geographic scenarios. Targets of the same class may exhibit certain visual feature differences in different geographic environments, making it difficult to achieve effective information extraction through a unified semantic guidance mechanism, thus requiring remote sensing image inference segmentation methods to have scene-aware capabilities when applied to large-scale remote sensing information extraction tasks. Due to limited computing resources, information extraction based on deep learning models inevitably requires image segmentation, which can result in the same geographic scenario being segmented into different image blocks. Notably, similar features of geographic objects may exhibit significant visual feature differences when they appear in different image blocks containing different geographic scenarios. This phenomenon makes it challenging to achieve effective information extraction through a unified semantic guidance mechanism when applying remote sensing image inference segmentation methods to large-scale remote sensing image segmentation, thus resulting in weak cross-scene generalization capabilities for such tasks. Existing methods mainly focus on pixel-level cognition, lacking comprehensive scene understanding capabilities, which significantly reduces cross-scene transfer performance. Specifically, when the same semantic class exhibits visual changes in different geographic regions, existing methods struggle to handle this cross-scene feature heterogeneity problem. At the same time, existing methods lack context-dependent semantic explanation capabilities, failing to provide scene-aware guidance based on different geographic environments. Furthermore, traditional self-attention mechanisms only operate in the spatial domain, limiting the model's ability to capture unique textures and spatial patterns in remote sensing images, exhibiting limited representation capabilities. These issues collectively result in weak generalization capabilities for existing methods when addressing large-scale remote sensing information extraction tasks, making it difficult to maintain consistent performance across different geographic regions. SUMMARY

[0004] The purpose of the present application is to provide a remote sensing image inference segmentation method based on a scene perception guide network, which introduces a scene context text guide module and a context-rich adaptive feature transformation module, effectively processes cross-scene feature heterogeneity through a scene perception mechanism, enhances the representation ability of the model through frequency domain processing technology, and significantly improves the cross-scene generalization performance of large-scale remote sensing information extraction tasks.

[0005] To achieve the above purpose, the present application provides the following technical solutions: A remote sensing image inference segmentation method based on a scene perception guide network, characterized by the following steps: Step 1, constructing a multi-scene inference segmentation remote sensing dataset; Step 2, constructing a scene perception guide network based on an encoder-decoder structure, and processing the dataset constructed in step 1 to obtain a segmentation mask; The construction of the scene perception guide network includes: Step 2.1, constructing a multi-scene type recognition and semantic mapping module: performing scene type recognition on the input remote sensing image to generate a scene type probability distribution map, and dynamically generating a dedicated text-visual mapping function for each recognized scene type to achieve scene-specific semantic alignment; Step 2.2, constructing a scene condition feature reconstruction module: based on the scene classification probability distribution, performing scene condition adaptive reconstruction on the visual features, and dynamically adjusting the feature representation using scene condition normalization technology; Step 2.3, constructing a scene-text joint guided frequency domain enhancement module: combining scene adaptive processing in the spatial domain and dynamic frequency selection mechanism in the frequency domain, wherein the frequency domain filter weight is generated jointly according to the scene type and the text semantics.

[0006] Step 3, network training using an adaptive scene perception loss function; The loss function includes: Step 3.1, pixel-level segmentation loss; Step 3.2, scene classification auxiliary loss; Step 3.3, cross-scene feature consistency constraint loss.

[0007] Step 4, cross-scene generalization performance evaluation.

[0008] As a further scheme of the present application, the step 1 includes the following steps: Step 1.1, obtaining high-resolution satellite remote sensing images of multiple time phases and multiple geographic environments; Step 1.2, performing scene type labeling and target segmentation labeling on the obtained remote sensing satellite images; Step 1.3, generating a natural language description based on the scene context, the description containing scene type information, target category, spatial position relationship and relevance to the scene; Step 1.4, constructing a triple data set containing image-mask-description.

[0009] As a further scheme of the present application: in step 1.2, the acquired remote sensing satellite image is preprocessed, the preprocessing including radiation calibration, atmospheric correction, geometric correction and orthographic correction; semi-automatic labeling is performed through an intelligent labeling tool to generate a preliminary segmentation mask; scene type determination is performed on each labeled region, and the scene is divided into N_c categories, the number of scene categories N_c being adaptively determined through data set geographical environment diversity analysis:

[0010] Wherein represents a geographical environment diversity index, represents a landscape type distribution feature, is the number of scene categories.

[0011] As a further scheme of the present application: in step 2.1, the multi-scene type recognition and semantic mapping module includes a scene type recognizer and a scene-specific semantic projector; the scene type recognizer adopts a multi-scale convolutional neural network structure to classify each spatial position in the input feature map into a predefined scene type, generating a scene type probability distribution map:

[0012] Wherein, is the generated scene type probability distribution map, is the multi-scene type recognition and semantic mapping module, is the input visual feature, is the number of scene categories, is the batch size, is the number of feature channels, and is the spatial dimension; The scene-specific semantic projector creates a dedicated text-visual mapping function for each scene type, unlike the traditional single projection layer, the projector generates mapping parameters jointly according to the scene type and the text semantics:

[0013]

[0014] Wherein represents the scene type index, represents the text projection layer of the i-th scene type, and These are the learnable projection weights and biases for the i-th scene type. This represents the projection of the i-th scene type. The global text representation is obtained through attention-weighted averaging:

[0015] in The text features representing the i-th word, These are weights learned through a text attention mechanism. It is the length of the text sequence. It is a small constant that prevents division by zero; For each spatial location, weighted fusion is performed based on the scene probability distribution:

[0016] in This indicates element-wise multiplication. Scene-specific text guidance features.

[0017] As a further aspect of the present invention: In step 2.2, the scene condition feature reconstruction module adopts scene condition normalization technology, which differs from traditional batch normalization or instance normalization. This technology dynamically adjusts the normalization parameters of the features according to the scene type. For input features The scene conditions are normalized and calculated as follows:

[0018] in and These are features mean and variance and These are learnable scaling and translation parameters related to scene type i; For feature maps with multiple scene probabilities, a probability-weighted scene conditional normalization is used:

[0019] The scene condition feature reconstruction process includes:

[0020] in These are scene-specific text guidance features derived from step 2.1; Preserve original feature information through residual connections:

[0021] in These are learnable residual weight parameters.

[0022] As a further scheme of the present application: in step 2.3, the scene-text joint guided frequency domain enhancement module includes an adaptive scene-aware filter and a dual-domain feature fusion mechanism; the adaptive scene-aware filter generates frequency domain filter weights according to the scene type, text semantics and input feature dimensions, which is different from the traditional filter which only relies on input features:

[0023] wherein is the normalized frequency domain coordinate, is a weight generation network with learnable parameters , which receives the frequency domain coordinate, scene probability distribution and global text representation as input; The frequency domain feature processing procedure includes: Step 2.3.1, feature dimension reduction: the input feature with dimension C is reduced to intermediate feature with dimension C / r through scene adaptive convolution:

[0024] Step 2.3.2, frequency domain transformation: apply two-dimensional real fast Fourier transform:

[0025] Step 2.3.3, scene-aware frequency domain filtering: scene and text-aware adaptive filtering in the frequency domain:

[0026] Step 2.3.4, inverse transform back to spatial domain:

[0027] Step 2.3.5, frequency domain attention enhancement: adaptive enhancement by learning frequency domain importance weights:

[0028]

[0029] wherein represents the Sigmoid activation function; Dual-domain feature fusion: fuse the enhanced frequency domain features with the original spatial domain features:

[0030] Multi-stage integration: integrate the module in multiple stages of the decoder using residual connection:

[0031] wherein The processing function of the scene-text joint guided frequency domain enhancement module.

[0032] As a further scheme of the present application: the cross-scene adaptive guidance network further comprises a cross-scale scene consistency module for maintaining consistency of scene understanding at different feature pyramid levels: for different levels of the feature pyramid , scene probability distributions are respectively generated; and the cross-scale scene consistency constraint is calculated after the scene probability distributions of different scales are unified to the same spatial resolution through bilinear interpolation.

[0033] As a further scheme of the present application: in step 2, the cross-scene adaptive guidance network adopts an encoder-decoder structure, comprising a visual encoder, a text encoder, a multi-scene type recognition and semantic mapping module, a scene condition feature reconstruction module, a scene-text joint guided frequency domain enhancement module, and a segmentation decoder; a Swin Transformer is used as the visual encoder to extract multi-scale visual features ; a BERT is used as the text encoder to encode the natural language description into a text feature sequence . The forward propagation process of the network is: visual and text feature extraction; scene type recognition (at each scale); scene-specific semantic mapping; scene condition feature reconstruction (at each layer of the encoder); cross-scale scene consistency feature interaction (bidirectional path); application of scene-text joint guided frequency domain enhancement at each stage of the decoder; and generation of the final segmentation mask.

[0034] As a further scheme of the present application: in step 3, an adaptive scene-aware loss function is used to train the network end-to-end, and the total loss function is defined as:

[0035] wherein is the segmentation loss, which adopts a combination of cross-entropy loss and Dice loss:

[0036] wherein is the number of pixels, is the number of categories, is the real label, is the predicted probability. The first term is the standard cross-entropy loss, and the second term is the Dice loss; is the scene classification auxiliary loss, which is used to supervise the training of the scene type recognizer:

[0037] wherein The number of spatial locations in the training samples. For scene type real labels, Predict the probability for scene type; The loss is designed to constrain cross-scene feature consistency, including cross-scale scene consistency and feature consistency within the same scene:

[0038] Among them, the consistency loss across scale scenarios is:

[0039] in It is the number of levels in the feature pyramid. Denotes KL divergence, and They represent the first Layer and first The scene probability distribution of the layer (unified to the same spatial resolution through bilinear interpolation); Feature consistency loss within the same scene:

[0040] in Represents variance. Indicates the location in the diagram eigenvectors, Indicates position Belongs to scene type The probability, It is the scene probability threshold; Optimize the loss for boundary quality to improve the accuracy of boundary segmentation:

[0041] in Number of boundary pixels; weighting coefficients The values ​​were determined through performance tuning of the validation set and, in this embodiment, were set to {1.0, 0.3, 0.2, 0.5}. As a further aspect of the present invention: In step 4, the cross-geographical region generalization capability of the cross-scene adaptive guidance network is verified through multiple dimensions: Evaluation on standard datasets: Tests were conducted on the RefSegRS, RRSIS-D, and LandRef datasets, using the following evaluation metrics: mean intersection-over-union ratio (mIoU); overall intersection-over-union ratio (oIoU); and accuracy at different IoU thresholds (P@0.5 to P@0.9). Cross-geographic region testing: Remote sensing images of geographic regions not covered during training were selected for testing. Large-scale remote sensing images (size range 20,000×17,000 to 27,444×26,692 pixels) were used to perform target segmentation without any region-specific fine-tuning. Qualitative analysis: The rationality of the scene probability distribution map, the differences in feature responses of different scene types, the segmentation quality of cross-scene boundary regions, and the attention heatmap jointly guided by scene and text are evaluated through visualization methods. Ablation study: By removing the multi-scene type recognition and semantic mapping module, removing the scene conditional feature reconstruction module, removing the scene-text jointly guided frequency domain enhancement module, and analyzing the number of different scene categories. The impact on performance is verified by examining the independent contributions of each module; Cross-scene consistency assessment: Calculate the feature similarity and segmentation accuracy variance of the same target category in different scenarios to verify the cross-scene generalization ability.

[0042] Compared with the prior art, the beneficial effects of the present invention are: 1. This application achieves accurate understanding and execution of complex natural language reference expressions through a deep vision-language fusion architecture, and can handle diverse semantic descriptions including categories, attributes, and spatial relationships. At the same time, through a multimodal collaborative training strategy, it significantly improves the model's cross-scene generalization ability and maintains stable performance in completely unseen geographical areas, providing a reliable guarantee for practical applications.

[0043] 2. This application explicitly classifies remote sensing images into different geographic scene types through a multi-scene type recognizer and generates dedicated processing parameters for each scene type to achieve feature reconstruction of scene conditions, rather than simple spatial transformation or feature enhancement.

[0044] 3. This application innovatively combines scene type recognition results with text semantics to create dedicated text-visual mapping functions for different scene types, achieving scene-specific semantic alignment rather than unified text-visual fusion.

[0045] 4. The frequency domain filter weights in this application not only depend on the input features, but are also jointly controlled by the scene type and text semantics, thereby realizing scene-aware frequency domain adaptive processing, rather than an independent frequency domain transformation operation. Attached Figure Description

[0046] Fig. 1 This is a flowchart of this embodiment; Fig. 2 This is a model framework diagram for this embodiment; Fig. 3In the middle, the left side (a)-(d) represent the segmentation results of farmland under different scenarios, and the right side (a)-(d) represent the segmentation results of towns under different scenarios. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Please see Figs. 1-3 In this embodiment of the invention, a remote sensing image reasoning and segmentation method based on a scene-aware guided network includes the following steps: Step 1: Construct a multi-scene inference segmentation remote sensing dataset. In this embodiment, a multi-scene remote sensing inference segmentation dataset is constructed. Step 1.1: Acquire high-resolution satellite remote sensing images with multiple temporal and geographical environments, including images from the Gaofen series, Ziyuan-3, and Jilin-1 satellites, with a spatial resolution ranging from 0.65 to 2 meters, covering northern, central, and southern Anhui Province to capture different landscape patterns. The northern region is mainly agricultural plains, the central region includes a mix of urban areas and hills, and the southern region is characterized by mountainous terrain. Images acquired in different seasons between 2020 and 2022 were selected to ensure temporal diversity, taking into account seasonal variations such as vegetation phenology, water level changes, and urban development patterns. Standard preprocessing procedures were performed on all satellite images, including radiometric calibration, atmospheric correction, geometric correction, and orthorectification. Step 1.2: Semi-automatic annotation is performed using the SAM (Segment Anything Model) intelligent annotation tool. First, the preprocessed remote sensing image is cropped to a standard size of 512×512 pixels. Then, the SAM model is used for preliminary segmentation to generate candidate region masks. For six land cover types—woodland, grassland, wetland, cultivated land, urban areas, and roads—the masks generated by SAM are refined and corrected through manual interaction to ensure annotation quality and accuracy. SAM (Self-Automatic Annotation Tool) is an open-source tool that, with specific modifications to certain functions, adapts to our data and performs semi-automatic annotation. In the field of remote sensing image processing, semi-automatic annotation has become a very common and efficient method, especially for constructing large-scale remote sensing datasets. Using SAM for semi-automatic annotation can significantly reduce the workload of manual annotation and improve annotation efficiency and consistency.

[0049] Step 1.3 Automated description generation using large language models such as DeepSeek. The labeled remote sensing imagery and its corresponding segmentation mask are input into the DeepSeek model, which automatically generates natural language descriptions through multimodal understanding. The descriptions include target category, appearance features, spatial relationships, relative size features, and association with surrounding features.

[0050] Step 1.4: The set of labeled and naturally described remote sensing satellite images is used as the dataset. In this embodiment, a complete image-mask-description triplet dataset is constructed. Each remote sensing image is combined with its corresponding pixel-level segmentation mask and natural language description to form a complete training sample. Finally, a LandRef dataset containing 14,391 triplets is constructed, divided into a training set of 70%, a validation set of 10%, and a test set of 20% to ensure that samples from different geographical regions and seasons are evenly distributed across the subsets.

[0051] Step 2: Construct a scene-aware guidance network based on an encoder-decoder structure A scene-aware guided network based on an encoder-decoder structure is constructed, including a visual encoder, a text encoder, a multi-scene type recognition and semantic mapping module, a scene conditional feature reconstruction module, a frequency domain enhancement module jointly guided by scene and text, and a segmentation decoder. A Swin Transformer is used as the visual encoder to extract multi-scale visual features. Using BERT as a text encoder, natural language descriptions are encoded into text feature sequences. .

[0052] Constructing a scene-aware guidance network based on an encoder-decoder structure includes the following steps: Step 2.1: Construct a multi-scene type recognition and semantic mapping module: Perform scene type recognition on the input remote sensing image, generate a scene type probability distribution map, and dynamically generate a dedicated text-visual mapping function for each recognized scene type to achieve scene-specific semantic alignment; The multi-scene type recognition and semantic mapping module comprises two core components: a scene type recognizer and a scene-specific semantic projector. The scene type recognizer employs a multi-scale convolutional neural network structure to classify each spatial location in the input feature map into a predefined scene type, generating a scene type probability distribution map. Considering that the same land cover category (e.g., wetlands, farmland, urban areas) may exhibit significantly different visual features under different geographical backgrounds, the scene type recognizer is implemented through the following process:

[0053] in, It is a probability distribution map of the generated scene types. This is a module for multi-scene type recognition and semantic mapping. Represents the visual features of the input. For the number of scene categories, For batch size, The number of feature channels, and Spatial dimension; A scene-specific semantic projector creates a dedicated text-to-visual mapping function for each scene type. Unlike traditional single projection layers, this projector jointly generates mapping parameters based on scene type and text semantics. This design is based on an important observation: the same text description may point to different visual features in different geographic environments. First, a global text representation is obtained through attention-weighted averaging.

[0054] in The text features representing the i-th word, These are weights learned through a text attention mechanism. It is the length of the text sequence. It is a small constant to prevent division by zero. Then, a dedicated text projection is generated for each scene type:

[0055]

[0056] in Indicates the scene type index. The text projection layer representing the i-th scene type. and These are the learnable projection weights and biases for the i-th scene type. Let represent the projection of the i-th scene type. For each spatial location, weighted fusion is performed based on the scene probability distribution:

[0057] in This indicates element-wise multiplication. Scene-specific text guidance features.

[0058] Step 2.2: Constructing a Scene Conditional Feature Reconstruction Module: Based on the scene classification probability distribution, adaptive reconstruction of visual features is performed under scene conditions. Scene conditional normalization technology is used to dynamically adjust the feature representation. The scene conditional feature reconstruction module employs scene conditional normalization technology, which differs from traditional batch normalization or instance normalization. This technology dynamically adjusts the normalization parameters of the features according to the scene type. For input features The scene conditions are normalized and calculated as follows:

[0059] in and These are features mean and variance and These are learnable scaling and translation parameters related to scene type i; For feature maps with multiple scene probabilities, a probability-weighted scene conditional normalization is used:

[0060] The scene condition feature reconstruction process includes:

[0061] in These are scene-specific text guidance features derived from step 2.1; Preserve original feature information through residual connections:

[0062] in These are learnable residual weight parameters.

[0063] Step 2.3: Construct a scene-text jointly guided frequency domain enhancement module: This module combines scene adaptive processing in the spatial domain with a dynamic frequency selection mechanism in the frequency domain. The frequency domain filter weights are jointly generated based on scene type and text semantics. The adaptive scene-aware filter generates frequency domain filter weights based on scene type, text semantics, and input feature dimensions, unlike traditional filters that rely solely on input features.

[0064] in These are normalized frequency domain coordinates. It has learnable parameters A weight generation network that receives frequency domain coordinates and scene probability distribution. and global text representation As input, the weight generation network employs a multilayer perceptron (MLP) structure, dynamically adjusting the frequency domain filtering strategy by learning the joint representation of the scene and the text; The frequency domain feature processing flow includes: Step 2.3.1, Feature Dimensionality Reduction: The input features of dimension C are reduced to intermediate features of dimension C / r through scene-adaptive convolution.

[0065] Step 2.3.2, Frequency Domain Transformation: Applying Two-Dimensional Real-Number Fast Fourier Transform:

[0066] Step 2.3.3, Scene-Aware Frequency Domain Filtering: Adaptive filtering for scene and text awareness is performed in the frequency domain.

[0067] Step 2.3.4: Inverse transformation back to the spatial domain:

[0068] Step 2.3.5, Frequency Domain Attention Enhancement: Adaptive enhancement is performed by learning frequency domain importance weights.

[0069]

[0070] in This represents the Sigmoid activation function; Dual-domain feature fusion: fusing enhanced frequency domain features with original spatial domain features.

[0071] Multi-stage integration: This module is integrated using residual connections across multiple stages of the decoder.

[0072] in This represents the processing function of the frequency domain enhancement module guided by scene-text joint principles. In this embodiment, the module is integrated into four different resolution stages of the decoder, progressively refining the feature representation through progressive frequency domain enhancement. It captures fine-grained details at higher resolutions and captures global contextual information at lower resolutions.

[0073] Step 3: Train the network using an adaptive scene-aware loss function. This invention employs an end-to-end training strategy to optimize the scene-aware guidance network, using an adaptive scene-aware loss function for network training. The total loss function is defined as:

[0074] in For the segmentation loss, a combination of cross-entropy loss and Dice loss is used:

[0075] in For the number of pixels, The number of categories (in this embodiment, the number of target categories plus the number of background categories), For real labels, This is for predicting probabilities. The first term is the standard cross-entropy loss, and the second term is the Dice loss. The Dice loss is more sensitive to small targets and boundary pixels, and can effectively improve the accuracy of segmentation boundaries.

[0076] The scene classification auxiliary loss is used to supervise the training of the scene type recognizer:

[0077] in The number of spatial locations in the training samples. For scene type real labels, The auxiliary loss predicts probabilities for scene types. This ensures that the scene type recognizer can accurately identify different geographic environment types, providing reliable scene information for subsequent scene adaptation processing.

[0078] The loss is designed to constrain cross-scene feature consistency, including cross-scale scene consistency and feature consistency within the same scene:

[0079] Among them, cross-scale scene consistency loss:

[0080] in It is the number of levels in the feature pyramid. Denotes KL divergence, and They represent the first Layer and first The scene probability distribution of each layer is unified to the same spatial resolution through bilinear interpolation. This loss enables features at different scales to make consistent predictions about scene types, avoiding conflicts in scene understanding between multi-scale features.

[0081] Feature consistency loss within the same scene:

[0082] in Represents variance. Indicates the location in the diagram eigenvectors, Indicates position Belongs to scene type The probability, This is the scene probability threshold. This loss, by minimizing the variance of features within the same scene, prompts the model to learn a more compact and discriminative scene-specific feature representation.

[0083] Optimize the loss for boundary quality to improve the accuracy of boundary segmentation:

[0084] in This represents the number of boundary pixels; the boundary pixels are obtained by performing morphological operations (dilation-erosion) on the real mask. Boundary position The predicted probability at the location. This loss makes the model pay more attention to the accuracy of segmentation boundaries, which is especially important for the accurate extraction of target boundaries in remote sensing images.

[0085] Weighting coefficient settings: Weighting coefficient The weights were determined through performance tuning on the validation set and, in this embodiment, were set to {1.0, 0.3, 0.2, 0.5}. These weights balance the contributions of different loss terms. Maximizing segmentation accuracy is the primary optimization objective. and The settings ensure that the scene perception mechanism is fully trained. The settings emphasize the importance of boundary quality.

[0086] Training Strategy: The AdamW optimizer was used for model training, with weight decay set to 0.01 to prevent overfitting. Different learning rate settings were adopted for different datasets: for the RefSegRS dataset, the initial learning rate was set to 0.00005, and training was conducted for 50 epochs; for the RRSIS-D and LandRef datasets, the initial learning rate was set to 0.00003, and training was conducted for 40 epochs. The batch size was set to 8, and a cosine annealing learning rate scheduling strategy was used to dynamically adjust the learning rate during training. Data augmentation strategies were employed during training, including random flipping, rotation (0°, 90°, 180°, 270°), scaling (0.8-1.2x), and color jittering, to enhance the model's generalization ability. During training, the mean Intersection over Union (mIoU), overall Intersection over Union (oIoU), and accuracy metrics at different IoU thresholds on the validation set were monitored. When the validation set performance did not show a significant improvement within 10 consecutive epochs, an early stopping mechanism was triggered to prevent overfitting. The model parameters were initialized using pre-trained weights. The visual encoder used Swin Transformer weights pre-trained on ImageNet-22K, the text encoder used pre-trained BERT weights, and new modules were initialized using Xavier.

[0087] Step 4: Cross-scenario generalization performance evaluation This invention verifies the cross-scene generalization ability of the scene-aware guided network through multiple dimensions. First, benchmark tests are conducted on standard datasets, including RefSegRS, RRSIS-D, and a self-constructed dataset. Standard evaluation metrics are employed, including mean Intersection over Union (mIoU), overall Intersection over Union (oIoU), and accuracy metrics (P@0.5 to P@0.9) at different IoU thresholds. mIoU calculates the average IoU value of all test samples, oIoU calculates the ratio of the total intersection area to the total union area, and the accuracy metric measures the proportion of test samples that meet a specified IoU threshold. To verify true cross-scene generalization ability, a test experiment is designed for completely unseen geographical areas. Large-scale remote sensing images (20,000×17,000 pixel urban area images and 27,444×26,692 pixel agricultural area images) are used for testing, requiring the model to perform target segmentation tasks without any region-specific fine-tuning. Performance was evaluated using a combination of qualitative and quantitative analysis. Qualitative analysis included visualizing segmentation results, boundary accuracy, and error case analysis, while quantitative analysis included comparative experiments with state-of-the-art methods and ablation studies. Ablation studies validated the independent contributions of the multi-scene type recognition and semantic mapping module, the scene conditional feature reconstruction module, and the scene-text jointly guided frequency domain enhancement module. The impact of each module on overall performance was quantified by progressively removing or adding different components. Furthermore, the impact of the number of different scene categories on model performance was analyzed to determine the optimal scene segmentation granularity. The working mechanisms of different modules were visualized using feature maps to verify the effectiveness of the scene perception mechanism and frequency domain processing. Evaluation results show that the proposed method significantly outperforms existing methods in all test scenarios, particularly demonstrating a significant improvement in high-precision threshold metrics. It maintains stable segmentation quality in completely unseen geographical areas, effectively addressing the insufficient generalization ability of existing methods in cross-scene applications.

[0088] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A remote sensing image reasoning and segmentation method based on a scene-aware guided network, characterized in that, Includes the following steps: Step 1: Construct a multi-scenario inference segmentation remote sensing dataset; Step 2: Construct a scene-aware guidance network based on an encoder-decoder structure, and process the dataset constructed in Step 1 to obtain the segmentation mask; The construction of the scene-aware guidance network includes: Step 2.1: Construct a multi-scene type recognition and semantic mapping module: Perform scene type recognition on the input remote sensing image, generate a scene type probability distribution map, and dynamically generate a dedicated text-visual mapping function for each recognized scene type to achieve scene-specific semantic alignment; Step 2.2: Construct a scene condition feature reconstruction module: Based on the scene classification probability distribution, adaptively reconstruct the visual features according to scene conditions, and dynamically adjust the feature representation using scene condition normalization technology; Step 2.3: Construct a scene-text jointly guided frequency domain enhancement module: combining scene adaptive processing in the spatial domain and dynamic frequency selection mechanism in the frequency domain, wherein the frequency domain filter weights are jointly generated based on scene type and text semantics; Step 3: Train the network using an adaptive scene-aware loss function; Adaptive scene-aware loss functions include: Step 3.1, pixel-level segmentation loss; Step 3.2, Scene classification auxiliary loss; Step 3.3, Cross-scene feature consistency constraint loss; Step 4: Cross-scenario generalization performance evaluation.

2. The remote sensing image reasoning and segmentation method based on a scene-aware guided network according to claim 1, characterized in that, Step 1 includes the following steps: Step 1.1: Acquire high-resolution satellite remote sensing images with multiple temporal phases and geographical environments; Step 1.2: Perform scene type labeling and target segmentation labeling on the acquired remote sensing satellite images; Step 1.3: Generate a natural language description based on the scene context. The description includes scene type information, target category, spatial location relationship, and relevance to the scene. Step 1.4: Construct a dataset of triples containing image-mask-description.

3. The remote sensing image reasoning and segmentation method based on a scene-aware guided network according to claim 2, characterized in that, In step 1.2, the acquired remote sensing satellite imagery is preprocessed, including radiometric calibration, atmospheric correction, geometric correction, and orthorectification. Semi-automatic annotation is performed using an intelligent annotation tool to generate a preliminary segmentation mask. For each annotated area, scene type is determined, categorizing scenes into N_c classes. The number of scene classes N_c is adaptively determined through dataset geographic environment diversity analysis. in Indicators representing geographical environmental diversity Indicates the distribution characteristics of landscape types, This represents the number of scene categories.

4. The remote sensing image reasoning and segmentation method based on a scene-aware guided network according to claim 1, characterized in that, In step 2.1, the multi-scene type recognition and semantic mapping module includes a scene type recognizer and a scene-specific semantic projector; the scene type recognizer adopts a multi-scale convolutional neural network structure to classify each spatial location in the input feature map into a predefined scene type, generating a scene type probability distribution map: in, It is a probability distribution map of the generated scene types. This is a module for multi-scene type recognition and semantic mapping. Represents the visual features of the input. For the number of scene categories, For batch size, The number of feature channels, and Spatial dimension; The scene-specific semantic projector creates a dedicated text-to-visual mapping function for each scene type. Unlike traditional single projection layers, this projector jointly generates mapping parameters based on scene type and text semantics. in Indicates the scene type index. The text projection layer representing the i-th scene type. and These are the learnable projection weights and biases for the i-th scene type. This represents the projection of the i-th scene type. The global text representation is obtained through attention-weighted averaging: in The text features representing the i-th word, These are weights learned through a text attention mechanism. It is the length of the text sequence. It is a small constant that prevents division by zero; For each spatial location, weighted fusion is performed based on the scene probability distribution: in This indicates element-wise multiplication. Scene-specific text guidance features.

5. The remote sensing image reasoning and segmentation method based on a scene-aware guided network according to claim 1, characterized in that, In step 2.2, the scene condition feature reconstruction module uses scene condition normalization technology to dynamically adjust the normalization parameters of the features according to the scene type: For input features The scene conditions are normalized and calculated as follows: in and These are features mean and variance and These are learnable scaling and translation parameters related to scene type i; For feature maps with multiple scene probabilities, a probability-weighted scene conditional normalization is used: The scene condition feature reconstruction process includes: in These are scene-specific text guidance features derived from step 2.1; Preserve original feature information through residual connections: in These are learnable residual weight parameters.

6. The remote sensing image reasoning and segmentation method based on a scene-aware guided network according to claim 1, characterized in that, In step 2.3: the scene-text jointly guided frequency domain enhancement module includes an adaptive scene-aware filter and a dual-domain feature fusion mechanism; Adaptive scene-aware filters jointly generate frequency domain filter weights based on scene type, text semantics, and input feature dimensions, unlike traditional filters that rely solely on input features. in These are normalized frequency domain coordinates. It has learnable parameters A weight generation network that receives frequency domain coordinates and scene probability distribution. and global text representation As input; The frequency domain feature processing flow includes: Step 2.3.1, Feature Dimensionality Reduction: The input features of dimension C are reduced to intermediate features of dimension C / r through scene-adaptive convolution. Step 2.3.2, Frequency Domain Transformation: Applying Two-Dimensional Real-Number Fast Fourier Transform: Step 2.3.3, Scene-Aware Frequency Domain Filtering: Adaptive filtering for scene and text awareness is performed in the frequency domain. Step 2.3.4: Inverse transformation back to the spatial domain: Step 2.3.5, Frequency Domain Attention Enhancement: Adaptive enhancement is performed by learning frequency domain importance weights. in This represents the Sigmoid activation function; Dual-domain feature fusion: fusing enhanced frequency domain features with original spatial domain features. Multi-stage integration: This module is integrated using residual connections across multiple stages of the decoder. in This represents the processing function of the frequency domain enhancement module guided by the scene-text joint approach.

7. The remote sensing image reasoning and segmentation method based on a scene-aware guided network according to claim 1, characterized in that, The cross-scene adaptive guidance network also includes a cross-scale scene consistency module, used to maintain consistency in scene understanding at different feature pyramid levels: for different levels of the feature pyramid Generate scene probability distributions respectively After unifying the probability distributions of scenes at different scales to the same spatial resolution through bilinear interpolation, cross-scale scene consistency constraints are calculated.

8. The remote sensing image reasoning and segmentation method based on a scene-aware guided network according to claim 1, characterized in that, In step 2, the cross-scene adaptive guidance network adopts an encoder-decoder structure, including a visual encoder, a text encoder, a multi-scene type recognition and semantic mapping module, a scene condition feature reconstruction module, a scene-text joint guidance frequency domain enhancement module, and a segmentation decoder; a Swing Transformer is used as the visual encoder to extract multi-scale visual features. ; BERT is used as a text encoder to encode natural language descriptions into text feature sequences. The forward propagation process of the network is as follows: visual and textual feature extraction; scene type identification at each scale; scene-specific semantic mapping; scene condition feature reconstruction at each layer of the encoder; cross-scale scene consistency feature interaction of the bidirectional path; Application scenarios in various stages of the decoder - frequency domain enhancement guided by text; Generate the final segmentation mask.

9. The remote sensing image reasoning and segmentation method based on a scene-aware guided network according to claim 1, characterized in that, In step 3, an adaptive scene-aware loss function is used to train the network end-to-end. The total loss function is defined as follows: in For the segmentation loss, a combination of cross-entropy loss and Dice loss is used: in For the number of pixels, For the number of categories, For real labels, To predict probabilities. The first term is the standard cross-entropy loss, and the second term is the Dice loss; The scene classification auxiliary loss is used to supervise the training of the scene type recognizer: in The number of spatial locations in the training samples. For scene type real labels, Predict the probability for scene type; The loss is designed to constrain cross-scene feature consistency, including cross-scale scene consistency and feature consistency within the same scene: Among them, the consistency loss across scale scenarios is: in It is the number of levels in the feature pyramid. Indicates the KL divergence. and They represent the first Layer and first The scene probability distribution of the layer (unified to the same spatial resolution through bilinear interpolation); Feature consistency loss within the same scene: in Represents variance. Indicates the location in the diagram eigenvectors, Indicates position Belongs to scene type The probability, It is the scene probability threshold; Optimize the loss for boundary quality to improve the accuracy of boundary segmentation: in Number of boundary pixels; weighting coefficients The values ​​were determined through performance tuning of the validation set and are set to {1.0, 0.3, 0.2, 0.5} in this embodiment. Evaluation on standard datasets: Tests were conducted on the RefSegRS, RRSIS-D, and LandRef datasets, using the following evaluation metrics: mean intersection-over-union ratio (mIoU); overall intersection-over-union ratio (oIoU); and accuracy at different IoU thresholds (P@0.5 to P@0.9). Cross-geographic region testing: Select remote sensing images of geographic regions that were not covered during training for testing. Use large-scale remote sensing images to perform target segmentation without any region-specific fine-tuning. Qualitative analysis: The rationality of the scene probability distribution map, the differences in feature responses of different scene types, the segmentation quality of cross-scene boundary regions, and the attention heatmap jointly guided by scene and text are evaluated through visualization methods. Ablation study: By removing the multi-scene type recognition and semantic mapping module, removing the scene conditional feature reconstruction module, removing the scene-text jointly guided frequency domain enhancement module, and analyzing the number of different scene categories. The impact on performance is verified by examining the independent contributions of each module; Cross-scene consistency assessment: Calculate the feature similarity and segmentation accuracy variance of the same target category in different scenarios to verify the cross-scene generalization ability.

Images