Remote sensing image segmentation method based on foreground sensing network

Abstract

The invention relates to the technical field of remote sensing image segmentation, and discloses a remote sensing image segmentation method based on a foreground sensing network, and the method comprises the steps: constructing the foreground sensing network based on an encoder and decoder structure, and constructing a combined loss function; training the foreground sensing network; segmenting by using the trained foreground sensing network to obtain a category probability graph of each pixel; and based on the dynamic multi-resolution attention, performing multi-scale grading on the category probability graph in combination with the environmental characteristics, dynamically adapting a post-processing strategy, and performing post-processing according to the adapted post-processing strategy to optimize a segmentation result. According to the method, an end-to-end intelligent segmentation framework is constructed through multi-scale feature fusion, environment adaptive post-processing and geographical semantic constraint, the segmentation precision, robustness and practicability in a complex remote sensing scene are remarkably improved, and an efficient solution is provided for the fields of natural resource management, disaster emergency response and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image segmentation, and in particular to a remote sensing image segmentation method based on a foreground perception network. Background Art

[0002] With the rapid advancement of high-resolution remote sensing Earth observation technology, the spatial resolution of satellite data continues to increase. The precise extraction of surface features from high-resolution satellite remote sensing imagery has become a key research topic in remote sensing image visual information processing. Remote sensing imagery features a wide coverage area, complex and diverse object types, and fuzzy and difficult-to-discern object boundaries. Traditional machine learning-based remote sensing object extraction methods often rely on manually designed features, resulting in limited extraction accuracy and weak cross-scene transfer capabilities.

[0003] The rise of deep learning technologies, represented by convolutional neural networks, has greatly revolutionized the technical path of remote sensing feature extraction, significantly improving extraction accuracy and processing efficiency. The "encoder-decoder" architecture used by fully convolutional neural networks has pushed remote sensing image segmentation technology to a new level. Based on this classic architecture, a series of new networks have emerged. Most of these networks use a general backbone network as an encoder, such as VGG, ResNet deep residual network, and DenseNet dense convolutional network. By performing multi-level convolution operations on the input image, feature maps at different levels are generated; at the same time, by designing a differentiated decoder structure, the multi-layer features output by the encoder are cross-layer fused and semantically interpreted, and finally pixel-level segmentation results are output.

[0004] Although significant progress has been made in existing research, existing methods generally face the following challenges due to the complex scene characteristics of high-resolution remote sensing images:

[0005] 1) Category imbalance problem: The pixel ratios of foreground objects (such as buildings and roads) and background objects (such as vegetation and water bodies) in remote sensing images are significantly different. Traditional cross-entropy loss can easily lead to a decrease in the segmentation accuracy of the model for small objects.

[0006] 2) Difficulty in processing multi-scale targets: Different objects (such as small vehicles and large buildings) have large scale differences. Single-resolution feature extraction is difficult to take into account both details and global semantics at the same time, which easily leads to the loss of small targets or the blurring of the boundaries of large targets.

[0007] 3) Insufficient robustness to environmental interference: Environmental factors such as cloud occlusion, sensor noise, and spectral confusion (such as different objects with the same spectrum) can significantly reduce segmentation accuracy. Traditional post-processing strategies (such as morphological operations with fixed parameters) cannot dynamically adapt to complex scenarios.

[0008] 3) Lack of geographic semantic consistency: Existing methods lack explicit constraints on geographic rules (such as “no buildings should appear around water bodies”), which may cause the segmentation results to violate the actual distribution patterns of land objects.

[0009] Currently, no effective solutions have been proposed for the problems in related technologies. Summary of the Invention

[0010] In response to the problems in the related art, the present invention proposes a remote sensing image segmentation method based on a foreground perception network to overcome the above technical problems existing in the existing related art.

[0011] To this end, the specific technical solutions adopted in the present invention are as follows:

[0012] The present invention provides a remote sensing image segmentation method based on a foreground perception network, comprising the following steps:

[0013] S1. Obtain the remote sensing image to be segmented and preprocess it. Based on the encoder and decoder structure, build a foreground perception network and construct a combined loss function based on weighted cross entropy loss and Dice loss to alleviate the imbalance problem between foreground and background categories.

[0014] S2. Dynamically adjust the learning rate based on the optimizer and use the remote sensing dataset to train the foreground perception network; use the trained foreground perception network to segment the preprocessed remote sensing image to obtain the category probability map of each pixel;

[0015] S3. Based on dynamic multi-resolution attention, the category probability map is multi-scale graded in combination with environmental features, the post-processing strategy is dynamically adapted, and post-processing is performed according to the adapted post-processing strategy to optimize the segmentation results.

[0016] Preferably, the remote sensing image to be segmented is obtained and preprocessed, a foreground perception network is constructed based on an encoder and decoder structure, and a combined loss function is constructed based on weighted cross entropy loss and Dice loss to alleviate the imbalance problem between foreground and background categories, including the following steps:

[0017] S11. Obtain a remote sensing image to be segmented, and perform normalization and cropping on the remote sensing image; convert pixel-level segmentation labels into one-hot encoding, and perform morphological operations on the labels to optimize the continuity of the foreground boundary;

[0018] S12. Adopt the encoder and decoder structure, embed the foreground perception module and multi-scale feature fusion mechanism, and build a foreground perception network; construct a combined loss function based on weighted cross entropy loss and Dice loss to alleviate the imbalance problem between foreground and background categories.

[0019] Preferably, the embedded foreground perception module and the multi-scale feature fusion mechanism include:

[0020] A dual-channel attention mechanism consisting of channel attention and spatial attention is introduced into the high-level features of the encoder to enhance the target spectral characteristics and suppress background noise; an expanded convolution or feature pyramid network is added to the encoder to capture foreground targets of different scales.

[0021] Preferably, the method of dynamically adjusting the learning rate based on the optimizer, training the network foreground perception network using the remote sensing data set, and segmenting the pre-processed remote sensing image to be segmented using the trained foreground perception network to obtain the category probability map of each pixel includes the following steps:

[0022] S21. Initialize the learning rate using the optimizer and adjust the learning rate using the cosine annealing algorithm. Train the foreground perception network based on the adjusted learning rate and the remote sensing dataset.

[0023] S22. Input the pre-processed remote sensing image to be segmented into a trained foreground perception network, and use the foreground perception network to output a category probability map of each pixel.

[0024] Preferably, the method of performing multi-scale classification of the category probability map based on dynamic multi-resolution attention in combination with environmental features, dynamically adapting the post-processing strategy, and performing post-processing according to the adapted post-processing strategy to optimize the segmentation result includes the following steps:

[0025] S31, obtaining a class probability map and an original remote sensing image, and extracting environmental parameters of the original remote sensing image, wherein the environmental parameters include occlusion characteristics, noise characteristics, resolution characteristics, and spectral characteristics;

[0026] S32, extracting the connected regions of the foreground objects by binarizing the class probability map, performing basic scale classification based on the area of the connected regions, and recording the geometric features of each connected region; correcting the basic scale classification based on environmental interference, and generating a mask map that marks the optimal processing resolution of each region;

[0027] S33. Use the decision model to divide the environmental parameters into processing modes and dynamically adjust the post-processing parameters according to different processing modes; process targets of different scales in parallel according to the resolution mask map and use the attention mechanism to fuse the processing results;

[0028] S34. Perform geographic rule verification and environment-specific repair on the fused preliminary segmentation results, and dynamically select the morphological kernel size for opening and closing operations based on the noise intensity to obtain the final segmentation result that conforms to geographic semantics and visual continuity.

[0029] Preferably, the extraction of the occlusion feature includes: detecting cloud layers or shadow areas based on HSV or Lab color space, and calculating the occlusion rate to determine;

[0030] The extraction of the noise feature includes: evaluating the noise intensity by local pixel gradient variance or median absolute deviation, and marking high noise areas for determination;

[0031] Extraction of the resolution feature includes: for UAV images, determining the actual ground resolution by combining the flight altitude or focal length; for satellite images, determining the spatial resolution by extracting the sensor metadata;

[0032] The extraction of the spectral features includes: determining by calculating the correlation between multi-spectral bands and identifying the risk area of heteromorphic objects in the same spectrum.

[0033] Preferably, performing basic scale classification according to the area of the connected regions and recording the geometric features of each connected region; correcting the basic scale classification according to environmental interference and generating a mask map marking the optimal processing resolution of each region includes:

[0034] Divide the foreground object connected region into the first target, the second target, and the third target according to the area of the connected region, and record the geometric features of each foreground object connected region; wherein the pixels of the first target, the second target, and the third target increase in sequence;

[0035] For occluded targets, the target scale level is increased and the pixels of the target's bounding rectangle are expanded by a preset percentage as the high-resolution processing area;

[0036] For targets in high noise areas, the target scale is reduced and marked as to be smoothed.

[0037] For areas where the spectral correlation is greater than a preset threshold, all targets are set to perform super-resolution mode;

[0038] A resolution mask map is generated, and the optimal processing resolution of each target area is marked in the resolution mask map.

[0039] Preferably, the method of dividing the processing modes of the environmental parameters by using the decision model includes:

[0040] A decision tree is used to map environmental parameters to different processing modes. When the occlusion rate is less than a first percentage threshold and the noise intensity is less than a noise threshold, the mode is mapped to a simple mode.

[0041] When the occlusion rate is greater than or equal to the first percentage threshold or the noise intensity is greater than or equal to the noise threshold, the mapping is a complex mode;

[0042] When the occlusion rate is greater than or equal to the second percentage threshold or the spectral correlation is greater than the correlation threshold, mapping is performed as an extreme mode;

[0043] The first percentage threshold is smaller than the second percentage threshold.

[0044] Preferably, the fused preliminary segmentation results are subjected to geographic rule verification and environment-specific repair, and the morphological kernel size is dynamically selected according to the noise intensity for opening and closing operations to obtain the final segmentation result that conforms to geographic semantics and visual continuity, including:

[0045] Delete the areas where the actual ground physical area corresponding to the connected area is smaller than the minimum physically feasible area, and correct the areas with abnormal aspect ratio;

[0046] For the missing areas covered by clouds, the distribution prior of unobstructed objects is used to complete the missing mask; misclassified pixels in high-noise areas are filtered through multi-scale voting;

[0047] The morphological kernel size is selected according to the noise intensity, and a morphological opening and closing operation is performed on the fusion mask to obtain the final segmentation result that conforms to geographic semantics and visual continuity.

[0048] Compared with the existing technology, the present invention provides a remote sensing image segmentation method based on foreground perception network, which has the following beneficial effects:

[0049] (1) This paper constructs an end-to-end intelligent segmentation framework through multi-scale feature fusion, environmental adaptive post-processing and geographic semantic constraints, which significantly improves the segmentation accuracy, robustness and practicality in complex remote sensing scenarios, and provides efficient solutions for natural resource management, disaster emergency response and other fields.

[0050] (2) The present invention can not only dynamically adjust the weights of foreground and background through the combined optimization of weighted cross entropy and Dice loss, effectively suppressing the interference of background noise on small targets, but also introduce a channel-spatial attention module at the high level of the encoder through a dual-channel attention mechanism, thereby strengthening the target spectral characteristics and suppressing background redundant information, thereby effectively improving the classification accuracy of homospectral heterogeneous objects in hyperspectral images.

[0051] (3) The present invention adopts dilated convolution or feature pyramid network to capture target features with different receptive fields. In high-resolution remote sensing images, it can effectively improve the boundary recall rate of linear targets such as roads. The dynamic multi-resolution processing strategy dynamically adjusts the processing resolution according to the target scale, so that the super-resolution of small targets is enhanced and the downsampling of large targets is accelerated. It can easily reduce the amount of calculation while preserving details and improve processing efficiency.

[0052] (4) The present invention divides the model into three modes: simple, complex, and extreme through a decision tree, so that parameters such as the morphological kernel size and the CRF smoothing strength can be dynamically adapted, thereby effectively reducing the missegmentation rate in high-noise scenarios. In addition, the present invention can eliminate unreasonable connected areas through geographic rule verification combined with the prior knowledge of ground object distribution, repair missing masks caused by cloud occlusion, and improve geographic semantic consistency.

[0053] (5) The present invention uses parallel processing and attention fusion to perform post-processing on targets of different scales in parallel, and fuses the results through the attention mechanism, effectively improving the inference speed while ensuring accuracy, and is suitable for real-time monitoring scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0055] Figure 1 The present invention is a flowchart of a remote sensing image segmentation method based on a foreground perception network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0057] According to an embodiment of the present invention, a remote sensing image segmentation method based on a foreground-aware network is provided.

[0058] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, according to an embodiment of the present invention, a remote sensing image segmentation method based on a foreground perception network is provided, comprising the following steps:

[0059] S1. Obtain the remote sensing image to be segmented and preprocess it. Based on the encoder and decoder structure, build a foreground perception network and construct a combined loss function based on weighted cross entropy loss and Dice loss to alleviate the imbalance problem between foreground and background categories.

[0060] The method of obtaining a remote sensing image to be segmented and preprocessing it, constructing a foreground perception network based on an encoder and decoder structure, and constructing a combined loss function based on weighted cross entropy loss and Dice loss to alleviate the imbalance problem between foreground and background categories includes the following steps:

[0061] S11. Obtain the remote sensing image to be segmented, normalize and crop the remote sensing image; convert the pixel-level segmentation labels into one-hot encoding, and perform morphological operations on the labels to optimize the continuity of the foreground boundary; specifically, the following steps are performed:

[0062] Image input preparation: Normalize remote sensing images (e.g., normalize pixel values to [-1, 1] or [0, 1]). Support single-band (e.g., grayscale) or multi-band (e.g., RGB, multispectral) input. Crop or resize the image to the network input requirements (e.g., 512×512 pixels), preserving the complete foreground and object context.

[0063] Label preprocessing: Convert pixel-level segmentation labels (e.g., foreground = 1, background = 0) into one-hot encoding to facilitate loss function calculation. Perform morphological operations (e.g., dilation / erosion) on the labels to optimize foreground boundary continuity.

[0064] It also includes data enhancement: applying enhancement methods such as rotation, flipping, scaling, Gaussian noise, contrast transformation, etc. to alleviate the problem of insufficient remote sensing data samples and improve the model's generalization ability.

[0065] S12. Adopt the encoder and decoder structure, embed the foreground perception module and multi-scale feature fusion mechanism, and build a foreground perception network; construct a combined loss function based on weighted cross entropy loss and Dice loss to alleviate the imbalance problem between foreground and background categories.

[0066] Specifically, the embedded foreground perception module and multi-scale feature fusion mechanism include:

[0067] A dual-channel attention mechanism consisting of channel attention and spatial attention is introduced into the high-level features of the encoder to enhance the target spectral characteristics and suppress background noise; an expanded convolution or feature pyramid network is added to the encoder to capture foreground targets of different scales.

[0068] Specifically, the foreground-aware network architecture is designed as follows:

[0069] 1) Encoder (feature extraction)

[0070] Basic feature extraction: Use a pre-trained backbone network (such as ResNet, Swin Transformer, or UNet++) to extract multi-stage feature maps from the bottom layer to the top layer (such as C1, C2, C3, C4, with decreasing resolution and gradually enhanced semantic information).

[0071] Foreground Perception Module (FPM): A dual-channel attention mechanism is introduced into the encoder's high-level features (such as C4). Channel Attention calculates the importance weight of each channel to the foreground through global average pooling and fully connected layers, enhancing the target's spectral characteristics (such as the specific band response of vegetation and buildings). Spatial Attention generates a spatial mask of the foreground area through convolution and sigmoid activation to suppress background noise (such as complex terrain and cloud interference).

[0072] Multi-scale feature fusion: Add atrous convolution or feature pyramid network (FPN) to the encoder to capture foreground objects of different scales (such as small vehicles and large buildings).

[0073] 2) Decoder (feature recovery and segmentation)

[0074] Upsampling and feature fusion: Upsampling layer by layer through transposed convolution or interpolation connects the encoder's high-level semantic features with low-level detail features (such as C1 and C2) across layers to restore spatial resolution. A foreground enhancement layer is inserted after each upsampling: Lightweight convolution blocks are used to further strengthen foreground features and suppress background interference.

[0075] Output layer: The last layer uses 1×1 convolution to map the feature map into a segmentation probability map. The number of channels is equal to the number of categories (for example, the number of channels is 2 for foreground / background binary classification), and softmax activation is used to generate pixel-level classification results.

[0076] To address the foreground-background category imbalance problem in remote sensing images, a combined loss function is used:

[0077] L=λ·L CE +(1-λ)·L Dice

[0078]

[0079] Where L represents the combined loss function, L CE represents the weighted cross entropy loss, L Dice represents the Dice loss, λ represents the balance coefficient (taken as 0.7 in this embodiment), w c represents the weight of category c (foreground weight > background weight), C represents the total number of categories, N represents the total number of pixels in the image, and y i,c Indicates the true label of pixel i belonging to category c, y i represents the true binary label of pixel i, p i,c represents the probability that the model predicts that pixel i belongs to category c, p irepresents the probability that the model predicts that pixel i is the foreground, and ε represents the smoothing factor (in this embodiment, it is 10 -5 ).

[0080] S2. Dynamically adjust the learning rate based on the optimizer and use the remote sensing dataset to train the foreground perception network; use the trained foreground perception network to segment the preprocessed remote sensing image to obtain the category probability map of each pixel;

[0081] The method of dynamically adjusting the learning rate based on the optimizer, training the network foreground perception network using the remote sensing data set, and segmenting the pre-processed remote sensing image to be segmented using the trained foreground perception network to obtain the category probability map of each pixel includes the following steps:

[0082] S21, use Adam or SGD optimizer, initialize the learning rate to 1e -4 , and the cosine annealing algorithm is used to adjust the learning rate. Based on the adjusted learning rate, the foreground perception network is trained in combination with the remote sensing dataset;

[0083] Specifically, in this embodiment, the Potsdam and Vaihingen datasets from the official website of the International Society for Photogrammetry and Remote Sensing (ISPRS) are selected as remote sensing datasets;

[0084] The Potsdam dataset has an image spatial resolution of 5 cm. It contains three-band remote sensing TIFF files (TagImage File Format). Each image also includes an affine transformation file to facilitate re-fragmentation of the image into smaller images when needed. The dataset also provides TIFF storage formats for different channel combinations, including IR-RG, RGB, and RGB-IR.

[0085] The Vaihingen dataset has an image spatial resolution of 9 cm and contains 33 real orthophotos with an average spatial size of 2494 × 2064. The ground truth contains the same seven categories as the ISPRS Potsdam benchmark.

[0086] In this example, we selected images with boundary labels as label images, including seven categories: impervious surfaces, buildings, low vegetation, trees, cars, debris, and background. These images were used as training, test, and validation sets, with a step size of 512 and no overlapping areas cropped.

[0087] S22. Input the pre-processed remote sensing image to be segmented into a trained foreground perception network, and use the foreground perception network to output a category probability map of each pixel.

[0088] S3. Based on dynamic multi-resolution attention, the category probability map is multi-scaled in combination with environmental features, the post-processing strategy is dynamically adapted, and post-processing is performed according to the adapted post-processing strategy to optimize the segmentation results.

[0089] The method of performing multi-scale classification of the category probability map based on dynamic multi-resolution attention and combining environmental features, dynamically adapting the post-processing strategy, and performing post-processing according to the adapted post-processing strategy to optimize the segmentation result includes the following steps:

[0090] S31, obtaining a class probability map and an original remote sensing image, and extracting environmental parameters of the original remote sensing image, wherein the environmental parameters include occlusion characteristics, noise characteristics, resolution characteristics, and spectral characteristics;

[0091] Specifically, the extraction of the occlusion feature includes: detecting cloud or shadow areas based on HSV or Lab color space and calculating the occlusion rate to determine;

[0092] The extraction of the noise feature includes: evaluating the noise intensity by local pixel gradient variance or median absolute deviation, and marking high noise areas for determination;

[0093] Extraction of the resolution feature includes: for UAV images, determining the actual ground resolution by combining the flight altitude or focal length; for satellite images, determining the spatial resolution by extracting the sensor metadata;

[0094] The extraction of the spectral features includes: determining by calculating the correlation between multi-spectral bands (such as Pearson coefficient) and identifying the risk area of the same spectrum foreign body (band combination with correlation>0.9).

[0095] S32, extracting the connected regions of the foreground objects by binarizing the class probability map, performing basic scale classification based on the area of the connected regions, and recording the geometric features of each connected region; correcting the basic scale classification based on environmental interference, and generating a mask map that marks the optimal processing resolution of each region;

[0096] Specifically, performing basic scale classification according to the area of the connected regions and recording the geometric features of each connected region; correcting the basic scale classification according to environmental interference and generating a mask map marking the optimal processing resolution of each region include:

[0097] Divide the foreground object connected region into the first, second, and third targets based on their area, and record the geometric features (center of mass, aspect ratio, perimeter) of each foreground object connected region. The pixel size of the first target (<100 pixels), second target (100-1000 pixels), and third target (≥1000 pixels) increases in order.

[0098] For occluded targets (overlapping with the occluded area > 30%), increase the target scale level, i.e., scale level +1 (e.g., first target → second target), and expand the target bounding rectangle by a preset percentage (20%) of pixels as the high-resolution processing area;

[0099] For targets in high-noise areas, reduce the target scale level, i.e., scale level -1 (e.g., second target → first target), and mark them as to be strongly smoothed.

[0100] For regions where the spectral correlation is greater than a preset threshold, all targets are set to perform super-resolution mode (processing resolution = original resolution × 2);

[0101] A resolution mask map is generated, and the optimal processing resolution of each target area is marked in the resolution mask map.

[0102] S33. Use the decision model to divide the environmental parameters into processing modes and dynamically adjust the post-processing parameters according to different processing modes; process targets of different scales in parallel according to the resolution mask map and use the attention mechanism to fuse the processing results;

[0103] Specifically, the use of the decision model to divide the processing modes of the environmental parameters includes:

[0104] A decision tree is used to map environmental parameters to different processing modes. When the occlusion rate is less than the first percentage threshold and the noise intensity is less than the noise threshold (i.e., the occlusion rate is less than 5% and the noise intensity is less than the threshold), it is mapped to a simple mode and basic post-processing (fixed threshold + 3×3 morphological kernel) is performed.

[0105] When the occlusion rate is greater than or equal to the first percentage threshold or the noise intensity is greater than or equal to the noise threshold (i.e., the occlusion rate ≥ 5% or the noise intensity ≥ the threshold), it is mapped to the complex mode and the enhancement module (dynamic threshold + adaptive CRF) is enabled;

[0106] When the occlusion rate is greater than or equal to the second percentage threshold or the spectral correlation is greater than the correlation threshold (i.e., occlusion rate ≥ 30% or spectral correlation > 0.95), it is mapped to the extreme mode and context completion + multi-scale fusion is activated;

[0107] The higher the noise, the larger the threshold segmentation window, the stronger the CRF spatial smoothing, and the larger the morphological kernel size; the more severe the occlusion, the more dependent on geographical priors (such as the distribution pattern of land objects) to fill in the missing areas; the stronger the spectral heterogeneity, the higher the band weight in the CRF, and the need to combine multispectral information to correct the segmentation; small targets require super-resolution to enhance details, large targets can be down-sampled to improve efficiency, and dynamic mixing processing is required in complex modes.

[0108] Dynamic adjustment of post-processing algorithm parameters:

[0109] Threshold segmentation: Adjust the local window size of the Otsu algorithm according to the noise intensity (the higher the noise, the larger the window, such as using a 15×15 window when the noise intensity is high and a 5×5 window when the noise intensity is low);

[0110] Morphological operation: configure the convolution kernel according to the target scale and environmental noise (3×3 erosion kernel for small targets and 7×7 smoothing kernel for large targets with high noise);

[0111] Conditional Random Field (CRF):

[0112] Spectrally complex regions: increase the spectral similarity weight (such as using multispectral band values as CRF appearance features);

[0113] Occlusion area: Introduce geographic prior weights (such as "no buildings should appear within 50 pixels around a water body").

[0114] Multi-resolution parallel processing and cross-scale result fusion:

[0115] Sub-resolution processing branch:

[0116] Small target branch: super-resolution reconstruction of the target area (such as bicubic interpolation to 2 times the resolution); application of edge-preserving filtering (such as guided filtering) to enhance details; fine-grained CRF optimization (iteration number = 5, spatial weight = 0.8).

[0117] Large target branch: downsample to 1 / 2 resolution to reduce computational effort; connect broken parts using region growing algorithm (similarity threshold = 0.7); and remove small noise using Gaussian smoothing (kernel size = 5 × 5).

[0118] Cross-scale fusion: Merge results of different resolutions through the attention mechanism: the weight of small target areas = classification entropy (the lower the entropy, the higher the weight, that is, the area with high confidence dominates the fusion); the weight of large target areas = geometric integrity (the closer the perimeter-area ratio is to a circle, the higher the weight); generate a preliminary fusion mask.

[0119] S34. Perform geographic rule verification and environment-specific repair on the fused preliminary segmentation results, and dynamically select the morphological kernel size for opening and closing operations based on the noise intensity to obtain the final segmentation result that conforms to geographic semantics and visual continuity.

[0120] Specifically, the fused preliminary segmentation results are subjected to geographic rule verification and environment-specific repair, and the morphological kernel size is dynamically selected according to the noise intensity for opening and closing operations to obtain the final segmentation results that conform to geographic semantics and visual continuity, including:

[0121] Delete the area where the actual ground physical area corresponding to the connected area is smaller than the minimum physically feasible area (such as buildings less than 50m 2, converted to pixel threshold according to resolution), and correct areas with abnormal aspect ratio (e.g., the aspect ratio of roads should be greater than 10, otherwise it will be regarded as noise);

[0122] For the missing areas covered by clouds, the distribution prior of unobstructed objects is used to complete the missing mask; misclassified pixels in high-noise areas are filtered through multi-scale voting;

[0123] The morphological kernel size is selected according to the noise intensity (kernel size = environmental noise correlation value), and a morphological opening and closing operation is performed on the fusion mask to obtain the final segmentation result that conforms to geographic semantics and visual continuity.

[0124] In summary, with the help of the above-mentioned technical solutions of the present invention, an end-to-end intelligent segmentation framework is constructed through multi-scale feature fusion, environmental adaptive post-processing and geographic semantic constraints, which significantly improves the segmentation accuracy, robustness and practicality in complex remote sensing scenarios, and provides efficient solutions for natural resource management, disaster emergency response and other fields.

[0125] In addition, the present invention can not only dynamically adjust the weights of foreground and background through the combined optimization of weighted cross entropy and Dice loss to effectively suppress the interference of background noise on small targets, but also introduce a channel-spatial attention module at the high level of the encoder through a dual-channel attention mechanism to enhance the target spectral characteristics and suppress background redundant information, thereby effectively improving the classification accuracy of homospectral heterogeneous objects in hyperspectral images.

[0126] In addition, the present invention uses dilated convolution or feature pyramid networks to capture target features in different receptive fields. In high-resolution remote sensing images, it can effectively improve the boundary recall rate of linear targets such as roads. The dynamic multi-resolution processing strategy dynamically adjusts the processing resolution according to the target scale, thereby enhancing the super-resolution of small targets and accelerating the downsampling of large targets. It can easily reduce the amount of calculation while preserving details and improve processing efficiency.

[0127] In addition, the present invention divides the model into three modes: simple, complex, and extreme through a decision tree, so that parameters such as the morphological kernel size and the CRF smoothing strength can be dynamically adapted, thereby effectively reducing the missegmentation rate in high-noise scenarios. It can also eliminate unreasonable connected areas through geographic rule verification combined with the prior knowledge of ground object distribution, repair missing masks caused by cloud occlusion, and improve geographic semantic consistency.

[0128] In addition, the present invention uses parallel processing and attention fusion to perform post-processing on targets of different scales in parallel, and fuses the results through the attention mechanism, effectively improving the inference speed while ensuring accuracy, and is suitable for real-time monitoring scenarios.

[0129] The technical features of the above-described embodiments can be combined in any combination. To simplify the description, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. A person of ordinary skill in the art will understand that all or part of the steps in the method of the above-described embodiment can be completed by instructing the relevant hardware through a program. The program can be stored in a computer-readable storage medium. When the program is executed, it includes the steps described in the above method. The storage medium is, for example, ROM / RAM, a magnetic disk, an optical disk, etc.

[0130] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A remote sensing image segmentation method based on foreground perception network, characterized in that: The following steps are involved: S1. Obtain the remote sensing image to be segmented and preprocess it. Based on the encoder and decoder structure, build a foreground perception network and construct a combined loss function based on weighted cross entropy loss and Dice loss to alleviate the imbalance problem between foreground and background categories. S2. Dynamically adjust the learning rate based on the optimizer and use the remote sensing dataset to train the foreground perception network; use the trained foreground perception network to segment the preprocessed remote sensing image to obtain the category probability map of each pixel; S3. Based on dynamic multi-resolution attention, the category probability map is multi-scale graded in combination with environmental features, the post-processing strategy is dynamically adapted, and post-processing is performed according to the adapted post-processing strategy to optimize the segmentation results.

2. The remote sensing image segmentation method based on foreground perception network according to claim 1, characterized in that: The method includes the following steps: obtaining a remote sensing image to be segmented and preprocessing it, constructing a foreground perception network based on an encoder and decoder structure, and constructing a combined loss function based on weighted cross entropy loss and Dice loss to alleviate the imbalance problem between foreground and background categories: S11. Obtain a remote sensing image to be segmented, and perform normalization and cropping on the remote sensing image; convert pixel-level segmentation labels into one-hot encoding, and perform morphological operations on the labels to optimize the continuity of the foreground boundary; S12. Adopt the encoder and decoder structure, embed the foreground perception module and multi-scale feature fusion mechanism, and build a foreground perception network; construct a combined loss function based on weighted cross entropy loss and Dice loss to alleviate the imbalance problem between foreground and background categories.

3. The remote sensing image segmentation method based on foreground perception network according to claim 2 is characterized in that: The embedded foreground perception module and multi-scale feature fusion mechanism include: A dual-channel attention mechanism consisting of channel attention and spatial attention is introduced into the high-level features of the encoder to enhance the target spectral characteristics and suppress background noise; an expanded convolution or feature pyramid network is added to the encoder to capture foreground targets of different scales.

4. The remote sensing image segmentation method based on foreground perception network according to claim 1, characterized in that: The method of dynamically adjusting the learning rate based on the optimizer and training the network foreground perception network using the remote sensing data set; segmenting the pre-processed remote sensing image to be segmented using the trained foreground perception network to obtain the category probability map of each pixel includes the following steps: S21. Initialize the learning rate using the optimizer and adjust the learning rate using the cosine annealing algorithm. Train the foreground perception network based on the adjusted learning rate and the remote sensing dataset. S22. Input the pre-processed remote sensing image to be segmented into a trained foreground perception network, and use the foreground perception network to output a category probability map of each pixel.

5. The remote sensing image segmentation method based on foreground perception network according to claim 1, characterized in that: The method of performing multi-scale classification of the category probability map based on dynamic multi-resolution attention and combining environmental features, dynamically adapting the post-processing strategy, and performing post-processing according to the adapted post-processing strategy to optimize the segmentation result includes the following steps: S31, obtaining a class probability map and an original remote sensing image, and extracting environmental parameters of the original remote sensing image, wherein the environmental parameters include occlusion characteristics, noise characteristics, resolution characteristics, and spectral characteristics; S32, extracting the connected regions of the foreground objects by binarizing the class probability map, performing basic scale classification based on the area of the connected regions, and recording the geometric features of each connected region; correcting the basic scale classification based on environmental interference, and generating a mask map that marks the optimal processing resolution of each region; S33. Use the decision model to divide the environmental parameters into processing modes and dynamically adjust the post-processing parameters according to different processing modes; process targets of different scales in parallel according to the resolution mask map and use the attention mechanism to fuse the processing results; S34. Perform geographic rule verification and environment-specific repair on the fused preliminary segmentation results, and dynamically select the morphological kernel size for opening and closing operations based on the noise intensity to obtain the final segmentation result that conforms to geographic semantics and visual continuity.

6. The remote sensing image segmentation method based on foreground perception network according to claim 5, characterized in that: The extraction of the occlusion feature includes: detecting cloud or shadow areas based on HSV or Lab color space and calculating the occlusion rate to determine; The extraction of the noise feature includes: evaluating the noise intensity by local pixel gradient variance or median absolute deviation, and marking high noise areas for determination; Extraction of the resolution feature includes: for UAV images, determining the actual ground resolution by combining the flight altitude or focal length; for satellite images, determining the spatial resolution by extracting the sensor metadata; The extraction of the spectral features includes: determining by calculating the correlation between multi-spectral bands and identifying the risk area of heteromorphic objects in the same spectrum.

7. The remote sensing image segmentation method based on foreground perception network according to claim 5, characterized in that: The basic scale classification is performed according to the area of the connected regions, and the geometric features of each connected region are recorded; Correcting the basic scale classification based on environmental interference and generating a mask map that marks the optimal processing resolution for each area includes: Divide the foreground object connected region into the first target, the second target, and the third target according to the area of the connected region, and record the geometric features of each foreground object connected region; wherein the pixels of the first target, the second target, and the third target increase in sequence; For occluded targets, the target scale level is increased and the pixels of the target's bounding rectangle are expanded by a preset percentage as the high-resolution processing area; For targets in high noise areas, the target scale is reduced and marked as to be smoothed. For areas where the spectral correlation is greater than a preset threshold, all targets are set to perform super-resolution mode; A resolution mask map is generated, and the optimal processing resolution of each target area is marked in the resolution mask map.

8. The remote sensing image segmentation method based on foreground perception network according to claim 5, characterized in that: The method of dividing the processing modes of the environmental parameters by using the decision model includes: A decision tree is used to map environmental parameters to different processing modes. When the occlusion rate is less than a first percentage threshold and the noise intensity is less than a noise threshold, the mode is mapped to a simple mode. When the occlusion rate is greater than or equal to the first percentage threshold or the noise intensity is greater than or equal to the noise threshold, the mapping is a complex mode; When the occlusion rate is greater than or equal to the second percentage threshold or the spectral correlation is greater than the correlation threshold, the mapping is to the extreme mode.

9. The remote sensing image segmentation method based on foreground perception network according to claim 8, characterized in that: The first percentage threshold is less than the second percentage threshold.

10. The remote sensing image segmentation method based on foreground perception network according to claim 5, characterized in that: The fused preliminary segmentation results are subjected to geographic rule verification and environment-specific repair, and the morphological kernel size is dynamically selected according to the noise intensity for opening and closing operations to obtain the final segmentation results that conform to geographic semantics and visual continuity. Delete the areas where the actual ground physical area corresponding to the connected area is smaller than the minimum physically feasible area, and correct the areas with abnormal aspect ratio; For the missing areas covered by clouds, the distribution prior of unobstructed objects is used to complete the missing mask; misclassified pixels in high-noise areas are filtered through multi-scale voting; The morphological kernel size is selected according to the noise intensity, and a morphological opening and closing operation is performed on the fusion mask to obtain the final segmentation result that conforms to geographic semantics and visual continuity.

Images