A method and system for water body contour extraction of high orbit satellite imagery

Abstract

The application provides a water body contour extraction method and system for high-orbit satellite images, comprising the following steps: performing sliding window cutting on an original image to generate an image block sequence; inputting the image block sequence into a heterogeneous double-flow encoder to extract double-branch heterogeneous features; realizing deep interaction on the double-branch heterogeneous features in the heterogeneous double-flow encoder to obtain deep heterogeneous collaborative features and shallow heterogeneous collaborative features in multiple feature extraction stages; processing the deep heterogeneous collaborative features and the shallow heterogeneous collaborative features to obtain a probability prediction map; recombining the probability prediction maps of all image blocks by adopting a square table shape weighted fusion strategy to obtain a full-image probability prediction map; and performing binary division on the probability prediction map to obtain a final binary mask map. The application solves the problems that inherent noise in high-orbit SAR images makes the texture inside the water body uneven, small river networks are difficult to maintain the integrity of their topological structure, and gaps and misplacement marks are left during recombination and splicing.

Description

Technical Field

[0001] This invention relates to the field of high-orbit SAR image processing technology, specifically to a method and system for extracting water body contours from high-orbit satellite images. Background Technology

[0002] With the rapid development of aerospace remote sensing technology, high-orbit synthetic aperture radar (SAR) satellites, with their all-weather detection and watershed-level ultra-wide coverage capabilities, have become an important technical means for dynamic water resource management, ecological monitoring, and flood disaster monitoring. High-orbit SAR imagery can cover an area of ​​hundreds of square kilometers, making it significant for large-scale macroscopic surface monitoring tasks.

[0003] However, high-orbit SAR imagery also faces many problems. For example, SAR satellite imagery is usually subject to a lot of noise interference. The shadow effect caused by the tilt imaging mechanism is very similar to the texture of water bodies, which can lead to misjudgment by the detection model. Traditional water body extraction methods based on threshold segmentation or edge detection are difficult to achieve high-precision automated extraction of large water bodies, which seriously affects the reliability of subsequent task analysis.

[0004] In recent years, deep learning-based semantic segmentation methods (such as U-Net and DeepLab) have made significant progress in extracting small-scale optical or low-orbit images. However, when directly applied to large-scale high-orbit SAR satellite images, the following technical bottlenecks exist:

[0005] 1. The inherent noise in large-scale high-orbit SAR images makes the internal texture of water bodies uneven. Existing convolutional neural networks are limited by local receptive fields and have difficulty establishing long-distance dependencies in the context of strong noise, resulting in discontinuous extraction results for large water bodies. The traditional Transformer method has a global field of view, but the computational complexity increases exponentially with the image, making it difficult to process large-scale high-orbit SAR images.

[0006] 2. When processing high-orbit imagery, models typically need to perform multiple downsampling operations to obtain sufficient semantic context information. This deep network stacking leads to the irreversible loss of high-resolution spatial details such as small river branches and irregular water body edges contained in the shallow network in the deep network, ultimately making it difficult for the small river network to maintain the integrity of its topological structure.

[0007] 3. Due to limitations in hardware memory, large-scale high-orbit SAR images must be segmented for prediction using a sliding window strategy in practical processing. However, conventional segmentation mechanisms result in a lack of global contextual references in the edge regions of image blocks, reducing the confidence of edge predictions and leaving obvious stitching gaps and misalignments when the final predicted images are reassembled and stitched together. Summary of the Invention

[0008] The purpose of this invention is to overcome the above-mentioned technical deficiencies and provide a method and system for extracting water body contours from high-orbit satellite images. This solves the technical problems in the prior art where the inherent noise in high-orbit SAR images causes uneven textures inside water bodies, makes it difficult for small river networks to maintain the integrity of their topological structure, and leaves obvious splicing gaps and misalignment marks during reconstruction and splicing.

[0009] To achieve the above-mentioned technical objectives, in a first aspect, the present invention provides a method for extracting water body contours from high-orbit satellite imagery, comprising the following steps:

[0010] The original image of the high-orbit SAR image is acquired, and the original image is divided into a sliding window to generate an image patch sequence;

[0011] A heterogeneous dual-stream encoder for high-orbit SAR images is constructed. The heterogeneous dual-stream encoder includes a water body local spatial prior flow and a water body global sequence modeling flow. The image patch sequence is input into the heterogeneous dual-stream encoder to extract dual-branch heterogeneous features.

[0012] Deep interaction is achieved on the dual-branch heterogeneous features in the heterogeneous dual-stream encoder to obtain deep heterogeneous collaborative features and shallow heterogeneous collaborative features in multiple feature extraction stages;

[0013] The deep heterogeneous collaborative features are processed by hollow spatial pyramid pooling, and the shallow heterogeneous collaborative features are processed by dimensionality reduction. The processed deep features are upsampled and fused with the dimensionality-reduced shallow features. After upsampling and activation function processing, a pixel-level probability prediction map matching the size of the input image block is output.

[0014] Based on the cutting size and step size of the image patch sequence, a square-shaped weighted fusion strategy is used to reassemble the probability prediction maps of all the image patches to generate a full-map probability prediction map with the same size as the original high-orbit SAR image. A threshold is set to perform binary partitioning on the probability prediction map to obtain the final binary mask map, thus completing the end-to-end large-body water extraction task.

[0015] Compared with the prior art, the beneficial effects of the present invention include:

[0016] The method for processing and extracting large rivers and lakes from high-orbit SAR images proposed in this invention has high extraction accuracy, fast extraction speed, and is easy to implement. It can be applied to the task of extracting large water bodies from high-orbit SAR images.

[0017] According to some embodiments of the present invention, inputting the image patch sequence into the heterogeneous dual-stream encoder to extract dual-branch heterogeneous features includes the following steps:

[0018] The local spatial prior flow of the water body uses convolutional layer stacking to capture the geometric boundary and texture information of the water body, and the global sequence modeling flow of the water body uses a multi-path scanning mechanism to transform the two-dimensional image into a one-dimensional sequence and establish the long-distance pixel dependency of the entire image range.

[0019] According to some embodiments of the present invention, the prior flow in the local space of the water body adopts a convolutional neural network architecture, with an initial convolutional pooling layer at the input, followed by multiple convolutional feature extraction stages connected sequentially, and local features F are extracted using convolutional kernels. cnn ;

[0020] The global sequence modeling stream for water bodies incorporates an image patch serialization embedding layer at the input end. It employs a state-space model for long sequence modeling. Within each Mamba block, a multi-path scanning strategy is used to scan and update features along the forward, reverse, and randomly shuffled directions of the one-dimensional sequence to extract long-distance dependent features F. mamba .

[0021] According to some embodiments of the present invention, deep interaction is achieved on the dual-branch heterogeneous features in the heterogeneous dual-stream encoder to obtain deep heterogeneous collaborative features and shallow heterogeneous collaborative features in multiple feature extraction stages, including the following steps:

[0022] The dual-branch heterogeneous features are input into the cross-view feature collaboration module for deep interactive processing:

[0023] For the i-th stage, the cross-view feature collaboration module receives three feature inputs, namely the i-th stage output feature F from the prior flow of the local spatial flow of the water body. cnn The output feature F from the i-th layer of the global sequence modeling flow of the water body mamba and the output features F from all preceding collaborative modules fusion The first stage does not require the output features of the preceding collaborative module;

[0024] Output feature F cnn、 Output feature F mamba and output features F fusion Alignment is performed in the spatial dimension, followed by concatenation in the channel dimension to obtain the hybrid feature F. cat ;

[0025] Mixed features F cat The data is fed into a multi-scale refinement layer, which contains parallel first and second convolutional branches. The first convolutional branch uses standard convolution to extract partial details, while the second convolutional branch uses dilated convolution to capture contextual information. The outputs of the two branches are then element-wise summed to generate an enhanced feature F. fedab ;

[0026] Enhance feature Ffedab Semantic correction is performed sequentially through a dual attention correction layer to generate dual-calibrated features;

[0027] The dual calibration features are convolved with the hybrid feature F after a 1×1 convolution. cat Perform residual connections and output the heterogeneous collaborative features of this stage.

[0028] According to some embodiments of the present invention, the dual attention correction layer performs semantic correction, including the following steps:

[0029] The dual attention correction layer comprises two parts: a channel attention correction module and a spatial attention correction module. The channel attention correction module performs global average pooling on the input feature map to obtain channel descriptors.

[0030] The channel descriptors are nonlinearly interacted using a multilayer perceptron, and a channel weight vector is generated by sigmoid activation. This vector is then multiplied element-wise with the input feature map to obtain F. ca And output it to the spatial attention correction module;

[0031] The spatial attention correction module receives the feature F after channel calibration. ca Channel compression is performed through a 1×1 convolutional layer, followed by sigmoid activation to generate a spatial weight mask, which is then compared with the feature map F. ca Pixel-by-pixel multiplication generates dual-calibration feature F * ca .

[0032] According to some embodiments of the present invention, outputting a pixel-level probability prediction map that matches the size of an input image patch includes the following steps:

[0033] The deep heterogeneous collaborative features output from the final heterogeneous stage of the cross-perspective feature collaboration module are input into the hollow space pyramid pooling module to capture multi-scale contextual information.

[0034] The shallow heterogeneous collaborative features output from the first stage of the cross-view feature collaboration module are dimensionality reduced by a convolutional layer to obtain the dimensionality-reduced shallow features.

[0035] After upsampling the deep features processed by the hollow space pyramid pooling module, they are concatenated with the dimensionality-reduced shallow features in the channel dimension to obtain the concatenated features.

[0036] The concatenated features are fused through a series of convolutional layers and mapped to a prediction probability map.

[0037] According to some embodiments of the present invention, based on the cutting size and step size of the image patch sequence, a square-shaped weighted fusion strategy is used to reassemble the probability prediction maps of all the image patches to generate a full-map probability prediction map with the same size as the original high-orbit SAR image, including the following steps:

[0038] The image size of the sliding window cropping is H×H, the step size of the sliding window is S, the central region of the weight matrix is ​​a rectangular range of (H-2S)×(H-2S), the weight value is always 1.0, the edge region of the weight matrix is ​​a range of width S surrounding the central region, and the weight value linearly decreases from 1.0 to 0 from the inside out.

[0039] During the reconstruction, a full-image probability accumulation matrix and a full-image weight accumulation matrix are established separately. The predicted image of each image block is multiplied pixel by pixel with the weight matrix and then accumulated into the full-image probability accumulation matrix. The weight matrix is ​​accumulated into the full-image weight accumulation matrix. Finally, the full-image probability accumulation matrix is ​​divided by the full-image weight accumulation matrix to obtain the final full-image probability prediction result.

[0040] Secondly, the present invention provides a water body contour extraction system for high-orbit satellite imagery, comprising:

[0041] The preprocessing module acquires the original image of the high-orbit SAR image, performs sliding window segmentation on the original image, and generates an image block sequence.

[0042] A heterogeneous dual-stream encoder for high-orbit SAR images, comprising: a local spatial prior flow of water and a global sequence modeling flow of water, wherein the image patch sequence is input into the heterogeneous dual-stream encoder to extract dual-branch heterogeneous features;

[0043] A cross-perspective feature collaboration module is connected to the local spatial prior flow of the water body and the global sequence modeling flow of the water body, respectively, and is used to achieve deep interaction of the dual-branch heterogeneous features in the heterogeneous dual-stream encoder to obtain deep heterogeneous collaborative features and shallow heterogeneous collaborative features in multiple feature extraction stages.

[0044] The water mask generation module performs hollow spatial pyramid pooling processing on the deep heterogeneous collaborative features and dimensionality reduction processing on the shallow heterogeneous collaborative features. After upsampling the processed deep features, it merges them with the dimensionality-reduced shallow features. Then, after upsampling and activation function processing, it outputs a pixel-level probability prediction map that matches the size of the input image block.

[0045] The binary mask generation module reassembles the probability prediction maps of all image blocks according to the cutting size and step size of the image block sequence using a square-shaped weighted fusion strategy, generating a full-map probability prediction map with the same size as the original high-orbit SAR image. A threshold is set to perform binary partitioning on the probability prediction map to obtain the final binary mask map, thus completing the end-to-end large-body water extraction task.

[0046] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0047] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein the abstract drawings are to be completely consistent with one of the drawings in the specification:

[0048] Figure 1 This is a step diagram of a method for extracting water body contours from high-orbit satellite imagery according to an embodiment of the present invention;

[0049] Figure 2 This is a structural diagram of a large water body extraction model provided in a high-orbit SAR image according to an embodiment of the present invention;

[0050] Figure 3 This is a structural diagram of a cross-view feature collaboration module provided in one embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of image block reconstruction provided in one embodiment of the present invention;

[0052] Figure 5 This is a diagram illustrating the extraction effect obtained using various processing and extraction methods, as provided in one embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0054] It should be noted that although functional modules are divided in the system diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than the module division in the system or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0055] Reference Figures 1 to 5 , Figure 1This is a step diagram of a method for extracting water body contours from high-orbit satellite imagery according to an embodiment of the present invention; Figure 2 This is a structural diagram of a large water body extraction model provided in a high-orbit SAR image according to an embodiment of the present invention; Figure 3 This is a structural diagram of a cross-view feature collaboration module provided in one embodiment of the present invention; Figure 4 This is a schematic diagram of image block reconstruction provided in one embodiment of the present invention; Figure 5 This is a diagram illustrating the extraction effect obtained using various processing and extraction methods, as provided in one embodiment of the present invention.

[0056] In one embodiment, the water body contour extraction method for high-orbit satellite imagery includes the following steps: acquiring the original image of the high-orbit SAR imagery; performing sliding window segmentation on the original imagery to generate an image patch sequence; constructing a heterogeneous dual-stream encoder for the high-orbit SAR imagery, the heterogeneous dual-stream encoder comprising: a water body local spatial prior flow and a water body global sequence modeling flow; inputting the image patch sequence into the heterogeneous dual-stream encoder to extract dual-branch heterogeneous features; performing deep interaction on the dual-branch heterogeneous features in the heterogeneous dual-stream encoder to obtain deep heterogeneous collaborative features and shallow heterogeneous collaborative features at multiple feature extraction stages; and sampling the deep heterogeneous collaborative features. Hollow spatial pyramid pooling is used to reduce the dimensionality of shallow heterogeneous collaborative features. The processed deep features are then upsampled and fused with the dimensionality-reduced shallow features. After further upsampling and activation function processing, a pixel-level probability prediction map matching the size of the input image patch is output. Based on the cutting size and stride of the image patch sequence, a square-shaped weighted fusion strategy is used to reassemble the probability prediction maps of all image patches to generate a full-map probability prediction map with the same size as the original high-orbit SAR image. A threshold is set to perform binary partitioning of the probability prediction map to obtain the final binary mask map, thus completing the end-to-end large-body water extraction task.

[0057] The water contour extraction method for high-orbit satellite imagery provided by this invention has the following advantages:

[0058] 1. In high-orbit SAR imagery, the interior texture of water bodies often appears blurred due to speckle noise, easily causing traditional models to identify complete water bodies as incomplete blocky fragments. The heterogeneous dual-stream encoder provided in this invention utilizes the linear long-sequence modeling capability of the state-space model to capture dependencies between pixels over a large area, while simultaneously leveraging the sensitivity of convolutional branches to local information to capture edge information. The branching based on two perspectives enables coherent extraction of large water bodies.

[0059] 2. To address the issue of small river targets in high-orbit SAR imagery being easily lost during deep network extraction, this invention provides a cross-view feature collaboration module based on a heterogeneous dual-stream encoder for high-orbit SAR imagery. This module constructs a cross-level feature backtracking and reuse mechanism, which can directly transfer the high-resolution spatial information retained by the shallow network to the deep semantic network. This can effectively compensate for the loss of small target information caused by the deep network, thereby improving the model's ability to extract small rivers.

[0060] 3. To address the differences in feature representation space between the convolutional neural network in the local spatial prior stream and the visual state space model in the global sequence modeling stream in heterogeneous dual-stream encoders of high-orbit SAR images, the cross-view feature collaboration module of this invention achieves deep fusion of heterogeneous features through multi-scale interaction and dual attention correction mechanisms.

[0061] The aforementioned SAR imagery of large water bodies employs a deep convolutional neural network architecture based on stacked convolutional layers. This branch utilizes the local receptive field of the convolutional kernels to capture the geometric boundaries and texture features of the water body, including the following processing steps:

[0062] A1. Input the input image into the initial convolutional pooling layer of the network. For the single-band high-orbit SAR image used in this invention, the input channel of the initial convolutional pooling layer is adapted to 1, and the input image is subjected to preliminary feature mapping and dimensionality reduction processing.

[0063] A2. The image goes through four progressive feature extraction stages (stage 1-stage 4) in sequence. Each stage consists of multiple stacked convolutional feature extraction blocks. Each convolutional feature extraction block contains multiple cascaded convolutional layers, batch normalization (BN) layers, and nonlinear activation functions (Rectified Linear Unit, ReLU) to extract the geometric texture features of the water body at different resolutions step by step.

[0064] The SAR imagery global sequence modeling workflow for large water bodies adopts a state-space model-based architecture. This branch utilizes the linear, complex, and long sequence modeling capabilities of the state-space model to transform two-dimensional images into one-dimensional sequences for processing. This is used to establish long-distance dependencies between pixel blocks across the entire image area for feature extraction of large water bodies, and includes the following processing steps:

[0065] B1. Divide the input image into non-overlapping image blocks, flatten the image blocks, and then project them linearly into feature vectors to form a one-dimensional label sequence;

[0066] B2. The one-dimensional label sequence passes through the four stages of the state-space network (stage 1-stage 4) in sequence, and each stage consists of multiple cascaded Mamba blocks;

[0067] B3. Within each Mamba block, a multi-path scanning mechanism is employed. For the input one-dimensional labeled sequence, scanning and modeling are performed simultaneously in the following three directions:

[0068] B3.1, Direction 1: Process sequentially along a one-dimensional sequence to capture regular dependencies;

[0069] B3.2, Direction 2: Process in reverse order along the one-dimensional sequence to supplement the reverse correlation information that is complementary to the sequential processing;

[0070] B3.3, Direction 3: The one-dimensional sequence is randomly shuffled and then scanned to supplement the global correlation in the image that has no fixed order. At the same time, this scanning method can help the model identify irregular water bodies.

[0071] B4. In each scan path, the sequence is updated using the discretized state-space equation. The core formula is as follows:

[0072]

[0073] Formula 1

[0074] Where t represents a one-dimensional position index in the input sequence. The input feature is the pixel position at the current time. This means that before processing the t-th position, the model has accumulated historical global information. Represents global information after the current position is updated. Output features. Matrix , It is dynamically generated through an input-dependent selective mechanism, which allows the model to adaptively reset or save its state based on the image content.

[0075] B4. Gated fusion of the sequence features output from multiple scanning channels, and reverse reshaping into a two-dimensional feature map.

[0076] Among them, the cross-view feature collaboration module based on the heterogeneous dual-stream encoder of high-orbit SAR imagery is used to perform deep interaction on the cross-view heterogeneous features output by each level (stage 1-stage 4) of the heterogeneous dual-stream encoder. The specific steps are as follows:

[0077] C1. For the i-th feature level of the network (where i = 1, 2, 3, 4), this module first receives feature inputs from three different branches, concatenates the features from the three branches along the channel dimension, and obtains the mixed feature F. catThe three branches are the i-th stage output from the local spatial prior flow, the i-th stage output from the global sequence modeling flow, and the output features from all preceding cross-view feature collaboration modules.

[0078] Note: Stage 1 does not have the output features of the cross-perspective feature collaboration module of the preceding stage, so it only needs the input of two branches, namely the output of Stage 1 from the local spatial prior flow and the output of Stage 1 from the global sequence modeling flow.

[0079] C2, Mixing features F cat The process involves splitting the flow and constructing a feature-enhanced dual-attention block. This structure contains two parallel branches, as detailed below:

[0080] C2.1, Branch 1: This branch uses a 3×3 convolution and adds a BN layer and ReLU activation function. To preserve the original receptive field, it can extract fine local edge features. ;

[0081] C2.2, Branch Two: This branch employs dilated convolution, incorporating a BN layer and a ReLU activation function. This expands the receptive field, enabling the capture of greater environmental contextual information. ;

[0082] C3. Add the output feature maps of the two branches element-wise, then perform information exchange and dimensionality compression through 1×1 convolution to generate multi-scale enhanced features F. fedab ;

[0083] C4. Enhance multi-scale features F fedab As input, it is fed into the dual attention correction layer; the dual attention correction layer specifically includes a channel attention calibration module and a spatial attention calibration module connected in series, as follows:

[0084] C4.1, Channel attention calibration for input features F fedab Global average pooling and a multilayer perceptron are performed, followed by sigmoid activation to generate channel weight vectors, which are then combined with the input features F. fedab Element-wise multiplication yields the channel calibration feature F. ca ;

[0085] C4.2, Spatial Attention Calibration Module Receive Channel After Calibration: Feature F ca Next, a 1×1 convolutional layer is used to compress the channel dimension, followed by sigmoid activation to generate a spatial weight mask, which is then combined with the input feature F. ca Element-wise multiplication yields the double attention-calibrated feature F. * ca .

[0086] C4.3, The dual attention calibration feature F, the final output of C4.2 * ca The input is fed into a 1×1 convolutional layer, and then a global residual connection is introduced. The output of this 1×1 convolutional layer is then combined with the mixed feature F from step C1. cat Element-wise addition is performed, and the results are then processed through the ReLU activation function to finally output the heterogeneous collaborative features of this level. .

[0087] The module for generating probabilistic maps of large water bodies from SAR images is used to generate probabilistic prediction maps of image patches. The specific steps are as follows:

[0088] D1. Input the deep heterogeneous collaborative features generated in the last stage of the cross-view feature collaboration module into the Atrous Spatial Pyramid Pooling (ASPP) module to capture multi-scale contextual information.

[0089] D2. The shallow heterogeneous collaborative features output from the first stage of the cross-view feature collaborative module are reduced in dimensionality by a 1×1 convolutional layer.

[0090] D3. After performing 4x bilinear sampling on the features output from step D1, concatenate them with the features output from step D2 in the channel dimension, and then feed the concatenated features into two cascaded 3×3 convolutional layers for deep fusion.

[0091] D4. The features fused in step D3 are passed through a 1×1 convolutional layer and then upsampled to the size of the input image patch. After passing through the Sigmoid activation function, the probability prediction map of the image patch is output.

[0092] The specific implementation of the square-shaped weighted fusion strategy is as follows:

[0093] E1. The sliding cutting strategy adopted aims to eliminate the low accuracy of image patch edge prediction and stitching artifacts. A square-shaped weight matrix W with the same size as the image patch is constructed. The weight matrix W exhibits a "square center high, edge decay" trend. The specific weight distribution is defined as follows:

[0094] E1.1 The central region of the weight matrix is ​​a rectangular area of ​​(H-2S)×(H-2S), and its weight value is always 1.0. H is the size of the image patch for sliding window cutting, and S is the step size used for sliding window cutting.

[0095] E1.2 The edge region of the weight matrix W is a range of width S surrounding the central region. Its weight value linearly decreases from 1.0 to 0 from the inside out. For any pixel (x, y) in the weight matrix W (where 0 ≤ x, y ≤ H), the specific weight value W(x, y) is calculated using the following formula:

[0096] W(x,y)=min( ) Formula 2

[0097] Where S is the sliding window cutting step size, H is the cutting size of the sliding window, x is the horizontal coordinate of the pixel, and y is the vertical coordinate of the pixel.

[0098] E2. Initialize two floating-point all-zero matrices with the same size as the original high-orbit SAR image, namely the full-image probability accumulation matrix M. prob and the total weight cumulative matrix M weigth .

[0099] E2.1, where the total graph probability accumulation matrix M prob Used to store the weighted sum of predicted probabilities;

[0100] E2.2, where the total graph weight cumulative matrix M weigth This is used to record the total weight value of each pixel that is covered, which is then used for subsequent normalization.

[0101] E3. Traverse all image patches that have completed inference. Let F be the probability prediction graph of the k-th image patch. local_k Its starting coordinates in the upper left corner of the original image are (u k ,v k For a local pixel (x, y) in this image patch, map it to global coordinates (u). k +x,v k +y), perform pixel-level cumulative updates according to the following formula:

[0102] M * prob (u k +x,v k +y)=M prob (u k +x,v k +y)+F local_k (x,y)×W(x,y)

[0103] Formula 3

[0104] Where M prob (u k +x,v k +y) represents the cumulative probability matrix M of the entire graph before accumulation. probPixels in k +x,v k The probability value of M + y) * prob (u k +x,v k +y) represents the cumulative probability matrix M of the entire graph after accumulation. prob Pixels in k +x,v k The probability value of M + y); weigth (u k +x,v k +y) represents the cumulative weight matrix M of the entire graph before accumulation. weight Medium pixel (u k +x,v k The weight value of M (+y), * weigth (u k +x,v k +y) represents the cumulative weight matrix M of the entire graph after accumulation. weight Medium pixel (u k +x,v k The weight values ​​are calculated as follows: +y). According to this formula, the pixel values ​​of the overlapping region will be fused with prediction information from multiple adjacent image patches, and the prediction result closer to the center of the image patch will contribute more.

[0105] E4. Using the full graph weight accumulation matrix M weigth The probability accumulation matrix of the entire image is normalized pixel by pixel to obtain the probability distribution map M of the entire image. final The specific calculation formula is as follows:

[0106] Formula 4

[0107] Where (x, y) are the pixel coordinates of the entire image, and a is a very small constant to prevent division by zero.

[0108] E5. Based on the set threshold, the probability distribution map of the entire image is binarized. Finally, the image is morphologically processed according to actual needs to obtain an extracted binary mask map of a large water body of the same size as the original high-orbit SAR image.

[0109] In one embodiment, the large-scale reasonable lake extraction model based on high-orbit SAR imagery mainly includes a heterogeneous dual-stream encoder for high-orbit SAR imagery, a cross-view feature collaboration module, and a large-scale water body probability map generation module for SAR imagery. The specific processing flow is as follows:

[0110] A1. Local prior flow first uses the first layer of convolution to extract initial features;

[0111] A2. The initial features sequentially pass through four residual stages (Stage 1-Stage 4). In each stage, the feature map passes through a series of residual blocks containing 1×1, 3×3, and 1×1 convolutional kernels. This branch is mainly used to capture the geometric boundaries and texture information of the water body. The output features from the four stages are denoted as F. 1 cnn F 2 cnn F 3 cnn F 4 cnn Their resolutions are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input features, respectively;

[0112] A3. The global sequence modeling flow divides the input image into 4×4 non-overlapping patches, flattens them, and maps them into a one-dimensional labeled sequence through a projection layer;

[0113] A4. The one-dimensional sequence sequentially undergoes four feature modeling stages. Each stage consists of several stacked Mamba blocks. Within each Mamba block, long sequence modeling is performed using a visual state space model, and pixel relationships across the entire image are captured through a multi-path scanning mechanism. The four stage features output by this branch are denoted as F. 1 mamba F 2 mamba F 3 mamba F 4 mamba Its scale is consistent with the corresponding stage of the local spatial a priori flow.

[0114] A5. The output features F of the local spatial prior flow at each stage. i cnn Output features F of global sequence modeling stream i mamba The data is fed into the cross-view feature collaboration module, which also needs to receive heterogeneous collaborative features output from all preceding stages. Stage 4 needs to receive F... 1 fusion、 F 2 fusion、 F 3 fusion The final stage 4 of the cross-perspective feature collaboration module will output the final heterogeneous collaborative feature F. 4 fusion ;

[0115] A6. The deep heterogeneous collaborative features F output from stage 4 of the cross-perspective feature collaboration module. 4 fusion The ASPP module is used to capture multi-scale context information.

[0116] A7. Extract the shallow heterogeneous collaborative features F from the first stage output of the cross-perspective feature collaboration module. 1 fusion Dimensionality reduction is achieved using a 1×1 convolutional layer;

[0117] A8. After performing 4x bilinear sampling on the features output in step A6, concatenate them with the features output in step A7 along the channel dimension, and then feed the concatenated features into two cascaded 3×3 convolutional layers for deep fusion.

[0118] A9. The features fused in step A8 are passed through a 1×1 convolutional layer and then upsampled to the size of the input image patch. After passing through the Sigmoid activation function, the predicted image of the image is output.

[0119] In one embodiment, the data processing flow of the cross-view feature collaboration module is as follows:

[0120] B1. For the i-th stage (i=1, 2, 3, 4), the module first receives three inputs: features F from the local spatial prior flow. i cnn Features F from the global sequence modeling stream i mamba The output features F from all preceding collaborative modules n fusion (n=1,2,3 and n) <i);

[0121] B2, F i mamba Reconstructing a one-dimensional sequence into a two-dimensional feature map, F n fusion Perform a 2x upsampling to align the sizes of the three input features;

[0122] B3. Concatenate the three aligned features along the channel dimension to form a hybrid feature F that includes local texture, contextual information, and preceding stage information. cat ;

[0123] B4. Mixed features F cat The data is fed into a multi-scale extraction layer, which contains two branches. One branch uses a standard 3×3 convolution to extract edge details, and the other branch uses a dilated 3×3 convolution with a dilation rate of 2. The features output from the two branches are then element-wise summed to generate the multi-scale enhanced feature F. fedab ;

[0124] B5. Enhance multi-scale features F fedab The data is fed into a dual attention correction layer, which specifically includes a channel attention correction module and a spatial attention correction module connected in series, as detailed below:

[0125] B5.1, Channel attention calibration for input features F fedab Global average pooling and a multilayer perceptron are performed, followed by sigmoid activation to generate channel weight vectors, which are then combined with the input features F. fedab Element-wise multiplication yields the channel calibration feature F. ca ;

[0126] B5.2, Feature F after spatial attention calibration module receives channel calibration ca Next, a 1×1 convolutional layer is used to compress the channel dimension, followed by sigmoid activation to generate a spatial weight mask, which is then combined with the input feature F. ca Element-wise multiplication yields the features after dual attention calibration.

[0127] B5.3. Input the dual attention calibration features from the final output of B5.2 into a 1×1 convolutional layer, and then introduce a global residual connection. Combine the output of this 1×1 convolutional layer with the mixed features F from step B3. cat Element-wise addition is performed, and the results are then processed through the ReLU activation function to finally output the heterogeneous collaborative features F of this level. i fusion .

[0128] In this invention, training data is labeled using LabelMe; the extraction model is trained using the PyTorch framework. The training loss function of the extraction model is obtained by weighting a mixture of binary cross-entropy loss, Dice loss, and Focal loss, and the model parameters are updated using the backpropagation algorithm until the loss function converges.

[0129] In one embodiment, a weighted average fusion strategy is used to remove stitching gaps and reconstruct a final mask extraction map of the same size as the original high-orbit SAR image. The specific implementation steps are as follows:

[0130] C1. Load the trained model weight file (.pt or .pth) into the large water body extraction model of the high-orbit SAR image. The inference process is accelerated by a high-performance GPU.

[0131] C2. Read the image sequence to be inferred obtained by sliding window segmentation, and finally output the prediction result by the SAR image large water body probability map generation module. After the model output layer is processed by the Sigmoid activation function, a pixel-level probability map F with a size of 1024*1024 is generated. local_k , where each pixel value P(x,y)∈[0,1] represents the confidence level that the pixel belongs to the water body, and x and y represent the position of the pixel in row x and column y;

[0132] C3. In this embodiment, a cutting strategy with a size of 1024*1024 and a step size of 256 is adopted. To eliminate prediction uncertainty and stitching artifacts at the edges of image blocks, a square weight matrix W with the same size as the image block (1024*1024) is constructed. The weight matrix W adopts a trend of increasing in the center and decreasing linearly at the edges, so that the prediction results of the image block center region dominate during fusion, reducing the weight coefficient of the image edge region, thereby reducing the uncertainty of the edge region and making the overall stitched image have better continuity and accuracy. For any pixel (x,y) in the weight matrix W (where 0≤x,y≤H), the specific weight value W(x,y) is calculated using the following formula:

[0133] W(x,y)=min( ) Formula 5

[0134] Where x is the x-coordinate of the pixel and y is the y-coordinate of the pixel.

[0135] C4. Construct two all-zero matrices with the same size as the original high-orbit SAR image, namely the probability accumulation matrix M. prob and weight cumulative matrix M weight .

[0136] C5. Traverse all image patches that have completed inference. Let F be the probability prediction graph of the k-th image patch. local_k Its starting coordinates in the upper left corner of the original image are (u k ,v k For a local pixel (x, y) in this image patch, map it to global coordinates (u). k +x,v k +y), perform pixel-level cumulative updates according to the following formula:

[0137] M * prob (u k +x,v k +y)=M prob (u k +x,v k +y)+F local_k (x,y)×W(x,y)

[0138] M * weigth (u k +x,v k +y)=M prob (u k +x,v k +y)+W(x,y) Formula 6

[0139] Where Mprob (u k +x,v k +y) represents the cumulative probability matrix M of the entire graph before accumulation. prob Pixels in k +x,v k The probability value of M + y) * prob (u k +x,v k +y) represents the cumulative probability matrix M of the entire graph after accumulation. prob Pixels in k +x,v k The probability value of M + y); weight (u k +x,v k +y) represents the cumulative weight matrix M of the entire graph before accumulation. weight Medium pixel (u k +x,v k The weight value of M (+y), * weigth (u k +x,v k +y) represents the cumulative weight matrix M of the entire graph after accumulation. weight Medium pixel (u k +x,v k The weight values ​​are calculated as follows: +y). According to this formula, the pixel values ​​of the overlapping region will be fused with prediction information from multiple adjacent image patches, and the prediction result closer to the center of the image patch will contribute more.

[0140] C6. Using the full graph weight accumulation matrix M weigth The probability accumulation matrix of the entire image is normalized pixel by pixel to obtain the probability distribution map M of the entire image. final The specific calculation formula is as follows:

[0141] Formula 7

[0142] Where (x, y) are the pixel coordinates of the entire image, and a is a very small constant to prevent division by zero.

[0143] C7. Based on the set threshold, the probability distribution map of the whole image is binarized. Finally, the image is morphologically processed according to actual needs to obtain a binary mask map of a large water body of the same size as the original high-orbit SAR image.

[0144] According to the formula The large-scale water extraction effect of the present invention is shown in Table 1.

[0145] Table 1 Comparison of extraction effects of different methods

[0146] IoU 0.3705 0.4568 0.7403 0.8133

[0147] As shown in Table 1, the method of this invention has the highest IoU value in extracting large water bodies in the same large-scale high-orbit SAR image dataset, compared to the OTSU method, the watershed algorithm, the adaptive threshold algorithm, the combination of texture analysis, and the Deeplabv3plus deep learning algorithm.

[0148] Figure 4 The Deeplabv3plus algorithm and the method provided in this invention are used to extract large water bodies in high-orbit SAR images, including local magnified images, by combining the watershed algorithm, adaptive threshold segmentation method and texture analysis.

[0149] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

[0150] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for extracting water body contours from high-orbit satellite imagery, characterized in that, Includes the following steps: The original image of the high-orbit SAR image is acquired, and the original image is divided into a sliding window to generate an image patch sequence; A heterogeneous dual-stream encoder for high-orbit SAR images is constructed. The heterogeneous dual-stream encoder includes a water body local spatial prior flow and a water body global sequence modeling flow. The image patch sequence is input into the heterogeneous dual-stream encoder to extract dual-branch heterogeneous features. Deep interaction is achieved on the dual-branch heterogeneous features in the heterogeneous dual-stream encoder to obtain deep heterogeneous collaborative features and shallow heterogeneous collaborative features in multiple feature extraction stages; The deep heterogeneous collaborative features are processed by hollow spatial pyramid pooling, and the shallow heterogeneous collaborative features are processed by dimensionality reduction. The processed deep features are upsampled and fused with the dimensionality-reduced shallow features. After upsampling and activation function processing, a pixel-level probability prediction map matching the size of the input image block is output. A square-shaped weighted fusion strategy is used to reconstruct the probability prediction maps of all the image blocks; The sliding cutting strategy adopted constructs a square weight matrix W with the same size as the image patch. The numerical distribution of the weight matrix W shows the trend of "high in the center of the square and decaying at the edges". For any pixel (x, y) in the weight matrix W, the weight value is calculated using the following formula: W(x,y)=min( ) The values ​​of x and y are 0 ≤ x ≤ H-1 and 0 ≤ y ≤ H-1, respectively. H is the size of the image block to be cut by the sliding window, and S is the step size used for the sliding window cutting. Initialize the full graph probability accumulation matrix M prob and the total weight cumulative matrix M weigth Iterate through all the image patches that have completed the reasoning. Let F be the probability prediction map of the k-th image patch. local_k Perform pixel-level cumulative update of the full-image probability accumulation matrix M. prob and the total weight cumulative matrix M weigth ; Use the cumulative weight matrix M of the entire graph after cumulative update. weigth For the total graph probability accumulation matrix M prob Pixel-by-pixel normalization is performed to obtain the overall probability distribution map M. final ; A threshold is set to binarize the probability distribution map of the entire image. Morphological processing is performed on the image as needed to obtain a binary mask map of the same size as the original high-orbit SAR image for extracting large water bodies, thus completing the end-to-end large water body extraction task.

2. The method for extracting water body contours from high-orbit satellite imagery according to claim 1, characterized in that, The image patch sequence is input into the heterogeneous dual-stream encoder to extract dual-branch heterogeneous features, including the following steps: The local spatial prior flow of the water body uses convolutional layer stacking to capture the geometric boundary and texture information of the water body, and the global sequence modeling flow of the water body uses a multi-path scanning mechanism to transform the two-dimensional image into a one-dimensional sequence and establish long-distance dependencies between pixel blocks across the entire image.

3. The method for extracting water body contours from high-orbit satellite imagery according to claim 2, characterized in that, The prior flow in the local water body adopts a convolutional neural network architecture. The input end is set with an initial convolutional pooling layer, followed by multiple convolutional feature extraction stages in sequence. Local features F are extracted using convolutional kernels. cnn ; The global sequence modeling flow for water bodies has an image patch serialization embedding layer at the input end, uses a state-space model for long sequence modeling, and employs a multi-path scanning mechanism within each Mamba block. For the input one-dimensional labeled sequence, it simultaneously performs scanning and modeling in the following three directions: Direction 1: Process sequentially along a one-dimensional sequence to capture regular dependencies; Direction 2: Process in reverse order along the one-dimensional sequence to supplement the reverse correlation information that is complementary to the sequential processing; Direction 3: Randomly shuffle the one-dimensional sequence and then scan it to supplement the global correlation in the image that has no fixed order. At the same time, this scanning method can help the model identify irregular water bodies. In each scan path, the sequence features are updated using the discretized state-space equation. The core formula is as follows: ； ； Where t represents a one-dimensional position index in the input sequence. The input feature is the pixel position at the current time. This means that before processing the t-th position, the model has accumulated historical global information. Represents global information after the current position is updated. Output features. Matrix , It is dynamically generated through an input-dependent selective mechanism, which allows the model to adaptively reset or save its state based on the image content.

4. The method for extracting water body contours from high-orbit satellite imagery according to claim 3, characterized in that, Deep interaction is achieved on the dual-branch heterogeneous features in the heterogeneous dual-stream encoder to obtain deep heterogeneous collaborative features and shallow heterogeneous collaborative features at multiple feature extraction stages, including the following steps: The dual-branch heterogeneous features are input into the cross-view feature collaboration module for deep interactive processing: For the i-th stage, the cross-view feature collaboration module receives three feature inputs, namely the i-th stage output feature F from the prior flow of the local spatial flow of the water body. cnn The output feature F from the i-th layer of the global sequence modeling flow of the water body mamba and the output features F from all preceding collaborative modules fusion The first stage does not require the output features of the preceding collaborative module; Output feature F cnn、 Output feature F mamba and output features F fusion Alignment is performed in the spatial dimension, followed by concatenation in the channel dimension to obtain the hybrid feature F. cat ; Mixed features F cat The data is fed into a multi-scale refinement layer, which contains parallel first and second convolutional branches. The first convolutional branch uses standard convolution to extract partial details, while the second convolutional branch uses dilated convolution to capture contextual information. The outputs of the two branches are then element-wise summed to generate an enhanced feature F. fedab ; Enhance feature F fedab Semantic correction is performed sequentially through a dual attention correction layer to generate dual-calibrated features; The dual calibration features are convolved with the hybrid feature F after a 1×1 convolution. cat Perform residual connections and output the heterogeneous collaborative features of this stage.

5. The method for extracting water body contours from high-orbit satellite imagery according to claim 4, characterized in that, The dual attention correction layer performs semantic correction, including the following steps: The dual attention correction layer comprises two parts: a channel attention correction module and a spatial attention correction module. The channel attention correction module performs global average pooling on the input feature map to obtain channel descriptors. The channel descriptors are nonlinearly interacted using a multilayer perceptron, and a channel weight vector is generated by sigmoid activation. This vector is then multiplied element-wise with the input feature map to obtain F. ca And output it to the spatial attention correction module; The spatial attention correction module receives the feature F after channel calibration. ca Channel compression is performed through a 1×1 convolutional layer, followed by sigmoid activation to generate a spatial weight mask, which is then compared with the feature map F. ca Pixel-by-pixel multiplication generates dual-calibration feature F * ca .

6. The method for extracting water body contours from high-orbit satellite imagery according to claim 5, characterized in that, Output a pixel-level probability prediction map that matches the size of the input image patch, including the following steps: The deep heterogeneous collaborative features output from the final heterogeneous stage of the cross-perspective feature collaboration module are input into the hollow space pyramid pooling module to capture multi-scale contextual information. The shallow heterogeneous collaborative features output from the first stage of the cross-view feature collaboration module are dimensionality reduced by a convolutional layer to obtain the dimensionality-reduced shallow features. After upsampling the deep features processed by the hollow space pyramid pooling module, they are concatenated with the dimensionality-reduced shallow features in the channel dimension to obtain the concatenated features. The concatenated features are fused through a series of convolutional layers and mapped to a prediction probability map.

7. A water body contour extraction system for high-orbit satellite imagery, employing the water body contour extraction method for high-orbit satellite imagery as described in any one of claims 1-6, characterized in that, include: The preprocessing module acquires the original image of the high-orbit SAR image, performs sliding window segmentation on the original image, and generates an image block sequence. A heterogeneous dual-stream encoder for high-orbit SAR images, comprising: a local spatial prior flow of water and a global sequence modeling flow of water, wherein the image patch sequence is input into the heterogeneous dual-stream encoder to extract dual-branch heterogeneous features; A cross-perspective feature collaboration module is connected to the local spatial prior flow of the water body and the global sequence modeling flow of the water body, respectively, and is used to achieve deep interaction of the dual-branch heterogeneous features in the heterogeneous dual-stream encoder to obtain deep heterogeneous collaborative features and shallow heterogeneous collaborative features in multiple feature extraction stages. The water mask generation module performs hollow spatial pyramid pooling processing on the deep heterogeneous collaborative features and dimensionality reduction processing on the shallow heterogeneous collaborative features. After upsampling the processed deep features, it merges them with the dimensionality-reduced shallow features. Then, after upsampling and activation function processing, it outputs a pixel-level probability prediction map that matches the size of the input image block. The binary mask generation module reassembles the probability prediction maps of all image blocks according to the cutting size and step size of the image block sequence using a square-shaped weighted fusion strategy, generating a full-map probability prediction map with the same size as the original high-orbit SAR image. A threshold is set to perform binary partitioning on the probability prediction map to obtain the final binary mask map, thus completing the end-to-end large-body water extraction task.

Images