River and lake remote sensing image segmentation method and system based on convolutional neural network

Abstract

This invention relates to the field of remote sensing image segmentation technology, specifically to a method and system for river and lake remote sensing image segmentation based on convolutional neural networks. The method includes: acquiring and preprocessing multi-source remote sensing images to construct a multi-scale image pyramid; performing preliminary extraction of target-level images based on a water body spectral feature library to generate candidate water body masks; constructing a convolutional neural network with a dilated convolutional encoder and a skip connection decoder; concatenating the candidate masks with the original image channels as network input for training and optimization; using the trained model to infer new remote sensing images, outputting a probability map, and then post-processing to obtain the final water body segmentation result. This method improves the accuracy of water body segmentation and its adaptability to complex scenes by integrating spectral prior knowledge with deep learning.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image segmentation technology, and in particular to a method and system for segmenting river and lake remote sensing images based on convolutional neural networks. Background Technology

[0002] Precise segmentation of river and lake water areas is a crucial aspect of water resource monitoring and management. Existing technical solutions mainly fall into two categories: semi-automatic methods based on physical models or spectral features, and end-to-end semantic segmentation methods based on deep learning.

[0003] Spectral feature-based methods extract water body information by designing specific spectral indices or setting empirical thresholds. The core drawback of these methods lies in their heavy reliance on prior expert knowledge, and the insufficient adaptability of pre-defined feature rules to complex geographical environments and variable imaging conditions. When processing images with spectral obfuscation or noise interference, these methods struggle to maintain stable segmentation accuracy and integrity, and boundary information is easily lost.

[0004] Deep learning-based methods, such as those employing encoder-decoder structures, can automatically learn feature representations. However, these methods typically use only raw imagery as input, and their performance is severely limited by the quality and size of the training dataset. Given the high cost of remote sensing image annotation, models are prone to overfitting to training samples, posing a challenge to their generalization ability. Furthermore, the network needs to learn water features from scratch from massive amounts of data, failing to effectively utilize mature prior knowledge in the remote sensing field, leading to unstable segmentation results when data is limited or the scene is unfamiliar.

[0005] How to design a technical solution that can integrate domain prior knowledge and deep feature learning capabilities to reduce dependence on large-scale labeled data and improve the segmentation robustness and accuracy of the model in complex and variable remote sensing scenarios is a problem that needs to be solved. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a method and system for segmenting remote sensing images of rivers and lakes based on convolutional neural networks.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a river and lake remote sensing image segmentation method based on convolutional neural networks, comprising:

[0008] Acquire multi-source remote sensing image data, and perform radiometric correction and geometric registration processing on the multi-source remote sensing image data to generate a standardized remote sensing image sequence;

[0009] Multi-scale pyramid construction is performed on the standardized remote sensing image sequence to generate image pyramid hierarchical data containing different spatial resolutions;

[0010] The target level is selected from the image pyramid level data, and the water body region is initially extracted from the target level image based on the water body spectral feature library to generate a candidate water body mask;

[0011] A convolutional neural network model with an encoder-decoder structure is constructed. The encoder part uses dilated convolutional layers to extract features at multiple scales. The decoder part uses progressive upsampling and skip connections to fuse deep semantic features and shallow detail features.

[0012] The candidate water body mask is stitched together with the corresponding target layer image to form enhanced neural network input data;

[0013] The enhanced neural network input data is fed into the convolutional neural network model for end-to-end training, and the network parameters are optimized by minimizing the difference between the segmentation results and the real labels.

[0014] The optimized convolutional neural network model is used to perform forward inference on the new remote sensing image and output the initial water body segmentation probability map.

[0015] Adaptive threshold segmentation and morphological post-processing are performed on the initial water body segmentation probability map to generate a river and lake water area segmentation result map.

[0016] As a further aspect of the present invention, the step of performing radiometric correction and geometric registration processing on the multi-source remote sensing image data to generate a standardized remote sensing image sequence includes:

[0017] Read raw remote sensing image data and its corresponding metadata information acquired by different sensors;

[0018] Based on the radiometric calibration parameters in the metadata information, the original remote sensing image data is radiometrically calibrated to convert the pixel brightness values ​​into surface reflectance.

[0019] Atmospheric correction models were used to eliminate atmospheric effects on the radiometrically calibrated images to obtain true surface reflectance data.

[0020] A remote sensing image with high geometric accuracy is selected as a reference benchmark, and the control point features in the reference benchmark image are extracted.

[0021] Control point matching and geometric transformation are performed on the remaining remote sensing images to achieve spatial alignment of all images with the reference image.

[0022] The geometrically registered images are resampled to unify the spatial resolution and pixel size, forming the standardized remote sensing image sequence.

[0023] As a further aspect of the present invention, the step of constructing a multi-scale pyramid from the standardized remote sensing image sequence to generate image pyramid hierarchy data containing different spatial resolutions includes:

[0024] Set the number of pyramid levels and the downsampling factor of each level relative to the original image;

[0025] Starting with the original resolution image in the standardized remote sensing image sequence, it serves as the bottom layer of the pyramid, namely the zeroth layer;

[0026] The image at the current level is smoothed using a Gaussian convolution kernel, and then downsampled at a specified rate to generate the image at the next higher level.

[0027] Repeat the smoothing and downsampling process until image data of all preset levels are generated;

[0028] All levels of image data are organized according to resolution from high to low to construct the image pyramid hierarchical data.

[0029] As a further aspect of the present invention, the preliminary water body region extraction of the target-level image based on the water body spectral feature library to generate a candidate water body mask includes:

[0030] Load a predefined water body spectral feature library, which contains the spectral reflectance curves and statistical characteristics of typical water bodies in different spectral bands;

[0031] Calculate the spectral feature vector of each pixel in the target layer image;

[0032] Calculate the similarity measure between the spectral feature vector of each pixel and the standard water body spectral features in the water body spectral feature library;

[0033] A similarity threshold is set, and pixels with a similarity metric higher than the threshold are marked as potential water body pixels;

[0034] Perform region growing operations on the marked potential water body pixels to merge spatially adjacent and spectrally similar pixels into connected regions.

[0035] Remove excessively small noisy connected regions to generate the candidate water body mask.

[0036] As a further aspect of the present invention, the construction of the convolutional neural network model with an encoder-decoder structure includes:

[0037] The encoder is designed to consist of four downsampling stages. Each downsampling stage contains two convolutional layers and one max pooling layer. The convolutional layers use a 3x3 convolutional kernel and a modified linear unit activation function. The max pooling layer uses a 2x2 pooling window to halve the feature map size.

[0038] Four parallel dilated convolutional layers are introduced after the last downsampling stage of the encoder section. The four parallel dilated convolutional layers use different dilation coefficients to extract multi-scale contextual information.

[0039] The decoder part is designed to consist of four upsampling stages. Each upsampling stage uses bilinear interpolation to double the size of the feature map. Then, the upsampling results are concatenated with the feature maps of the corresponding stages in the encoder part by channel dimension.

[0040] Two convolutional layers are set after each upsampling stage in the decoder section, and the convolutional layers use a one-to-one convolutional kernel to gradually reduce the number of channels to the number of target categories;

[0041] At the end of the decoder section, a pixel-level classification layer is used, which outputs the probability value of each pixel belonging to the water body category through an activation function.

[0042] As a further aspect of the present invention, the step of channel-by-channel stitching of the candidate water body mask with the corresponding target layer image to form enhanced neural network input data includes:

[0043] Adjust the candidate water body mask to the same size as the target layer image;

[0044] The candidate water body mask is used as additional channel data and superimposed on the original multispectral band data of the target layer image;

[0045] The spliced ​​multi-channel data is normalized to unify the numerical range of each channel to the interval between zero and one.

[0046] The normalized multi-channel data is organized into the input data format required by the convolutional neural network model to form the enhanced neural network input data.

[0047] As a further aspect of the present invention, the step of inputting the enhanced neural network input data into the convolutional neural network model for end-to-end training includes:

[0048] Prepare a training dataset containing a large number of labeled river and lake water areas;

[0049] The enhanced neural network input data is input to the encoder part, which extracts image features step by step through stacked convolutional and pooling layers, and uses dilated convolution to expand the receptive field to capture multi-scale contextual information.

[0050] The decoder receives the high-level feature map output by the encoder and gradually restores the spatial resolution of the feature map through deconvolution layers or upsampling layers.

[0051] During the upsampling process of the decoder part, the shallow feature maps and deep feature maps of the corresponding levels of the encoder part are fused through skip connections to combine detailed information and semantic information.

[0052] The final output is a feature map of the same size as the input image, which is then processed by an activation function to obtain the probability that each pixel belongs to the water body category.

[0053] The error between the network output probability and the true label is calculated using a loss function, and the weight parameters of the convolutional neural network model are updated using the backpropagation algorithm.

[0054] The iteration process continues until the model converges or reaches the preset number of training rounds.

[0055] As a further aspect of the present invention, the step of calculating the error between the network output probability and the true label using a loss function, and updating the weight parameters of the convolutional neural network model using a backpropagation algorithm, includes:

[0056] The weighted sum of the cross-entropy loss function and the Dice coefficient loss function is used as the overall loss function, where the cross-entropy loss function is used to measure the pixel-level classification accuracy, and the Dice coefficient loss function is used to optimize the segmentation integrity of the water body region.

[0057] After calculating the network output probability in each forward propagation, the network output probability is compared with the real annotation pixel by pixel, and the cross-entropy loss term and Dice loss term are calculated respectively.

[0058] The cross-entropy loss term and the Dice loss term are weighted and summed according to the preset weight ratio to obtain the overall loss value of the current training batch.

[0059] The gradient of the overall loss value with respect to the weight parameters of each layer of the convolutional neural network model is calculated using automatic differentiation techniques.

[0060] The gradient is optimized using an adaptive moment estimation algorithm, and the learning rate of each weight parameter is dynamically adjusted.

[0061] Update all weight parameters of the convolutional neural network model according to the calculated gradient direction to complete one training iteration.

[0062] As a further aspect of the present invention, the step of performing adaptive threshold segmentation and morphological post-processing on the initial water body segmentation probability map to generate a river and lake water area segmentation result map includes:

[0063] An adaptive segmentation threshold is calculated for the initial water body segmentation probability map using either the maximum inter-class variance method or a method based on local image statistical characteristics.

[0064] The calculated adaptive segmentation threshold is used to binarize the probability map. Pixels with probability values ​​greater than the threshold are classified as water bodies, and those without are classified as non-water bodies, thus obtaining a binary segmentation map.

[0065] A morphological opening operation is performed on the binary segmentation image to eliminate small noise points and smooth water body boundaries;

[0066] Connectivity analysis is performed on the morphologically processed binary segmentation map to remove isolated regions with an area smaller than the preset minimum water body area.

[0067] The processed binary image is output as the final segmentation result image of the river and lake water area.

[0068] As a further aspect of the present invention, the present invention also includes a river and lake remote sensing image segmentation system based on a convolutional neural network. The system includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the river and lake remote sensing image segmentation method based on a convolutional neural network as described above.

[0069] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0070] Based on a pre-built water body spectral feature library, preliminary water body region extraction is performed on the target-level image to generate candidate water body masks. This step transforms the domain knowledge in traditional remote sensing interpretation into a structured spatial prior information. This candidate mask, acting as a high-confidence guiding signal, initially focuses the neural network's attention on possible regions, effectively filtering out a large amount of obvious non-water background interference. This allows the network to focus its learning resources on solving more challenging tasks such as boundary refinement, small target detection, and the identification of spectral obfuscations such as shadows, clouds, and buildings, rather than learning from scratch to recognize the most basic water body spectral features. This enhances the model's discrimination ability and robustness in complex scenes.

[0071] The generated candidate water body masks are concatenated with the corresponding original target-level images along the channel dimension to form enhanced neural network input data. This data input method creates a "feature collaboration" mechanism, enabling the convolutional neural network to process the rich spectral texture information of the original image and the spatial structure priors provided by the candidate masks in parallel during the same forward propagation. The network naturally learns through convolutional layers how to balance and fuse these two types of information, i.e., how to trust or correct the initial segmentation results. The candidate masks, as additional guiding channels, assist the decoder in more accurately locating water body boundaries and connecting broken sections during feature upsampling and detail recovery, while suppressing false positive regions caused by spectral confounding. This method essentially organically integrates traditional methods and deep learning methods at the data level, achieving complementary advantages. Attached Figure Description

[0072] Figure 1 This is a flowchart of the river and lake remote sensing image segmentation method based on convolutional neural networks described in this invention;

[0073] Figure 2 A flowchart for standardizing and preprocessing remote sensing data;

[0074] Figure 3 Flowchart for constructing a multi-scale image pyramid;

[0075] Figure 4 This represents the quantized correspondence between image pyramid level resolution and processing time.

[0076] Figure 5 Correlation analysis curves of water body segmentation accuracy and processing time at different spatial resolutions are presented. Detailed Implementation

[0077] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0078] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0079] See Figure 1 This process involves acquiring multi-source remote sensing image data from different platforms or sensors. Preprocessing this data, including radiometric correction and geometric registration, generates a standardized remote sensing image sequence with unified radiometric metrics and spatial references. A multi-scale pyramid is constructed from this standardized image sequence, generating a hierarchical image pyramid data set ranging from the original resolution to various lower resolutions. A target level is selected from the image pyramid data based on task requirements. Preliminary water body region extraction is performed on the target level image using a pre-built water body spectral feature library, generating a candidate water body mask that identifies the potential water body's extent. A convolutional neural network model with an encoder-decoder structure is constructed. The encoder part uses dilated convolutional layers for multi-scale feature extraction, while the decoder part uses progressive upsampling and skip connections to fuse deep semantic features and shallow detail features. The candidate water body mask is concatenated with the corresponding target level image along the channel dimension, forming neural network input data that enhances prior information. This enhanced neural network input data is then fed into the constructed convolutional neural network model for end-to-end training. The network parameters are optimized by minimizing the difference between the model's segmentation results and the ground truth annotations. An optimized convolutional neural network model is used for forward inference on new remote sensing images, and the model outputs an initial water body segmentation probability map. Adaptive thresholding and morphological post-processing are then applied to the initial water body segmentation probability map to generate the final, accurate binarized river and lake water area segmentation result map.

[0080] See Figure 2In one embodiment of the present invention, raw remote sensing image data and their corresponding metadata information acquired by different sensors are read. The raw remote sensing image data may originate from multispectral sensors, such as the OLI sensor of the Landsat-8 satellite and the MSI sensor of the Sentinel-2 satellite. The metadata information includes imaging time, solar elevation angle, radiometric calibration coefficients, and atmospheric parameters. Based on the radiometric calibration parameters in the metadata information, the raw remote sensing image data is radiometrically calibrated, converting pixel brightness values ​​into apparent reflectance or radiance of the upper atmosphere. An atmospheric correction model is used to eliminate atmospheric effects on the radiometrically calibrated image. The atmospheric correction model, such as the 6S model or the FLAASH model, calculates the atmospherically corrected surface reflectance data by inputting aerosol optical thickness and water vapor content parameters at the time of imaging. In a specific implementation, a remote sensing image with high geometric accuracy is selected as a reference benchmark. The high geometric accuracy remote sensing image is, for example, a remote sensing image calibrated with precise orbit and ground control points. Clear and stable feature points are manually or automatically selected from the reference benchmark image as control points. In some embodiments, control point matching and geometric transformation are performed on the remaining remote sensing images. Control point matching employs a feature-based matching algorithm, such as the scale-invariant feature transformation algorithm, to find corresponding points on the image to be registered that correspond to the control points of the reference image. Based on the coordinates of multiple sets of corresponding points, a polynomial transformation model is used to solve the geometric transformation parameters, achieving spatial alignment of all images with the reference image. Optionally, resampling is performed on the geometrically registered images. The resampling method employs bilinear interpolation or cubic convolution to unify all images to the same spatial resolution and pixel size, forming a standardized remote sensing image sequence. Each image in the standardized remote sensing image sequence has the same geographic extent, projected coordinates, and pixel size.

[0081] In some embodiments, radiometric calibration processing converts the digital quantization values ​​of the original image into physical quantities based on calibration coefficients provided by the sensor. The conversion process follows the formula:

[0082]

[0083] in: For band Surface reflectivity, For the band obtained after radiometric calibration Radiance The distance between the Earth and the Sun. For band The average solar irradiance at the top of the atmosphere. The solar zenith angle is used. It can be understood that atmospheric correction models use the radiative transfer equation to simulate the propagation of electromagnetic waves in the atmosphere, thereby eliminating the influence of atmospheric scattering and absorption on the surface reflection signal and obtaining true surface reflectance data. During control point matching and geometric transformation, the matching accuracy of corresponding points directly affects the spatial alignment result. The matching error is evaluated using the root mean square error (RMSE). When the RMS error exceeds a preset threshold, control point pairs need to be reselected or adjusted. It can be understood that resampling operations, while changing the image pixel size, introduce the recalculation of pixel values. Bilinear interpolation achieves a balance between computational efficiency and accuracy and is a commonly used resampling method.

[0084] See Figure 3 In one embodiment of the present invention, the number of pyramid levels and the downsampling factor of each level relative to the original image are preset. The number of pyramid levels can be set to four, and the downsampling factor of each level is set to twice that of the previous level, i.e., the resolution is halved sequentially. Starting from the original resolution image in the standardized remote sensing image sequence, this is taken as the bottom layer of the pyramid, i.e., the zeroth layer. The zeroth layer image has the highest spatial resolution, for example, 10 meters. A Gaussian convolution kernel is used to smooth the current layer image. The size of the Gaussian convolution kernel is 5x5, and the standard deviation is set to 1.0. The smoothing process is used to suppress the aliasing effect that may be caused by downsampling. In a specific implementation, the smoothed image is downsampled at a specified factor. The downsampling operation is achieved by extracting pixels by alternating rows and columns to generate the next layer image. The resolution of the next layer image becomes twice that of the current layer image, for example, from 10 meters to 20 meters. Repeat the smoothing and downsampling process until all preset levels of image data are generated, for example, generating four levels of images with resolutions of 10 meters, 20 meters, 40 meters, and 80 meters. Organize all levels of image data according to resolution from high to low to construct an image pyramid hierarchical data structure. The image pyramid hierarchical data structure is used as a whole for subsequent steps.

[0085] In some embodiments, a predefined water body spectral feature library is loaded. This library contains spectral reflectance curves and statistical characteristics of typical water bodies in different spectral bands. The spectral reflectance curves of typical water bodies can be obtained through field measurements or from a standard spectral database. The statistical characteristics include the mean and variance of reflectance for each band. The spectral feature vector of each pixel in the target layer image is calculated. This vector is composed of the reflectance values ​​of that pixel in multiple selected bands, such as near-infrared and short-wave infrared bands, arranged sequentially. The similarity measure between the spectral feature vector of each pixel and the standard water body spectral features in the water body spectral feature library is calculated using Mahalanobis distance. The Mahalanobis distance formula is:

[0086]

[0087] in: For the spectral feature vector of the target pixel, This represents the average spectral feature vector of standard water bodies in the water body spectral feature library. This is the inverse of the covariance matrix of the spectral feature vectors. It can be understood that a similarity threshold is set, and pixels with a Mahalanobis distance lower than the threshold are marked as potential water body pixels. The similarity threshold is determined by analyzing the Mahalanobis distance distribution between water and non-water body pixels in the training samples. Optionally, a region growing operation is performed on the marked potential water body pixels. Using each potential water body pixel as a seed point, the spectral feature vectors of its eight neighboring pixels are checked. If the Euclidean distance between the spectral feature vectors of the neighboring pixels and the spectral feature vectors of the seed pixel is less than the region growing threshold, then the neighboring pixels are merged into the current region. The region growing operation is iterated until no new pixels can be merged. Noisy connected regions with excessively small areas are removed. The area threshold is set based on the spatial resolution of the image and the minimum identifiable water body target; for example, connected regions with fewer than ten pixels are removed, generating the final candidate water body mask.

[0088] In one embodiment of the invention, the encoder consists of four downsampling stages. Each downsampling stage includes two convolutional layers and one max-pooling layer. The convolutional layers use 3x3 convolutional kernels combined with a modified linear unit activation function (MRU), which introduces non-linearity after the convolution operation. The max-pooling layer uses a 2x2 pooling window to halve the feature map size. Each downsampling stage reduces the spatial size of the feature map to half that of the input while increasing the number of feature channels. After the last downsampling stage of the encoder, four parallel dilated convolutional layers are introduced. These four parallel dilated convolutional layers use different dilation coefficients, such as 2, 4, 8, and 12, to extract multi-scale contextual information. The outputs of the four parallel dilated convolutional layers are concatenated along the channel dimension. In some embodiments, the decoder consists of four upsampling stages. Each upsampling stage doubles the feature map size using bilinear interpolation. The upsampling result is then concatenated along the channel dimension with the feature map of the corresponding stage in the encoder. This concatenation operation integrates deep semantic features and shallow detail features. Two convolutional layers are set after each upsampling stage in the decoder section. These layers use a 1x1 convolutional kernel to progressively reduce the number of channels to the number of target categories (two categories: water and non-water). At the end of the decoder section, a pixel-level classification layer is used, outputting the probability value of each pixel belonging to the water category through an activation function, specifically the sigmoid function. It can be understood that the dilated convolutional layer expands the receptive field of the convolutional kernel by introducing holes without increasing the number of parameters. The formula for calculating its output feature map size is:

[0089]

[0090] in: To determine the output feature map size, For the input feature map size, For the number of fill pixels, The void coefficient is... This is the original kernel size. is the convolution stride.

[0091] In specific implementation, the candidate water body mask is adjusted to the same size as the target layer image. The size adjustment operation uses the nearest neighbor interpolation method to ensure the binary attribute of the mask. The candidate water body mask is used as additional channel data and superimposed on the original multispectral band data of the target layer image. For example, if the target layer image has four spectral bands, superimposing the candidate water body mask forms five-channel input data. Optionally, the stitched multi-channel data is normalized to unify the numerical range of each channel to the interval between zero and one. The normalization formula is to subtract the global minimum value of each channel from its pixel value and then divide by the channel's numerical range. The normalized multi-channel data is organized into the input data format required by the convolutional neural network model. The input data format is a four-dimensional tensor, with dimensions being batch size, image height, image width, and number of channels, forming enhanced neural network input data. In some embodiments, bilinear interpolation upsampling calculates the value of the newly inserted pixel by the distance-weighted average of the values ​​of the four surrounding known pixels, gradually restoring the spatial resolution of the feature map. Skip connections directly pass feature maps from the encoder with the same spatial dimensions as the current decoder layer, and concatenate them with the upsampled feature maps from the decoder along the channel dimension. In essence, the 1x1 convolutional layer primarily adjusts the channel dimensions of the feature maps and performs feature combination without changing the spatial dimensions of the feature maps, ultimately adjusting the number of channels to match the number of target classes.

[0092] In one embodiment of the present invention, a training dataset is prepared, comprising a large number of remote sensing image samples of labeled river and lake water areas. Each image in the training dataset is accompanied by a precise ground truth map of the water area. The ground truth map is a binary image in which the pixel value of the water area is one and the pixel value of the non-water area is zero. The enhanced neural network input data is input to the encoder part. The encoder part extracts image features step by step through stacked convolutional layers and pooling layers, and uses dilated convolution to expand the receptive field to capture multi-scale contextual information. The number of channels of the feature map increases with the depth of downsampling and the spatial size gradually decreases. The encoder receives the high-level feature map output from the encoder and gradually restores the spatial resolution of the feature map through deconvolution layers or upsampling layers. The upsampling operation uses transposed convolution or interpolation to enlarge the size of the feature map. During the upsampling process in the decoder, the shallow and deep feature maps of the corresponding levels in the encoder are fused through skip connections to combine detailed information and semantic information. The fusion method is channel-dimensional concatenation. The network finally outputs a feature map with the same size as the input image. After activation function processing, the probability of each pixel belonging to the water body category is obtained, with the probability value ranging from zero to one. In some embodiments, the error between the network output probability and the ground truth label is calculated using a loss function, and the weight parameters of the convolutional neural network model are updated using the backpropagation algorithm. The overall loss function is a weighted sum of the cross-entropy loss function and the Dice coefficient loss function, where the cross-entropy loss function measures pixel-level classification accuracy, and the Dice coefficient loss function optimizes the segmentation integrity of water regions. After each forward propagation calculation to obtain the network output probability, the network output probability is compared pixel-by-pixel with the ground truth label, and the cross-entropy loss term and the Dice loss term are calculated respectively. The cross-entropy loss term calculates the difference between the predicted probability and the ground truth label for each pixel, and the Dice loss term calculates the overlap between the predicted segmentation region and the ground truth segmentation region. The cross-entropy loss term and the Dice loss term are weighted and summed according to a preset weight ratio to obtain the overall loss value of the current training batch. The weight ratio is determined through cross-validation. Optionally, multiple hyperparameter settings are involved in the training process, see Table 1.

[0093] Table 1: Training Parameter Settings Table

[0094] Parameter name Example value Batch size 8 Initial learning rate 0.001 Loss weight ω 0.7 Optimizer Adaptive moment estimation algorithm Number of training rounds 100

[0095] Automatic differentiation (ADE) is used to calculate the gradient of the overall loss value relative to the weight parameters of each layer in the convolutional neural network (CNN) model. ADE tracks the derivatives of all operations based on the computation graph and uses an adaptive moment estimation (IME) algorithm to optimize the gradient, dynamically adjusting the learning rate of each weight parameter. The IEM maintains the first and second moment estimates for each parameter. All weight parameters of the CNN model are updated according to the calculated gradient direction, completing one training iteration. The training process iterates until the model converges or reaches the preset number of training epochs. The overall loss function can be understood to have the following form:

[0096]

[0097] in: This represents the total loss value. Represents the cross-entropy loss term. This represents the Dice coefficient loss term. The weighting coefficients that balance the two losses have values ​​between zero and one. In some embodiments, the cross-entropy loss term... The calculation formula is the sum of the predicted probability of each pixel and the cross-entropy of the true label, with the Dice coefficient loss term. The calculation formula is 1 minus the Dice similarity coefficient between the predicted region and the real region. Optionally, the preparation of the training dataset needs to ensure the diversity and representativeness of the image samples, covering remote sensing images of different seasons, different lighting conditions, and different water body types. The real annotation map needs to be generated through manual visual interpretation or high-precision automatic extraction methods. It can be understood that when skip connections fuse features, the feature map of the encoder part may be adjusted to match the number of channels of the decoder part through a 1x1 convolutional layer before being passed, to ensure that the stitching operation is feasible.

[0098] See Figure 4 The study reveals the core performance characteristics of the multi-scale pyramid construction stage, specifically: Layer 0 (the original image) maintains 100% resolution, corresponding to a processing time of approximately 20 seconds; as the layers increase (from Layer 1 to Layer 4), the spatial resolution of the image is gradually reduced through Gaussian smoothing and downsampling operations. The resolution of Layer 1 drops to approximately 50%, and the processing time is shortened to approximately 12 seconds; Layer 2 has a resolution of approximately 25%, and a processing time of approximately 8 seconds; Layer 3 has a resolution of approximately 12%, and a processing time of approximately 5 seconds; and Layer 4 has a resolution of approximately 6%, and a processing time of approximately 3 seconds. This pattern of change verifies the hierarchical design logic of the image pyramid: high-resolution layers (such as Layer 0) retain all the detailed information of the original image but require the longest processing time; low-resolution layers (such as Layer 4) achieve a significant improvement in processing efficiency by sacrificing spatial details, providing a performance basis for subsequent multi-scale feature extraction and target layer selection.

[0099] In one embodiment of the present invention, the Otsu's method is used to calculate an adaptive segmentation threshold for the initial water body segmentation probability map. Otsu's method iterates through all possible probability thresholds between zero and one, dividing image pixels into water and non-water classes, and calculating the inter-class variance between the two classes. The probability value with the largest inter-class variance is selected as the adaptive segmentation threshold. The calculated adaptive segmentation threshold is used to binarize the initial water body segmentation probability map. Pixels with probability values ​​greater than the adaptive segmentation threshold are classified as water and assigned a value of one, while pixels with probability values ​​less than or equal to the adaptive segmentation threshold are classified as non-water and assigned a value of zero, resulting in a preliminary binary segmentation map. In some embodiments, a morphological opening operation is performed on the binary segmentation map. The morphological opening operation is a composite operation of erosion followed by dilation. The erosion operation uses a 3x3 rectangular structuring element, and the dilation operation uses the same 3x3 rectangular structuring element to eliminate small noise points and smooth water body boundaries. Optionally, connected component analysis is performed on the morphologically processed binary segmentation image. The connected component analysis uses an eight-neighborhood decision rule, marking all connected pixel regions in the image and calculating the area of ​​each connected region. Isolated regions with areas smaller than a preset minimum water body area are removed. The preset minimum water body area is set based on the actual application scenario, for example, corresponding to an area of ​​500 square meters on the ground. The processed binary image is output as the final river and lake water segmentation result image. The segmentation result image is a single-band image containing only zero and one values, where regions with a pixel value of one represent the segmented river and lake water areas.

[0100] In some embodiments, the method based on local statistical characteristics of the image calculates an adaptive segmentation threshold for the initial water body segmentation probability map. This method divides the image into multiple non-overlapping local sub-blocks, and calculates a separate segmentation threshold within each sub-block, for example, using a linear combination of the average and standard deviation of the pixel probability values ​​within the sub-block. The thresholds of all sub-blocks are interpolated to form an adaptive threshold surface with the same size as the original probability map. Each pixel is then judged using its corresponding local threshold. It can be understood that the maximum inter-class variance method aims to find an optimal threshold that maximizes the separation between the two classes of pixels after segmentation. Its inter-class variance calculation formula is:

[0101]

[0102] in: Is the threshold value Inter-class variance over time The probability value is less than or equal to The proportion of pixels in the entire image. It is the average probability value of this type of pixel. Is the probability value greater than The pixel ratio, This is the average probability value of this type of pixel. Optionally, the area calculation in connected component analysis is converted based on the spatial resolution of the image. For example, for an image with a spatial resolution of ten meters, one pixel represents a surface area of ​​one hundred square meters, and a connected component with five pixels represents five hundred square meters. It can be understood that the erosion operation in morphological opening can eliminate small foreground noise points, but may cause the water body region boundary to shrink inward. The subsequent dilation operation aims to restore the approximate shape of the main water body region, but cannot restore the small structures that have been completely eroded, thus achieving the effect of smoothing the boundary and removing small noise points.

[0103] See Figure 5 This study presents the changing trends in segmentation accuracy and processing time as the spatial resolution increases from 2.5 m / pixel to 20.0 m / pixel. Quantitative results show a significant linear decrease in segmentation accuracy as the spatial resolution decreases (the area represented by each pixel increases): at a high resolution of 2.5 m / pixel, the segmentation accuracy reaches approximately 0.97, while at 20.0 m / pixel, the accuracy is only 0.82. This phenomenon stems from the fact that high-resolution images contain richer details of water body boundaries and small water features, while low-resolution images are prone to blurring of water body pixels with surrounding land features, leading to segmentation ambiguity. Simultaneously, processing time also decreases synchronously with resolution: approximately 12 seconds at 2.5 m / pixel and only about 2 seconds at 20.0 m / pixel. This is because low-resolution images have fewer pixels, significantly reducing the computational load for forward inference and post-processing. This study reveals the trade-off between accuracy and efficiency in water segmentation tasks, providing a quantitative basis for selecting the optimal spatial resolution based on scenario requirements in practical applications.

[0104] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for segmenting river and lake remote sensing images based on convolutional neural networks, characterized in that, Includes the following steps: Acquire multi-source remote sensing image data, and perform radiometric correction and geometric registration processing on the multi-source remote sensing image data to generate a standardized remote sensing image sequence; Multi-scale pyramid construction is performed on the standardized remote sensing image sequence to generate image pyramid hierarchical data containing different spatial resolutions; The target level is selected from the image pyramid level data, and the water body region is initially extracted from the target level image based on the water body spectral feature library to generate a candidate water body mask; A convolutional neural network model with an encoder-decoder structure is constructed. The encoder part uses dilated convolutional layers to extract features at multiple scales. The decoder part uses progressive upsampling and skip connections to fuse deep semantic features and shallow detail features. The candidate water body mask is stitched together with the corresponding target layer image to form enhanced neural network input data; The enhanced neural network input data is fed into the convolutional neural network model for end-to-end training, and the network parameters are optimized by minimizing the difference between the segmentation results and the real labels. The optimized convolutional neural network model is used to perform forward inference on the new remote sensing image and output the initial water body segmentation probability map. Adaptive threshold segmentation and morphological post-processing are performed on the initial water body segmentation probability map to generate a river and lake water area segmentation result map.

2. The river and lake remote sensing image segmentation method based on convolutional neural networks as described in claim 1, characterized in that, The step of performing radiometric correction and geometric registration on the multi-source remote sensing image data to generate a standardized remote sensing image sequence includes: Read raw remote sensing image data and its corresponding metadata information acquired by different sensors; Based on the radiometric calibration parameters in the metadata information, the original remote sensing image data is radiometrically calibrated to convert the pixel brightness values ​​into surface reflectance. Atmospheric correction models were used to eliminate atmospheric effects on the radiometrically calibrated images to obtain true surface reflectance data. A remote sensing image with high geometric accuracy is selected as a reference benchmark, and the control point features in the reference benchmark image are extracted. Control point matching and geometric transformation are performed on the remaining remote sensing images to achieve spatial alignment of all images with the reference image. The geometrically registered images are resampled to unify the spatial resolution and pixel size, forming the standardized remote sensing image sequence.

3. The river and lake remote sensing image segmentation method based on convolutional neural networks as described in claim 2, characterized in that, The process of constructing a multi-scale pyramid from the standardized remote sensing image sequence to generate image pyramid hierarchy data containing different spatial resolutions includes: Set the number of pyramid levels and the downsampling factor of each level relative to the original image; Starting with the original resolution image in the standardized remote sensing image sequence, it serves as the bottom layer of the pyramid, namely the zeroth layer; The image at the current level is smoothed using a Gaussian convolution kernel, and then downsampled at a specified rate to generate the image at the next higher level. Repeat the smoothing and downsampling process until image data of all preset levels are generated; All levels of image data are organized according to resolution from high to low to construct the image pyramid hierarchical data.

4. The river and lake remote sensing image segmentation method based on convolutional neural networks as described in claim 3, characterized in that, The preliminary water body region extraction based on the water body spectral feature library to generate candidate water body masks includes: Load a predefined water body spectral feature library, which contains the spectral reflectance curves and statistical characteristics of typical water bodies in different spectral bands; Calculate the spectral feature vector of each pixel in the target layer image; Calculate the similarity measure between the spectral feature vector of each pixel and the standard water body spectral features in the water body spectral feature library; A similarity threshold is set, and pixels with a similarity metric higher than the threshold are marked as potential water body pixels; Perform region growing operations on the marked potential water body pixels to merge spatially adjacent and spectrally similar pixels into connected regions. Remove excessively small noisy connected regions to generate the candidate water body mask.

5. The river and lake remote sensing image segmentation method based on convolutional neural networks as described in claim 1, characterized in that, The construction of the convolutional neural network model with an encoder-decoder structure includes: The encoder is designed to consist of four downsampling stages. Each downsampling stage contains two convolutional layers and one max pooling layer. The convolutional layers use a 3x3 convolutional kernel and a modified linear unit activation function. The max pooling layer uses a 2x2 pooling window to halve the feature map size. Four parallel dilated convolutional layers are introduced after the last downsampling stage of the encoder section. The four parallel dilated convolutional layers use different dilation coefficients to extract multi-scale contextual information. The decoder part is designed to consist of four upsampling stages. Each upsampling stage uses bilinear interpolation to double the size of the feature map. Then, the upsampling results are concatenated with the feature maps of the corresponding stages in the encoder part by channel dimension. Two convolutional layers are set after each upsampling stage in the decoder section, and the convolutional layers use a one-to-one convolutional kernel to gradually reduce the number of channels to the number of target categories; At the end of the decoder section, a pixel-level classification layer is used, which outputs the probability value of each pixel belonging to the water body category through an activation function.

6. The river and lake remote sensing image segmentation method based on convolutional neural networks as described in claim 4, characterized in that, The step of concatenating the candidate water body mask with the corresponding target layer image to form enhanced neural network input data includes: Adjust the candidate water body mask to the same size as the target layer image; The candidate water body mask is used as additional channel data and superimposed on the original multispectral band data of the target layer image; The spliced ​​multi-channel data is normalized to unify the numerical range of each channel to the interval between zero and one. The normalized multi-channel data is organized into the input data format required by the convolutional neural network model to form the enhanced neural network input data.

7. The river and lake remote sensing image segmentation method based on convolutional neural networks as described in claim 6, characterized in that, The step of inputting the enhanced neural network input data into the convolutional neural network model for end-to-end training includes: Prepare a training dataset containing a large number of labeled river and lake water areas; The enhanced neural network input data is input to the encoder part, which extracts image features step by step through stacked convolutional and pooling layers, and uses dilated convolution to expand the receptive field to capture multi-scale contextual information. The decoder receives the high-level feature map output by the encoder and gradually restores the spatial resolution of the feature map through deconvolution layers or upsampling layers. During the upsampling process of the decoder part, the shallow feature maps and deep feature maps of the corresponding levels of the encoder part are fused through skip connections to combine detailed information and semantic information. The final output is a feature map of the same size as the input image, which is then processed by an activation function to obtain the probability that each pixel belongs to the water body category. The error between the network output probability and the true label is calculated using a loss function, and the weight parameters of the convolutional neural network model are updated using the backpropagation algorithm. The iteration process continues until the model converges or reaches the preset number of training rounds.

8. The river and lake remote sensing image segmentation method based on convolutional neural networks as described in claim 7, characterized in that, The step of calculating the error between the network output probability and the true label using a loss function, and updating the weight parameters of the convolutional neural network model using a backpropagation algorithm, includes: The weighted sum of the cross-entropy loss function and the Dice coefficient loss function is used as the overall loss function, where the cross-entropy loss function is used to measure the pixel-level classification accuracy, and the Dice coefficient loss function is used to optimize the segmentation integrity of the water body region. After calculating the network output probability in each forward propagation, the network output probability is compared with the real annotation pixel by pixel, and the cross-entropy loss term and Dice loss term are calculated respectively. The cross-entropy loss term and the Dice loss term are weighted and summed according to the preset weight ratio to obtain the overall loss value of the current training batch. The gradient of the overall loss value with respect to the weight parameters of each layer of the convolutional neural network model is calculated using automatic differentiation techniques. The gradient is optimized using an adaptive moment estimation algorithm, and the learning rate of each weight parameter is dynamically adjusted. Update all weight parameters of the convolutional neural network model according to the calculated gradient direction to complete one training iteration.

9. The river and lake remote sensing image segmentation method based on convolutional neural networks as described in claim 7, characterized in that, The process of performing adaptive threshold segmentation and morphological post-processing on the initial water body segmentation probability map to generate a river and lake water area segmentation result map includes: An adaptive segmentation threshold is calculated for the initial water body segmentation probability map using either the maximum inter-class variance method or a method based on local image statistical characteristics. The calculated adaptive segmentation threshold is used to binarize the probability map. Pixels with probability values ​​greater than the threshold are classified as water bodies, and those without are classified as non-water bodies, thus obtaining a binary segmentation map. A morphological opening operation is performed on the binary segmentation image to eliminate small noise points and smooth water body boundaries; Connectivity analysis is performed on the morphologically processed binary segmentation map to remove isolated regions with an area smaller than the preset minimum water body area. The processed binary image is output as the final segmentation result image of the river and lake water area.

10. A river and lake remote sensing image segmentation system based on convolutional neural networks, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the river and lake remote sensing image segmentation method based on convolutional neural networks as described in any one of claims 1 to 9.

Images