River ice regime recognition method based on optical-sar fusion and adaptive segmentation

Abstract

The present application relates to a kind of river ice regime identification method based on optical-SAR fusion and adaptive segmentation, belong to remote sensing image processing and application technical field.Extract river ice regime feature to optical remote sensing image, for Ka-SAR image, using improved high-resolution network extracts Ka-SAR feature map;Multi-modal multi-scale feature fusion is carried out to the extracted feature, scale-specific feature extraction is carried out using Gaussian pyramid, then river ice regime feature and Ka-SAR feature are weightedly fused in each scale based on the weight distribution principle of scale-specific, then multi-scale fusion feature is aggregated into final fusion feature map using the bottom-up pyramid reconstruction strategy;Improved Kuan filter is used to optimize fusion feature map, and adaptive threshold segmentation is used to realize binary classification of ice surface and non-ice surface in feature map.The present application can realize accurate segmentation of ice regime area, and improve the recognition accuracy of thin ice area.

Description

Technical Field

[0001] This invention relates to a method for identifying river ice conditions based on optical-SAR fusion and adaptive segmentation, belonging to the field of remote sensing image processing and application technology. Background Technology

[0002] As a vital water resource in northern my country, monitoring the ice conditions of the Yellow River during winter is crucial for flood prevention, water conservancy management, and navigation safety. Traditional manual surveying methods are inefficient, risky, and difficult to implement for large-scale synchronous observation. In recent years, remote sensing technology, especially optical and synthetic aperture radar (SAR) remote sensing, has become the main technical means for large-scale and efficient monitoring of river ice conditions.

[0003] Currently, methods for identifying river ice conditions based on remote sensing imagery mainly rely on a single data source. Optical remote sensing methods primarily utilize the differences in reflectance of ice and water in different spectral bands (such as visible light and near-infrared) to identify them by constructing spectral indices such as the Normalized Differential Snow Index (NDSI) and the Normalized Differential Water Index (NDWI). These methods have clear physical meaning and are computationally simple. However, optical remote sensing is heavily dependent on weather conditions; data is severely lacking under the cloud, fog, and haze commonly seen in the Yellow River during winter, and all-weather observation is impossible, greatly limiting its operational application during critical ice jam seasons. SAR remote sensing methods primarily utilize C-band or L-band SAR data, using thresholding, classifiers, or image segmentation algorithms to distinguish between ice and water. However, traditional SAR bands have relatively low spatial resolution and are insufficient for capturing the edge details of narrow waterways, broken ice blocks, and thin ice. In recent years, although some studies have attempted to fuse optical and SAR data to improve identification accuracy, the following problems typically exist: (1) The fusion level is relatively shallow, mostly staying at the pixel level or decision level, and failing to give full play to the complementary advantages of multi-source data at the feature level; (2) Lack of targeted adaptation to emerging Ka-band SAR images. Ka-band SAR (hereinafter referred to as Ka-SAR images) has extremely high spatial resolution and rich texture details, and is extremely sensitive to ice surface microstructure. However, its scattering characteristics are significantly different from those of long-wavelength SAR, and its noise model is more complex. Directly applying the design scheme based on C / L bands will not fully realize its performance potential. (3) In the feature processing and segmentation stage, traditional filtering methods tend to blur the fragile ice-water boundary when suppressing noise, and the fixed global segmentation threshold is difficult to adapt to the gray-level distribution changes within the image and between different time phases, resulting in low thin ice recognition accuracy and inaccurate boundary positioning. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a river ice condition identification method based on optical-SAR fusion and adaptive segmentation. This method can better adapt to Ka-band SAR image applications while integrating the features of optical and SAR images to perform adaptive segmentation and identification of ice conditions in the Yellow River.

[0005] The technical solution adopted in this invention is as follows: A method for identifying river ice conditions based on optical-SAR fusion and adaptive segmentation includes the following steps: S1. Preprocess and register Ka-SAR imagery with optical remote sensing imagery data; S2. Calculate NDUWI (Normalized Differential Unfrozen Water Index) for optical remote sensing images and extract river ice condition features. For Ka-SAR images, use the improved high-resolution network HRNet to perform multi-stage feature extraction, then enhance the features through the attention enhancement module, and finally perform multi-scale fusion to obtain Ka-SAR feature maps. S3. Multimodal and multiscale feature fusion is performed on the extracted river ice condition features and Ka-SAR features. First, the features are normalized separately, and then Gaussian pyramid is used for multiscale feature analysis. Different feature extraction strategies are used at different scales to extract scale-specific features. Then, based on the scale-specific weight allocation principle, the fusion weights of river ice condition features and Ka-SAR features at each scale are designed. For each scale, optical features and Ka-SAR features are fused according to the weights. Then, a bottom-up pyramid reconstruction strategy is used to aggregate the multiscale fused features into the final fused feature map. S4. An improved Kuan filter is used to optimize the fused feature map, and then adaptive threshold segmentation is used to achieve binary classification of ice surface and non-ice surface in the feature map.

[0006] In the above method, the calculation formula for NDUWI in step S2 is as follows: , In the formula For green band reflectivity, This refers to the blue band reflectivity.

[0007] In the improved high-resolution network HRNet, during multi-stage feature extraction, subsequent stages have multiple parallel branches that process features at different resolutions, and information exchange is achieved through repeated multi-scale fusion. The attention enhancement module enhances the network's ability to focus on important features, improving its perception of key ice conditions. After receiving a high-resolution feature map from a parallel branch of HRNet, the attention enhancement module generates channel attention weights using global average pooling and a lightweight multilayer perceptron. Global pooling is then performed along the channel dimension, and spatial attention weights are further refined through convolutional layers. Finally, the channel and spatial attention weights are fused to obtain a comprehensive attention map.

[0008] In step S3, scale-specific feature extraction selects the following different combinations of extraction strategies at each scale: edge enhancement, texture analysis, region feature extraction, structural pattern analysis, statistical feature extraction, distribution pattern analysis, contextual feature analysis, and semantic pattern analysis.

[0009] The multi-scale fusion feature aggregation process using a bottom-up pyramid reconstruction strategy is as follows: a) Initialize starting from the coarsest scale. The coarsest scale (scale 4) features are directly used as the initial aggregation features: , b) Stepwise upsampling and feature fusion Bilinear interpolation is used for upsampling by a factor of 2 (denoted as ). The upsampled coarse features are then weighted and added to the current scale features: Level 1 Fusion , Second-level fusion , c) Final fusion (generating output feature maps) , Final feature map It integrates information from all four scales. In the formula Indicates the first Layer aggregation characteristics, For only containing the first The fusion results of NDUWI and Ka-SAR features within the scale. For the final multi-scale fusion output, , , The residual connection weights control the contribution of features at different scales.

[0010] The improved Kuan filter in step S4 optimizes the feature map as follows: a) Introduce a border protection mechanism. Introducing an effective data mask To prevent the spread of invalid data during the filtering process: ; I Ffinal Represents the feature values ​​within a local window; b) Perform local statistics with morphological constraints. When calculating local statistics, only the valid data area is considered: , , In the formula The mean within a local window. The variance within the local window. Indicates A local window centered on the user; (i, j) represents the current pixel position being processed, i.e., the coordinates of the center point, while (i, j) represents the pixel position being traversed within the window, i.e., the local coordinates. c) Calculate the adaptive weights. , In the formula This is the noise variance estimate. It is a very small constant to prevent division by zero errors; d) Calculate the filtered features using the following formula: , In the formula The mean within a local window. These are the feature values ​​within a local window. These are adaptive weighting coefficients.

[0011] The adaptive threshold segmentation process in step S4 is as follows: a) Bimodal analysis based on Gaussian mixture model, First, the filtered image... The formula for calculating a histogram is as follows: , In the formula The number of intervals in the histogram. For indicator functions, , They are respectively The maximum and minimum pixel values ​​in the data; The histogram was then smoothed using a Gaussian filter. , In the formula The standard deviation is Gaussian kernel, This represents the interval index of the histogram. This represents the offset relative to the current histogram interval b; Assuming the histogram consists of two Gaussian distributions, corresponding to ice and water respectively, the basic formula for the Gaussian mixture model is: , In the formula Indicates the first The weight coefficients in the mixture model are distributed in a Gaussian manner. For the first The probability density function of a Gaussian distribution. Indicates the first The mean parameter of a Gaussian distribution, Indicates the first The standard deviation of a Gaussian distribution; b) Determining the optimal threshold. The intersection of two Gaussian distributions is the theoretically optimal threshold. , The solution is: , In the formula , Let represent the mean parameters of the first and second Gaussian distributions, respectively. , These represent the standard deviations of the first and second Gaussian distributions, respectively. c) Finally, the feature image is segmented based on a threshold.

[0012] The beneficial effects of this invention are: (1) The present invention designs a multi-scale feature pyramid fusion strategy to extract and fuse ice condition features at different spatial scales, taking into account both local details and global distribution, and effectively improving the model’s ability to capture ice condition features at different scales. (2) An improved Kuan filtering method was introduced, which combined boundary protection mechanism, local statistics of morphological constraints and adaptive weight optimization to effectively maintain the integrity of ice-water boundary while suppressing noise and improve the recognition accuracy of thin ice area. (3) An adaptive threshold segmentation method based on Gaussian mixture model is proposed. Through bimodal analysis and theoretical optimal threshold solution, accurate segmentation of ice condition area is achieved, overcoming the limitations of traditional fixed threshold method. Attached Figure Description

[0013] Figure 1This is a flowchart of the method of the present invention; Figure 2 This is a diagram of the improved high-resolution network HRNet structure of this invention; Figure 3 This is a flowchart of the multimodal and multiscale feature fusion process of the present invention; Figure 4 This is a schematic diagram of optical and Ka-SAR remote sensing images of the study area in an embodiment of the present invention; a. Optical remote sensing image of the study area, b. Ka-SAR remote sensing image of the study area; Figure 5 The following are NDUWI and Ka-SAR feature maps of the study area in this embodiment of the invention: a. NDUWI feature map of the study area; b. Ka-SAR feature map of the study area. Figure 6 This invention presents the results of studying region feature fusion and improved Kuan filtering in an embodiment of the invention. Figure 7 This is a comparison chart of the histogram obtained after feature filtering and the histogram smoothing result in an embodiment of the present invention; Figure 8 This is the result of extracting the ice condition range of the river channel according to an embodiment of the present invention. Detailed Implementation

[0014] The present invention will be further described below with reference to embodiments.

[0015] Example 1: A method for identifying river ice conditions based on optical-SAR fusion and adaptive segmentation, comprising the following steps: S1. Preprocessing and image registration of Ka-SAR and optical remote sensing image data: First, radiometric calibration and multi-look processing are performed on the Ka-band SAR imagery. Simultaneously, radiometric calibration, atmospheric correction, and cloud mask generation are performed on the optical imagery. Then, a sub-pixel registration transformation model is used to register the Ka-SAR imagery with the corresponding optical imagery, ensuring that the number of rows and columns in both images is consistent. The transformation model is as follows: , In the formula , To register the target image coordinates, , The coordinates of the image to be registered. , These are the translation parameters.

[0016] S2. Calculate the NDUWI (Normalized Differential Unfrozen Water Index) from optical remote sensing images to extract river ice characteristics. For Ka-SAR images, an improved high-resolution network HRNet is used for multi-stage feature extraction, followed by enhancement through an attention enhancement module, and finally multi-scale fusion to obtain the Ka-SAR feature map. (1) For optical remote sensing images, calculate NDUWI (Normalized Difference Unfrozen Water Index). This index uses the blue-green bands in the optical remote sensing images to construct a normalized difference index for extracting river ice condition characteristics. The formula is as follows: , In the formula For green band reflectivity, This refers to the blue band reflectivity.

[0017] (2) For Ka-band SAR images, an improved high-resolution network HRNet (High-Resolution Network) is used for feature extraction, with an attention enhancement module added. The core idea is to maintain high-resolution representation throughout the process, rather than downsampling and then upsampling as in traditional networks. HRNet maintains rich feature and detail information by connecting high-resolution and low-resolution sub-networks in parallel. This cross-resolution fusion is repeated in each stage (usually 4 stages), forming a repetitive and dense multi-scale interaction. A 4-branch HRNet will perform cross-resolution fusion at least 3 times in each stage.

[0018] S3. Multimodal and multiscale feature fusion is performed on the extracted river ice condition features and Ka-SAR features. First, the features are normalized separately, and then Gaussian pyramid is used for multiscale feature analysis. Differentiated feature extraction strategies are used to extract scale-specific features at different scales. Then, based on the scale-specific weight allocation principle, the fusion weights of river ice condition features and Ka-SAR features at each scale are designed. Finally, a bottom-up pyramid reconstruction strategy is used to aggregate the multiscale fused features into the final feature map. (1) After obtaining the features of NDUWI and Ka-SAR images extracted from optical images, the features are normalized respectively, as shown in the following formula: , , In the formula , The normalized results of NDUWI and Ka-SAR features are shown below. , Here are the mean and standard deviation of the NDUWI features. , These represent the mean and standard deviation of the Ka-SAR features, respectively.

[0019] (2) A Gaussian pyramid structure is used to achieve multi-scale feature analysis. By constructing four feature layers of different scales, the characteristics of ice conditions at different spatial scales are captured. First, the input feature map is Gaussian smoothed to eliminate noise and prepare for downsampling. The Gaussian kernel function is defined as: , In the formula The standard deviation of the Gaussian kernel. , They represent respectively.

[0020] Subsequently, an iterative downsampling method was used to construct the Gaussian pyramid. Layered pyramid image Generate using the following formula: , In the formula This indicates a 2x downsampling operation, and the size of the downsampled image is: , In the formula Corresponding matrix height, Corresponding matrix width.

[0021] (3) Scale-specific feature extraction. Differentiated feature extraction strategies are adopted at different scales to fully leverage the advantages of each scale. For scale 1 feature extraction (original scale): , , In the formula Indicates the edge enhancement factor. Represents texture analysis operators. This is an image of the NDUWI pyramid at level 0. This is the image of the Ka-SAR feature pyramid at layer 0; For scale 2 feature extraction (intermediate scale): , , In the formula This represents a region feature extraction operator. Represents the structural pattern analysis operator. This is an image of the first layer of the NDUWI pyramid. This is the first layer of the Ka-SAR feature pyramid image; For scale 3 feature extraction (coarse scale): , , In the formula Represents statistical characteristic operators, Represents the distribution pattern analysis operator. This is an image of the second layer of the NDUWI pyramid. This is the second layer Ka-SAR feature pyramid image; Scale 4 Feature Extraction (Coarsest Scale): , , In the formula Represents context feature operators, This represents a semantic pattern analysis operator. This is an image of the third layer of the NDUWI pyramid. This is the image of the third layer of the Ka-SAR feature pyramid.

[0022] (4) Based on the scale-specific weight allocation principle, design the fusion weights of optical NDUWI and Ka-SAR features at each scale: , , The weight design satisfies the normalization condition: , In the formula The image feature weights for each layer of the optical NDUWI pyramid are given. The weights of the features in each pyramid image layer of Ka-SAR are given.

[0023] (5) Pyramid reconstruction and feature aggregation. First, for each scale... Optical and SAR features are fused according to weights: , Subsequently, a bottom-up pyramid reconstruction strategy is adopted to aggregate multi-scale features into the final feature map. First, a bilinear interpolation method is used to define an upsampling operator to achieve a 2x upsampling. Then, starting from the coarsest scale, upsampling is performed level by level and fused with features from the next scale. , , , , In the formula Indicates the first Layer aggregation characteristics, For only containing the first The fusion results of NDUWI and Ka-SAR features within the scale. For the final multi-scale fusion output, , , The residual connection weights control the contribution of features at different scales.

[0024] S4. For the fused features, an improved Kuan filter is used to optimize the feature map, thereby enabling adaptive threshold segmentation to achieve binary classification of ice surface and non-ice surface in the feature map: The fusion features obtained by the above method have a good ability to highlight ice condition features. In order to further highlight the ice condition features of the river channel and achieve significant distinction between the binary categories of ice surface and non-ice surface in the feature map, an improved Kuan filter is used to optimize the feature map, thereby making the adaptive threshold segmentation more accurate.

[0025] Traditional Kuan filtering is based on local statistical properties and aims to suppress multiplicative noise while preserving image details. Its basic formula is: , In the formula The mean within a local window. These are the feature values ​​within a local window. These are adaptive weighting coefficients.

[0026] This invention addresses the specific scenario of river ice condition identification by making three improvements to the traditional Kuan filter, making it more suitable for feature image processing in this scenario: a) Border protection mechanism, Introducing an effective data mask To prevent the spread of invalid data during the filtering process: ; b) Local statistics of morphological constraints When calculating local statistics, only the valid data area is considered: , , In the formula The mean within a local window. The variance within the local window. Indicates The local window is centered, where (x, y) represents the current pixel position being processed, i.e., the coordinates of the center point, and (i, j) represents the pixel position being traversed within the window, i.e., the local coordinates. c) Improved adaptive weights , In the formula This is the noise variance estimate. It is a very small constant to prevent division by zero errors.

[0027] Finally, the improved Kuan filter obtained the optimized feature map.

[0028] Adaptive threshold segmentation: (1) Bimodal analysis based on Gaussian mixture model, First, the filtered image... The formula for calculating a histogram is as follows: , In the formula This is the number of histogram intervals (usually 256). For indicator functions, , They are respectively The maximum and minimum pixel values ​​in the data.

[0029] The histogram was then smoothed using a Gaussian filter. , In the formula The standard deviation is Gaussian kernel, This represents the interval index of the histogram. Indicates the interval relative to the current histogram. The offset.

[0030] Assuming the histogram consists of two Gaussian distributions, corresponding to ice and water respectively, the basic formula for the Gaussian mixture model is: , In the formula Indicates the first The weight coefficients in the mixture model are distributed in a Gaussian manner. For the first The probability density function of a Gaussian distribution. Indicates the first The mean parameter of a Gaussian distribution, Indicates the first The standard deviation of a Gaussian distribution.

[0031] (2) Determining the optimal threshold, The intersection of two Gaussian distributions is the theoretically optimal threshold. , The solution is: , In the formula , Let represent the mean parameters of the first and second Gaussian distributions, respectively. , These represent the standard deviations of the first and second Gaussian distributions, respectively. Finally, the feature image is segmented based on a threshold to extract the ice condition area of ​​the river channel.

[0032] Select a Ka-SAR image of a certain region and its corresponding optical image, and display the optical and Ka-SAR images of a portion of the region as shown below. Figure 4 As shown; according to the method of the present invention, the NDUWI and Ka-SAR feature maps of the optical image are calculated respectively, as follows. Figure 5 ; After fusing NDUWI and Ka-SAR feature maps through multimodal and multi-scale features, and then processing them with an improved Kuan filter, the result is as follows: Figure 6 As shown.

[0033] The filtered result is then used to calculate its histogram, which is then smoothed using the method of this invention. The result is as follows: Figure 7 As shown; finally, based on the method of this invention, the optimal segmentation threshold of 0.04846762 was obtained by calculating the intersection of the two Gaussian distributions, and the resulting river ice condition range is shown in the figure. Figure 8 As shown.

Claims

1. A river ice condition identification method based on optical-SAR fusion and adaptive segmentation, characterized by: The steps include the following: S1. Preprocess and register Ka-SAR imagery with optical remote sensing imagery data; S2. Calculate NDUWI for optical remote sensing images to extract river ice condition features. For Ka-SAR images, an improved high-resolution network HRNet is used for multi-stage feature extraction, followed by enhancement through an attention enhancement module, and finally multi-scale fusion to obtain a Ka-SAR feature map. When the improved high-resolution network HRNet performs multi-stage feature extraction, there are multiple parallel branches in the later stages, which process features of different resolutions respectively, and information interaction is achieved through repeated multi-scale fusion. After receiving the high-resolution feature map from a parallel branch of HRNet, the attention enhancement module generates channel attention weights through global average pooling and a lightweight multilayer perceptron, then performs global pooling in the channel dimension, further refines the spatial attention weights through convolutional layers, and fuses the channel attention weights with the spatial attention weights to obtain a comprehensive attention map. S3. Multimodal and multiscale feature fusion is performed on the extracted river ice condition features and Ka-SAR features. First, the features are normalized separately, and then Gaussian pyramid is used for multiscale feature analysis. Different feature extraction strategies are used at different scales to extract scale-specific features. Then, based on the scale-specific weight allocation principle, the fusion weights of river ice condition features and Ka-SAR features at each scale are designed. For each scale, optical features and Ka-SAR features are fused according to the weights. Then, a bottom-up pyramid reconstruction strategy is used to aggregate the multiscale fused features into the final fused feature map. The multi-scale fusion feature aggregation process using a bottom-up pyramid reconstruction strategy is as follows: a) Initialize starting from the coarsest scale. The coarsest scale feature is used directly as the initial aggregation feature: ， b) Stepwise upsampling and feature fusion Is bilinear interpolation used for 2x upsampling, denoted as... The upsampled coarse features are then weighted and added to the current scale features: First-level fusion, ， Second-level fusion, ， c) Finally, the feature maps are fused to generate the output feature map. ， Final feature map It integrates information from all four scales. In the formula This represents the aggregated features of the k-th layer. This is the fusion result containing only NDUWI and Ka-SAR features within the k-th scale. For the final multi-scale fusion output, , , The residual connection weights control the contribution of features at different scales. S4. An improved Kuan filter is used to optimize the fused feature map, and then an adaptive threshold segmentation is used to achieve binary classification of ice surface and non-ice surface in the feature map; The improved Kuan filter optimizes the feature map as follows: a) Introduce a border protection mechanism. Introducing an effective data mask To prevent the spread of invalid data during the filtering process: ； I Ffinal Represents the feature values ​​within a local window; b) Perform local statistics with morphological constraints. When calculating local statistics, only the valid data area is considered: ， ， In the formula The mean within a local window. The variance within the local window. Indicates The local window centered on the pixel; (x, y) represents the current pixel position being processed, i.e., the coordinates of the center point, and (i, j) represents the pixel position being traversed within the window, i.e., the local coordinates; c) Calculate the adaptive weights. ， In the formula This is the noise variance estimate. It is a very small constant; d) Calculate the filtered features using the following formula: ， In the formula The mean within a local window. These are the feature values ​​within a local window. These are adaptive weighting coefficients.

2. The river ice condition identification method based on optical-SAR fusion and adaptive segmentation according to claim 1, characterized in that, The formula for calculating NDUWI in step S2 is as follows: ， In the formula For green band reflectivity, This refers to the blue band reflectivity.

3. The river ice condition identification method based on optical-SAR fusion and adaptive segmentation according to claim 1, characterized in that, In step S3, scale-specific feature extraction selects the following different combinations of extraction strategies at each scale: edge enhancement, texture analysis, region feature extraction, structural pattern analysis, statistical feature extraction, distribution pattern analysis, contextual feature analysis, and semantic pattern analysis.

4. The river ice condition identification method based on optical-SAR fusion and adaptive segmentation according to claim 1, characterized in that, The adaptive threshold segmentation process in step S4 is as follows: a) Bimodal analysis based on Gaussian mixture model, First, the filtered image... The formula for calculating a histogram is as follows: ， In the formula The number of intervals in the histogram. For indicator functions, , They are respectively The maximum and minimum pixel values ​​in the data; The histogram was then smoothed using a Gaussian filter. ， In the formula The standard deviation is Gaussian kernel, This represents the interval index of the histogram. Indicates the interval relative to the current histogram. The offset; Assuming the histogram consists of two Gaussian distributions, corresponding to ice and water respectively, the basic formula for the Gaussian mixture model is: ， In the formula Indicates the first The weight coefficients in the mixture model are distributed in a Gaussian manner. For the first The probability density function of a Gaussian distribution. Indicates the first The mean parameter of a Gaussian distribution, Indicates the first The standard deviation of a Gaussian distribution; b) Determining the optimal threshold. The intersection of two Gaussian distributions is the theoretically optimal threshold. ， The solution is: ， In the formula , Let represent the mean parameters of the first and second Gaussian distributions, respectively. , These represent the standard deviations of the first and second Gaussian distributions, respectively. c) Finally, the feature image is segmented based on a threshold.

Images