River channel ice condition identification method based on optical-SAR fusion and adaptive segmentation

Abstract

The invention relates to a riverway ice condition identification method based on optics-SAR fusion and adaptive segmentation, and belongs to the technical field of remote sensing image processing and application. Riverway ice condition features are extracted from the optical remote sensing image, and a Ka-SAR feature map is extracted from the Ka-SAR image by adopting an improved high-resolution network; carrying out multi-modal and multi-scale feature fusion on the extracted features, carrying out scale specificity feature extraction by adopting a Gaussian pyramid, and then carrying out weighted fusion on the river ice condition features and the Ka-SAR features on each scale based on a scale specificity weight distribution principle; aggregating the multi-scale fusion features into a final fusion feature map by adopting a bottom-up pyramid reconstruction strategy; improved Kuan filtering is used to optimize the fused feature map, and adaptive threshold segmentation is used to realize ice surface and non-ice surface binary classification in the feature map. The method can achieve the precise segmentation of the ice condition region, and improves the recognition precision of the thin ice region.

Description

TECHNICAL FIELD

[0001] The present application relates to a river ice condition recognition method based on optical-SAR fusion and adaptive segmentation, belonging to the technical field of remote sensing image processing and application. BACKGROUND

[0002] The Yellow River is an important water resource in northern China, and its winter river ice condition monitoring is crucial for ice flood disaster prevention, water conservancy regulation, and navigation safety. Traditional manual patrol methods are inefficient, risky, and difficult to achieve large-scale synchronous observation. In recent years, remote sensing technology, especially optical and synthetic aperture radar (SAR) remote sensing, has become a major technical means for large-scale and efficient monitoring of river ice conditions.

[0003] Currently, river ice condition recognition methods based on remote sensing images mainly rely on a single data source. Optical remote sensing methods: mainly use the reflectivity difference of ice and water in different spectral bands (such as visible light and near-infrared light) to recognize by constructing spectral indices such as normalized difference snow index (NDSI) and normalized difference water index (NDWI). This method has clear physical meaning and simple calculation. However, optical remote sensing is severely limited by weather conditions, and data is severely missing under the common cloud, fog, and haze cover in the winter of the Yellow River. It cannot be observed at all times, which greatly limits its business application in the critical ice flood period. SAR remote sensing methods: existing methods mostly use C-band or L-band SAR data to distinguish ice and water through threshold methods, classifiers, or image segmentation algorithms. However, the spatial resolution of traditional SAR bands is relatively low, and the ability to capture the edge details of small waterways, broken ice blocks, and thin ice is insufficient. In recent years, although some studies have attempted to fuse optical and SAR data to improve recognition accuracy, there are usually the following problems: (1) The fusion level is shallow, mostly staying at the pixel level or decision level fusion, and failing to fully utilize the complementary advantages of multi-source data in the feature level; (2) Lack of specific adaptation to emerging Ka-band SAR images. Ka-band SAR (hereinafter referred to as Ka-SAR image) has extremely high spatial resolution and rich texture details, and is sensitive to ice surface microstructure. However, its scattering characteristics differ significantly from long-wave SAR, and the noise model is more complex. Direct application of C / L-band-based design schemes cannot fully utilize its performance potential; (3) In the feature processing and segmentation stage, traditional filtering methods tend to blur the fragile ice-water boundary when suppressing noise, and fixed global segmentation thresholds cannot adapt to the changes in gray scale distribution within the image and between different time phases, resulting in low thin ice recognition accuracy and inaccurate boundary positioning. SUMMARY

[0004] The present application aims to overcome the deficiencies of the prior art and provide a river ice condition recognition method based on optical-SAR fusion and adaptive segmentation, which can be more suitable for the application of Ka-band SAR images while performing adaptive segmentation and recognition of the Yellow River ice condition by fusing the features of optical and SAR images.

[0005] The technical scheme adopted by the present application is: The river ice condition recognition method based on optical-SAR fusion and adaptive segmentation comprises the following steps: S1. Preprocess and image register the Ka-SAR image and optical remote sensing image data; S2. Calculate the NDUWI (normalized difference unfrozen water index) of the optical remote sensing image to extract the river ice condition features, and for the Ka-SAR image, use the improved high-resolution network HRNet to perform multi-stage feature extraction first, then enhance it through the attention enhancement module, and finally perform multi-scale fusion to obtain the Ka-SAR feature map; S3. Perform multi-modal and multi-scale feature fusion on the extracted river ice condition features and Ka-SAR features, normalize the features first, then use the Gaussian pyramid to perform multi-scale feature analysis, use different feature extraction strategies at different scales to perform scale-specific feature extraction, then design the fusion weights of the river ice condition features and the Ka-SAR features at each scale based on the scale-specific weight allocation principle, for each scale, fuse the optical features and the Ka-SAR features according to the weights, and then use the bottom-up pyramid reconstruction strategy to aggregate the multi-scale fusion features into the final fusion feature map; S4. Use the improved Kuan filter to optimize the fusion feature map, and then use adaptive threshold segmentation to realize the binary classification of ice and non-ice in the feature map.

[0006] In the above method, the calculation formula of NDUWI in step S2 is as follows: , In the formula, is the green band reflectivity, is the blue band reflectivity.

[0007] In the improved high-resolution network HRNet, during multi-stage feature extraction, subsequent stages have multiple parallel branches that process features at different resolutions, and information exchange is achieved through repeated multi-scale fusion. The attention enhancement module enhances the network's ability to focus on important features, improving its perception of key ice conditions. After receiving a high-resolution feature map from a parallel branch of HRNet, the attention enhancement module generates channel attention weights using global average pooling and a lightweight multilayer perceptron. Global pooling is then performed along the channel dimension, and spatial attention weights are further refined through convolutional layers. Finally, the channel and spatial attention weights are fused to obtain a comprehensive attention map.

[0008] In step S3, scale-specific feature extraction selects the following different combinations of extraction strategies at each scale: edge enhancement, texture analysis, region feature extraction, structural pattern analysis, statistical feature extraction, distribution pattern analysis, contextual feature analysis, and semantic pattern analysis.

[0009] The multi-scale fusion feature aggregation process using a bottom-up pyramid reconstruction strategy is as follows: a) Initialize starting from the coarsest scale. The coarsest scale (scale 4) features are directly used as the initial aggregation features: , b) Stepwise upsampling and feature fusion Bilinear interpolation is used for upsampling by a factor of 2 (denoted as ). The upsampled coarse features are then weighted and added to the current scale features: Level 1 Fusion , Second-level fusion , c) Final fusion (generating output feature maps) , Final feature map It integrates information from all four scales. In the formula Indicates the first Layer aggregation characteristics, For only containing the first The fusion results of NDUWI and Ka-SAR features within the scale. For the final multi-scale fusion output, , , The residual connection weights control the contribution of features at different scales.

[0010] The improved Kuan filter in step S4 optimizes the feature map as follows: a). A boundary protection mechanism is introduced, An effective data mask is introduced to prevent the spread of invalid data during the filtering process: ; I Ffinal represents the feature value in the local window; b). Perform morphological constraint local statistics, When calculating the local statistics, only the effective data area is considered: , , where is the mean value in the local window, is the variance in the local window, represents the local window centered on ; represents the pixel position currently processed, i.e. the center point coordinate, (i, j) represents the pixel position traversed in the window, i.e. the local coordinate; c). Calculate the adaptive weight, , where is the noise variance estimate, is a very small constant to prevent division by zero errors; d). Calculate the filtered feature, the formula is as follows: , where is the mean value in the local window, is the feature value in the local window, is the adaptive weight coefficient.

[0011] The adaptive threshold segmentation process in step S4 is as follows: a). Bimodal analysis based on Gaussian mixture model, First, calculate the histogram of the filtered image The formula is as follows: , where is the number of histogram intervals, is the indicator function, , are the maximum and minimum pixel values in , respectively; Then, Gaussian filtering is used to smooth the histogram: , In the formula The standard deviation is Gaussian kernel, This represents the interval index of the histogram. This represents the offset relative to the current histogram interval b; Assuming the histogram consists of two Gaussian distributions, corresponding to ice and water respectively, the basic formula for the Gaussian mixture model is: , In the formula Indicates the first The weight coefficients in the mixture model are distributed in a Gaussian manner. For the first The probability density function of a Gaussian distribution. Indicates the first The mean parameter of a Gaussian distribution, Indicates the first The standard deviation of a Gaussian distribution; b) Determining the optimal threshold. The intersection of two Gaussian distributions is the theoretically optimal threshold. , The solution is: , In the formula , Let represent the mean parameters of the first and second Gaussian distributions, respectively. , These represent the standard deviations of the first and second Gaussian distributions, respectively. c) Finally, the feature image is segmented based on a threshold.

[0012] The beneficial effects of this invention are: (1) The present invention designs a multi-scale feature pyramid fusion strategy to extract and fuse ice condition features at different spatial scales, taking into account both local details and global distribution, and effectively improving the model’s ability to capture ice condition features at different scales. (2) An improved Kuan filtering method was introduced, which combined boundary protection mechanism, local statistics of morphological constraints and adaptive weight optimization to effectively maintain the integrity of ice-water boundary while suppressing noise and improve the recognition accuracy of thin ice area. (3) An adaptive threshold segmentation method based on Gaussian mixture model is proposed. Through bimodal analysis and theoretical optimal threshold solution, accurate segmentation of ice condition area is achieved, overcoming the limitations of traditional fixed threshold method. Attached Figure Description

[0013] Figure 1Flow chart of the method of the present application; Figure 2 Improved high-resolution network HRNet structure diagram of the present application; Figure 3 Flow chart of multi-modal multi-scale feature fusion of the present application; Figure 4 Optical and Ka-SAR remote sensing images in the study area of the embodiment of the present application; a. optical remote sensing image of the study area, b. Ka-SAR remote sensing image of the study area; Figure 5 NDUWI and Ka-SAR feature maps in the study area of the embodiment of the present application; a. NDUWI feature map of the study area, b. Ka-SAR feature map of the study area; Figure 6 Feature fusion and improved Kuan filtering results in the study area of the embodiment of the present application; Figure 7 Histogram and histogram smoothing results after feature filtering of the embodiment of the present application; Figure 8 River ice condition range extraction results of the embodiment of the present application. DETAILED DESCRIPTION

[0014] The present application will be further described below in conjunction with embodiments.

[0015] Embodiment 1: River ice condition recognition method based on optical-SAR fusion and adaptive segmentation, comprising the following steps: S1. Preprocessing and image registration of Ka-SAR image and optical remote sensing image data: First, perform radiation calibration and multi-view processing on the Ka-band SAR image, and perform radiation calibration, atmospheric correction, and cloud mask generation on the optical image. Then, the sub-pixel registration transformation model is used to register the Ka-SAR image and the corresponding optical image data, and ensure that the row and column numbers of the two images are consistent. The transformation model is as follows: , In the formula, 、 is the coordinate of the registration target image, 、 is the coordinate of the image to be registered, 、 is the translation parameter.

[0016] S2. Calculate NDUWI (Normalized Difference Unfrozen Water Index) on optical remote sensing images to extract river ice features. For Ka-SAR images, use the improved high-resolution network HRNet to perform multi-stage feature extraction, then enhance it through the attention enhancement module, and finally perform multi-scale fusion to obtain the Ka-SAR feature map: (1) For optical remote sensing images, calculate NDUWI (Normalized Difference Unfrozen Water Index). This index uses the blue and green bands in optical remote sensing images to construct a normalized difference index to extract river ice features. The formula is as follows: , wherein is the green band reflectance, is the blue band reflectance.

[0017] (2) For Ka-band SAR images, use the improved high-resolution network HRNet (High-Resolution Network) for feature extraction. Add an attention enhancement module. The core idea is to maintain a high-resolution representation throughout the process, rather than using the traditional network method of downsampling and then upsampling. HRNet connects high-resolution and low-resolution subnetworks in parallel, which can maintain rich feature information and detailed information. This cross-resolution fusion is repeated at each stage (usually 4 stages), forming repeated and intensive multi-scale interactions. A 4-branch HRNet will perform at least 3 cross-resolution fusions at each stage.

[0018] S3. Perform multi-modal and multi-scale feature fusion on the extracted river ice features and Ka-SAR features. First, normalize the features, then use the Gaussian pyramid for multi-scale feature analysis, and use different feature extraction strategies at different scales for scale-specific feature extraction. Then, based on the scale-specific weight allocation principle, design the fusion weights of river ice features and Ka-SAR features at each scale. Finally, use a bottom-up pyramid reconstruction strategy to aggregate the multi-scale fusion features into the final feature map: (1) After obtaining the NDUWI extracted from the optical image and the features of the Ka-SAR image, normalize them respectively. The formula is as follows: , , wherein , denote the normalized results of NDUWI features and Ka-SAR features, , are the mean and standard deviation of NDUWI features, , The mean and standard deviation of the Ka-SAR feature, respectively.

[0019] (2) Multi-scale feature analysis is realized by using Gaussian pyramid structure. Four feature layers with different scales are constructed to capture the performance characteristics of ice conditions at different spatial scales. First, Gaussian smoothing is performed on the input feature map to eliminate noise and prepare for downsampling. The Gaussian kernel function is defined as: , where is the standard deviation of the Gaussian kernel, , respectively.

[0020] Then, the iterative downsampling method is used to construct the Gaussian pyramid. The first layer of the pyramid image is generated by the following formula: , where represents a 2x downsampling operation, and the size of the downsampled image is: , where corresponds to the height of the matrix, corresponds to the width of the matrix.

[0021] (3) Scale-specific feature extraction. Differentiated feature extraction strategies are used at different scales to fully exploit the advantages of each scale. For scale 1 feature extraction (original scale): , , where is the edge enhancement factor, is the texture analysis operator, is the 0th layer NDUWI gold pyramid image, is the 0th layer Ka-SAR feature pyramid image; For scale 2 feature extraction (medium scale): , , where is the region feature extraction operator, is the structural pattern analysis operator, is the 1st layer NDUWI gold pyramid image, is the 1st layer Ka-SAR feature pyramid image; For scale 3 feature extraction (coarse scale): , , where denotes the statistical feature operator, denotes the distribution pattern analysis operator, is the 2nd layer NDUWI pyramid image, is the 2nd layer Ka-SAR feature pyramid image; Scale 4 feature extraction (the coarsest scale): , , where denotes the contextual feature operator, denotes the semantic pattern analysis operator, is the 3rd layer NDUWI pyramid image, is the 3rd layer Ka-SAR feature pyramid image.

[0022] (4) Based on the scale-specific weight assignment principle, the fusion weights of optical NDUWI and Ka-SAR features at each scale are designed: , , The weight design satisfies the normalization condition: , where is the feature weight of each layer of the optical NDUWI pyramid image, is the feature weight of each layer of the Ka-SAR pyramid image.

[0023] (5) Pyramid reconstruction and feature aggregation. First, for each scale , the optical and SAR features are fused according to the weight: , Then, a bottom-up pyramid reconstruction strategy is used to aggregate multi-scale features into the final feature map. First, a bilinear interpolation method is used to define the up-sampling operator to achieve 2 times up-sampling, and then starting from the coarsest scale, the features are up-sampled and fused with the next scale features: , , , , where denotes the aggregated feature of the th layer, is the aggregated feature of the The fusion result of the intra-scale NDUWI and the Ka-SAR feature, is the final multi-scale fusion output, , , is the residual connection weight, which controls the contribution degree of different scale features.

[0024] S4. For the fused features, improved Kuan filtering is used to optimize the feature map, so that the adaptive threshold segmentation realizes the binary classification of ice and non-ice in the feature map: The fusion features obtained by the above method have good ice condition feature highlighting ability. In order to further highlight the river ice condition features and realize the significant distinction of the binary classification of ice and non-ice in the feature map, improved Kuan filtering is used to optimize the feature map, so that the adaptive threshold segmentation is more accurate.

[0025] The traditional Kuan filtering is based on local statistical characteristics, aiming to suppress multiplicative noise while preserving image details, and its basic formula is: , In the formula, is the mean value in the local window, is the feature value in the local window, is the adaptive weight coefficient.

[0026] The present application makes three improvements to the traditional Kuan filtering for the specific scene of river ice condition recognition, making it more suitable for feature image processing in this scene: a) Boundary protection mechanism, introducing effective data mask to prevent the spread of invalid data during filtering: ; b) Morphological constraint local statistics, When calculating the local statistics, only the effective data area is considered: , , In the formula, is the mean value in the local window, is the variance in the local window, represents the local window centered on , (x, y) represents the current processed pixel position, i.e. the center point coordinate, (i, j) represents the pixel position in the window, i.e. the local coordinate; c) Improved adaptive weight, , In the formula, is the noise variance estimate, is a very small constant to prevent division by zero error.

[0027] Finally, the optimized feature map is obtained by the improved Kuan filter.

[0028] Adaptive threshold segmentation: (1) Bimodal analysis based on Gaussian mixture model, First, the filtered image Calculate the histogram, the formula is as follows: , In the formula is the number of histogram intervals (usually take 256), is the indicator function, , are the maximum pixel value and minimum pixel value in , respectively.

[0029] Then, the histogram is smoothed by using Gaussian filter: , In the formula is the Gaussian kernel with standard deviation , is the interval index of the histogram, is the offset relative to the current histogram interval .

[0030] Assume that the histogram is composed of two Gaussian distributions, corresponding to ice and water body respectively, and the basic formula of Gaussian mixture model is: , In the formula is the weight coefficient of the th Gaussian distribution in the mixture model, is the probability density function of the th Gaussian distribution, is the mean parameter of the th Gaussian distribution, is the standard deviation of the th Gaussian distribution.

[0031] (2) Optimal threshold determination,The intersection of the two Gaussian distributions is the theoretical optimal threshold: , The solution is: , In the formula , respectively represent mean parameters of the first and second Gaussian distribution, , respectively represent standard deviations of the first and second Gaussian distribution; Finally, the feature image is segmented based on the threshold value, and the river ice region is extracted.

[0032] Select Ka-SAR image and its corresponding optical image of a certain area, select part of the optical and Ka-SAR image as shown in Figure 4 ; according to the method of the present application, the NDUWI of the optical image and the feature map of the Ka-SAR are calculated, as shown in Figure 5 ; After the NDUWI and the Ka-SAR feature map are fused by multi-modal multi-scale feature fusion, the result is obtained by improved Kuan filtering, as shown in Figure 6 .

[0033] For the filtered result, its histogram is calculated, and the histogram is smoothed based on the method of the present application, and the result is as shown in Figure 7 ; finally, based on the method of the present application, the intersection point of the two Gaussian distributions is calculated to obtain the optimal segmentation threshold value of 0.04846762, and the river ice range obtained thereby is as shown in Figure 8 .

Claims

1. A river ice condition identification method based on optical-SAR fusion and adaptive segmentation, characterized by: The steps include the following: S1. Preprocess and register Ka-SAR imagery with optical remote sensing imagery data; S2. Calculate NDUWI for optical remote sensing images and extract river ice condition features. For Ka-SAR images, use the improved high-resolution network HRNet to perform multi-stage feature extraction, then enhance the features through the attention enhancement module, and finally perform multi-scale fusion to obtain Ka-SAR feature maps. S3. Multimodal and multiscale feature fusion is performed on the extracted river ice condition features and Ka-SAR features. First, the features are normalized separately, and then Gaussian pyramid is used for multiscale feature analysis. Different feature extraction strategies are used at different scales to extract scale-specific features. Then, based on the scale-specific weight allocation principle, the fusion weights of river ice condition features and Ka-SAR features at each scale are designed. For each scale, optical features and Ka-SAR features are fused according to the weights. Then, a bottom-up pyramid reconstruction strategy is used to aggregate the multiscale fused features into the final fused feature map. S4. An improved Kuan filter is used to optimize the fused feature map, and then adaptive threshold segmentation is used to achieve binary classification of ice surface and non-ice surface in the feature map.

2. The river ice condition identification method based on optical-SAR fusion and adaptive segmentation according to claim 1, characterized in that, The formula for calculating NDUWI in step S2 is as follows: ， In the formula For green band reflectivity, This refers to the blue band reflectivity.

3. The river ice condition identification method based on optical-SAR fusion and adaptive segmentation according to claim 1, characterized in that, When the improved high-resolution network HRNet performs multi-stage feature extraction, there are multiple parallel branches in the later stages, which process features of different resolutions respectively, and achieve information interaction through repeated multi-scale fusion. After receiving the high-resolution feature map from a parallel branch of HRNet, the attention enhancement module generates channel attention weights through global average pooling and a lightweight multilayer perceptron, then performs global pooling on the channel dimension, and further refines the spatial attention weights through convolutional layers. The channel attention weights and spatial attention weights are then fused to obtain a comprehensive attention map.

4. The river ice condition identification method based on optical-SAR fusion and adaptive segmentation according to claim 1, characterized in that, In step S3, scale-specific feature extraction selects the following different combinations of extraction strategies at each scale: edge enhancement, texture analysis, region feature extraction, structural pattern analysis, statistical feature extraction, distribution pattern analysis, contextual feature analysis, and semantic pattern analysis.

5. The river ice condition identification method based on optical-SAR fusion and adaptive segmentation according to claim 1, characterized in that, Step S3 employs a bottom-up pyramid reconstruction strategy to aggregate multi-scale fusion features as follows: a) Initialize starting from the coarsest scale. The coarsest scale feature is used directly as the initial aggregation feature: ， b) Stepwise upsampling and feature fusion Bilinear interpolation is used for upsampling by a factor of 2, denoted as The upsampled coarse features are then weighted and added to the current scale features: First-level fusion, ， Second-level fusion, ， c) Finally, the feature maps are fused to generate the output feature map. ， Final feature map It integrates information from all four scales. In the formula This represents the aggregated features of the k-th layer. This is a fusion result that includes only NDUWI and Ka-SAR features within the k-th scale. For the final multi-scale fusion output, , , The residual connection weights control the contribution of features at different scales.

6. The river ice condition identification method based on optical-SAR fusion and adaptive segmentation according to claim 1, characterized in that, The improved Kuan filter in step S4 optimizes the feature map as follows: a) Introduce a border protection mechanism. Introducing an effective data mask To prevent the spread of invalid data during the filtering process: ； I Ffinal Represents the feature values ​​within a local window; b) Perform local statistics with morphological constraints. When calculating local statistics, only the valid data area is considered: ， ， In the formula The mean within a local window. The variance within the local window. Indicates The local window centered on the pixel; (x, y) represents the current pixel position being processed, i.e., the coordinates of the center point, and (i, j) represents the pixel position being traversed within the window, i.e., the local coordinates; c) Calculate the adaptive weights. ， In the formula This is the noise variance estimate. It is a very small constant; d) Calculate the filtered features using the following formula: ， In the formula The mean within a local window. These are the feature values ​​within a local window. These are adaptive weighting coefficients.

7. The river ice condition identification method based on optical-SAR fusion and adaptive segmentation according to claim 1, characterized in that, The adaptive threshold segmentation process in step S4 is as follows: a) Bimodal analysis based on Gaussian mixture model, First, the filtered image... The formula for calculating a histogram is as follows: ， In the formula The number of intervals in the histogram. For indicator functions, , They are respectively The maximum and minimum pixel values ​​in the data; The histogram was then smoothed using a Gaussian filter. ， In the formula The standard deviation is Gaussian kernel, This represents the interval index of the histogram. Indicates the interval relative to the current histogram. The offset; Assuming the histogram consists of two Gaussian distributions, corresponding to ice and water respectively, the basic formula for the Gaussian mixture model is: ， In the formula Indicates the first The weight coefficients in the mixture model are distributed in a Gaussian manner. For the first The probability density function of a Gaussian distribution. Indicates the first The mean parameter of a Gaussian distribution, Indicates the first The standard deviation of a Gaussian distribution; b) Determining the optimal threshold The intersection of two Gaussian distributions is the theoretically optimal threshold. ， The solution is: ， In the formula , Let represent the mean parameters of the first and second Gaussian distributions, respectively. , These represent the standard deviations of the first and second Gaussian distributions, respectively. c) Finally, the feature image is segmented based on a threshold.

Images