SAR image polar region sea ice detection method and system based on deep learning, and medium

Abstract

The invention provides an SAR image polar region sea ice detection method and system based on deep learning and a medium, the land and the water area are segmented by using a water area detection algorithm based on a DeeplabV3 network, the segmentation precision of the DeeplabV3 network is improved by means of cavity convolution, a spatial pyramid pooling module and a decoder, and the calculation and memory overhead is reduced by using a lightweight MobileNet; then, a sea ice detection algorithm based on a short-term dense cascading STDC network is used for precisely segmenting sea ice and water in a water area, the STDC network reduces the calculation complexity through specific module design and keeps multi-scale information, and meanwhile, multiple optimization means are adopted in decoder design to improve the detection performance; in addition, data is prepared and augmented, and the generalization ability of the model is enhanced. The sea ice detection precision and efficiency are improved, and the method is suitable for resource-limited environments such as on-satellite environments and the like.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a method, system, and medium for detecting polar sea ice in SAR images based on deep learning. Background Technology

[0002] As shipping routes and oil and gas exploration and production move to higher latitudes, sea ice has a significant impact on marine resource development and the shipping industry. In polar regions, the distribution and changes in sea ice have a major influence on global climate, marine ecology, and offshore operations. Synthetic Aperture Radar (SAR), with its all-weather, day-and-night observation capabilities, has become an important means of acquiring information on polar sea ice. However, detecting sea ice from SAR images faces many challenges:

[0003] Sea ice exhibits diverse types and complex radar echo characteristics: it can be categorized into new ice, one-year ice, and multi-year ice, as well as thin ice and thick ice. New ice and thin ice have smooth surfaces and uniform textures, resulting in lower echo intensity; one-year ice has moderate echo intensity and may contain cracks and melt pools; multi-year ice and thick ice have rough surfaces and distinct textures, leading to higher echo intensity. Furthermore, pores and air bubbles within sea ice cause multiple scattering, increasing echo intensity. The significant differences in radar echo characteristics among different types of sea ice further complicate detection.

[0004] Imaging is greatly affected by environmental factors: Weather conditions significantly impact SAR sea ice imaging results. Rain and snowfall attenuate and scatter signals, reducing image clarity. Wind speed and waves decrease the echo contrast between sea ice and the water surface, especially in the edge regions, making it difficult to distinguish between sea ice and seawater. Furthermore, temperature changes significantly affect the electromagnetic properties of sea ice; low temperatures result in stronger radar echoes, while high temperatures cause the sea ice surface to melt, forming melt pools and reducing echo intensity.

[0005] Traditional detection methods have drawbacks: In sea ice detection using on-orbit remote sensing images, the target area occupies a relatively small proportion of the large field-of-view image. Traditional candidate region extraction methods are mostly designed for simple, ideal scenarios, or directly extract sea ice regions indiscriminately from all areas of the entire image. This approach processes a large amount of redundant information, resulting in a large number of false alarms and the possibility of missing useful targets. Furthermore, sea ice and water are relatively dark in SAR images, while land is relatively bright, making direct differentiation difficult.

[0006] Patent application CN119169481A discloses a method for sea ice detection in SAR images, comprising the following steps: S1 acquiring SAR images of a preset target from a SAR image library to obtain a SAR image dataset for sea ice detection; S2 preprocessing the SAR image dataset to obtain a first sea ice mask image set and sea ice feature vectors; S3 training a pre-created support vector machine and classifying and predicting the SAR image dataset based on the trained support vector machine to obtain a second sea ice mask image; S4 randomly dividing the second sea ice mask image into a training set and a validation set according to a preset ratio; S5 inputting the training set into a pre-created HRU-Net model for model training, and confirming whether the trained model has converged based on the constructed model loss function and the validation set to obtain a SAR-affected sea ice detection model; S6 implementing sea ice detection in SAR images based on the support vector machine detection model. This patent uses support vector machines for sea ice detection in SAR images and does not involve the separation of sea ice and seawater.

[0007] Patent application CN108508427A discloses a method, apparatus, and device for sea ice region detection based on navigation radar. The method includes: S1 acquiring multiple pulse echo sequences from radar scanning of the sea surface, and acquiring multiple sampling points based on the multiple pulse echo sequences; S2 acquiring sampling points among the multiple sampling points whose pulse echo intensity is greater than a preset sea ice detection threshold, and using these as sea ice echo sampling points; S3 traversing each sea ice echo sampling point and performing sea ice region detection using a two-dimensional eight-connectivity judgment method. This patent uses shipborne radar for sea ice region detection and does not involve sea ice detection based on spaceborne SAR images.

[0008] Patent application CN117994666A discloses a sea ice recognition method combining residual networks and edge detection, comprising: S1 preprocessing sea ice remote sensing images to obtain ground truth images, resulting in a training set of sea ice images; S2 using the encoder of a Res-UNet model to downsample the training set images, i.e., image feature extraction; S3 using the decoder of a Res-UNet model to map the features output by the Res-UNet model encoder back to the size of the sea ice remote sensing images, and outputting the image recognition result; S4 finally using the Canny operator to detect the sea ice edges output by the Res-UNet model. This patent performs sea ice detection on spaceborne optical images, but does not involve sea ice detection on spaceborne SAR images.

[0009] Patent application CN115144411A discloses a method, apparatus, device, and medium for sea ice detection based on a satellite scatterometer. The method includes: S1 determining the residual value of each observation unit in the target observation area based on the microwave scattering signal from the microwave scatterometer in the target observation area during the current update cycle; S2 obtaining the prior probability distribution of each sea surface type in the target observation area during each update cycle in a preset time period prior to the current update cycle, and generating a sea surface type lookup table for the target observation area; S3 determining the posterior probability corresponding to each sea surface type in the target observation area during the current update cycle based on the residual value of each observation unit and the lookup table, and updating the sea ice coverage product of the target observation area based on the posterior probability corresponding to each sea surface type in each observation unit during the current update cycle. This patent utilizes a satellite scatterometer for sea ice detection and does not involve sea ice detection using high-resolution spaceborne SAR images.

[0010] Patent application CN108957444A describes a method and apparatus for detecting the contour lines of sea ice regions. This method utilizes a shipborne device to acquire omnidirectional information about floating ice or icebergs in sea ice regions. By selecting target sampling points where the spatial uniformity of echo amplitude drastically changes, and taking advantage of the characteristic that sampling points located at the boundary between seawater and sea ice regions exhibit significant spatial uniformity changes in echo amplitude, the contour lines of the sea ice regions are obtained. This allows for the detection of the outer contour lines of sea ice regions, thereby helping ships to rationally plan their routes and avoid navigational risks. This patent utilizes shipborne equipment for sea ice detection and does not involve sea ice detection using high-resolution SAR images from spacecraft.

[0011] Patent document CN201476779U discloses a system for detecting the thickness of polar sea ice. This utility model discloses a system for detecting the thickness of polar sea ice, which has high accuracy and efficiency in detecting sea ice thickness. This invention requires placing the instrument in the sea ice area.

[0012] To address the above issues, there is an urgent need to study sea ice detection technology using high-resolution SAR images in order to maximize the on-orbit value and effectiveness of high-resolution SAR and further improve the efficiency of sea ice detection. Summary of the Invention

[0013] In view of the shortcomings of the prior art, the purpose of this invention is to provide a method, system and medium for detecting polar sea ice in SAR images based on deep learning.

[0014] The deep learning-based SAR image polar sea ice detection method provided by the present invention includes:

[0015] Step 1: A lightweight convolutional neural network architecture with dilated convolutions and a spatial pyramid pooling module with dilated convolutions are used as the encoder part in the deep learning model to introduce multi-scale information. The decoder part in the deep learning model is used to fuse low-level features and high-level features to achieve land and water segmentation.

[0016] Step 2: Cut the SAR grayscale image into data of a preset size, prepare the ground truth image of the artificially marked water area into a marker image with pixels of 0 and 1, and cut it into the same size. Perform rotation, translation and scaling operations on the prepared marker image and grayscale image, wherein the marker image and the corresponding grayscale image are augmented at the same ratio.

[0017] Step 3: Using an encoding and decoding approach, the encoding network is designed with short-term dense cascaded modules and integrated into the encoder-decoder segmentation network architecture. The decoder uses a pre-trained short-term dense cascaded network as the base network and combines the context path encoding context information of the real-time semantic segmentation network to achieve accurate segmentation of sea ice and water.

[0018] Preferably, for the input feature map Where H is the height, W is the width, and C is the number of channels, the calculation process of dilated convolution is expressed as:

[0019]

[0020] in, Let H′, W′, and C′ represent the height, width, and number of channels of the output feature map, respectively. Let i, j, and k be the coordinate indices of the output feature map, and m, n, and p be the coordinate indices inside the convolution kernel. K is the convolution kernel, K is the kernel size, and r is the dilation rate parameter. The size of the receptive field is controlled by adjusting the value of r. When r = 1, the dilated convolution degenerates into a standard convolution. When r > 1, the receptive field expands as r increases.

[0021] For different output strides, the dilatancy parameter of the convolutional layer is set as the ratio of the input image resolution to the output feature map resolution. If the output stride is 8, the dilatancy parameter of the first few convolutional layers is set to 1, 2, 4 in sequence; if the output stride is 16, the dilatancy parameter is set to 1, 1, 2 in sequence.

[0022] Preferably, the spatial pyramid pooling module uses dilated convolutions with different dilation parameters to achieve multi-scale information extraction, and its structure includes a 1×1 convolutional layer, a 3×3 dilated convolutional layer with different dilation parameters, and a global average pooling layer.

[0023] 1×1 convolutional layer: Adjusts the number of channels in the input feature map, and outputs a feature map with dimension C1;

[0024] 3×3 dilated convolutional layers with different dilation parameters: Multiple 3×3 dilated convolutions with different dilation parameters are used to extract features at different scales, and the output feature maps have dimensions of C2, C3, and C4, respectively.

[0025] Global average pooling layer: Compresses the input feature map into a global feature vector, then adjusts the number of channels to C5 through 1×1 convolution, and upsamples to restore the original feature map size;

[0026] Features at different scales are concatenated along the channel dimension, i.e., the feature maps are stacked along the channel dimension to obtain a new feature map.

[0027] Preferably, a deep neural network is constructed using depthwise separable convolution, which decomposes standard convolution into depthwise convolution and pointwise convolution.

[0028] Depthwise convolution: For each channel of the input feature map, an independent convolution kernel is used to perform a convolution operation. The calculation process is as follows:

[0029]

[0030] in, The kernel is a depthwise convolution, and the number of channels in the output feature map is the same as that in the input.

[0031] Pointwise convolution: Adjusting the number of channels in the output of depthwise convolution using 1×1 convolution. The calculation process is as follows:

[0032]

[0033] in, The kernel is a pointwise convolution, and the number of channels in the output feature map becomes C′;

[0034] The h-swish activation function is used, and the formula is:

[0035]

[0036] Where x is the input value; the hyperparameters of the lightweight convolutional neural network architecture are set as follows: initial number of channels 32, number of layers 18, and dilation coefficient 6.

[0037] Preferably, the features obtained from the encoder are upsampled using bilinear interpolation to obtain a feature map of 4 times the size. Low-level features of the corresponding size are then extracted from the encoder, and 1×1 convolutions are used to reduce the dimensionality of these low-level features, increasing the number of channels from C0. low Reduced to C′ low The calculation formula is:

[0038]

[0039] Where X is the original low-level feature map. For dimensionality reduction convolution kernels, C low C′ is the number of channels in the original low-level feature map. low Let Y be the number of channels in the dimensionality-reduced feature map. The low-level features after dimensionality reduction are fused with the high-level features after upsampling by element-wise addition or concatenation to obtain the fused feature map. The fused feature map is then bilinearly interpolated again to restore it to the same size as the original image, resulting in the final sea ice segmentation prediction map.

[0040] Preferably, the short-term dense cascaded module consists of multiple convolutional branches, each using a convolutional kernel of a different size to achieve multi-scale feature extraction. Let the input feature map be... Where H and W are the height and width of the feature map, respectively, and C... in Given the number of input channels, the calculation process for each branch is as follows:

[0041] Branch 1: Use a 1×1 convolution kernel to output feature map X1, calculated as X1 = Conv 1×1 (X), where Conv 1×1 This represents a 1×1 convolution operation with C1 output channels;

[0042] Branch 2: Uses a 3×3 convolution kernel, outputting feature map X2 = Conv 3×3 (X), the number of output channels is C2;

[0043] Branch 3: Achieve the effect of a 5×5 convolution kernel through two cascaded 3×3 convolution kernels, outputting feature map X3 = Conv 3×3 (Conv 3×3 (X)), the number of output channels is C3;

[0044] The output feature maps of the three branches are concatenated along the channel dimension to obtain the output Y = [X1, X2, X3], which has C channels. out =C1+C2+C3;

[0045] The number of parameters P in a short-term dense cascaded module is determined by the input / output channel dimension, and the calculation formula is:

[0046] P = C in ×(C1+3 2 ×C2+3 2 ×3 2 ×C3)

[0047] Among them, C in =32, C1=16, C2=16, C3=16.

[0048] Preferably, the designed short-term dense concatenated modules are integrated into the encoder part of the encoder-decoder segmentation network architecture to form a short-term dense concatenated network. The encoder contains multiple downsampling stages, each of which consists of multiple short-term dense concatenated modules and a max pooling layer. In each downsampling stage, the feature map size is halved and the number of channels is increased through the max pooling layer, thereby extracting more abstract semantic features.

[0049] In the first downsampling stage, the input is the original image. After processing by two short, densely cascaded modules, it passes through a 2×2 max-pooling layer, changing the feature map size from H×W to [the original image size]. The number of channels increases accordingly, gradually realizing multi-level extraction of sea ice and water features.

[0050] Preferably, during the encoding process, feature maps with different downsampling ratios are obtained from different stages of the encoder. Global average pooling is then used to generate global context information for these feature maps with different downsampling ratios. The global average pooling operation is calculated as follows:

[0051]

[0052] Wherein, G is the generated global context vector. This operation compresses the feature map into a vector with global information. Subsequently, a 1×1 convolution operation is performed to adjust the number of channels, and the vector is upsampled to a preset size and fused with other feature maps to enhance the expression of the overall features of sea ice and water.

[0053] The decoder adopts a U-shape structure, which combines features from different levels of the encoding stage with the upsampled features. The specific process is as follows: First, the high-level semantic features output by the encoder are upsampled step by step. The upsampling method is bilinear interpolation. Each upsampling doubles the size of the feature map. At the same time, the feature map is obtained from the corresponding level of the encoding stage. The upsampled feature map is then concatenated with the feature map of the encoding stage in the channel dimension to achieve the fusion of high and low level features.

[0054] By introducing an attention mechanism, the fused features are further processed to enhance features related to sea ice and suppress irrelevant information. The specific calculation process is as follows: The input feature map... Where H ″ W ″ C ″ Given the height, width, and number of channels of the input feature map, a global average pooling process is first used to obtain the global feature vector. Then, the vector is passed through two 1×1 convolutional layers, with the ReLU activation function used in between. The calculation formula is as follows:

[0055] A=σ(W2·ReLU(W1·GAP(F in )))

[0056] Where W1 and W2 are the parameters of a 1×1 convolution kernel, σ is the sigmoid activation function, GAP is the global average pooling operation, and A is the generated attention weight map. The attention weight map is then multiplied element-wise with the input feature map to obtain the refined feature map.

[0057]

[0058] in, This indicates an element-wise multiplication operation, which highlights the characteristics of sea ice regions and improves segmentation accuracy.

[0059] The features processed by the attention mechanism are fused with other relevant features. The fusion method is either element-wise addition or channel concatenation followed by convolution.

[0060] The final feature map's channel count is converted to the number of categories using a 1×1 convolutional layer. Then, a Sigmoid activation function is applied to obtain the probability value of each pixel belonging to sea ice or water, thus generating the final segmentation prediction map. To optimize network training and alleviate the class imbalance problem of sea ice and water in SAR images, binary cross-entropy and Dice loss are jointly used for optimization. The binary cross-entropy loss function measures the difference between the predicted probability and the true label, and its calculation formula is as follows:

[0061]

[0062] Where N is the total number of pixels, y i For real labels, p i To predict probabilities;

[0063] Dice loss is used to measure the degree of overlap between the predicted result and the true label. The calculation formula is as follows:

[0064]

[0065] The joint loss function is: L = αL BCE +(1-α)L Dice , where α is the balance coefficient, and joint optimization enables the network to better learn the characteristics of sea ice and water, thereby improving segmentation performance.

[0066] The deep learning-based SAR image polar sea ice detection system provided by the present invention includes:

[0067] Module M1: It adopts a lightweight convolutional neural network architecture with dilated convolution and a spatial pyramid pooling module with dilated convolution as the encoder part in the deep learning model, introduces multi-scale information, and uses the decoder part in the deep learning model to fuse low-level features and high-level features to achieve land and water segmentation.

[0068] Module M2: Cuts the grayscale image of SAR into data of a preset size, prepares the ground truth image of artificially marked water area into a marker image with pixels of 0 and 1, and cuts it into the same size. Performs rotation, translation and scaling operations on the prepared marker image and grayscale image, wherein the marker image and the corresponding grayscale image are augmented at the same ratio.

[0069] Module M3: It adopts an encoding and decoding approach. The encoding network is designed with short-term dense cascaded modules and integrated into the encoder-decoder segmentation network architecture. The decoder uses a pre-trained short-term dense cascaded network as the base network and combines the context path encoding context information of the real-time semantic segmentation network to achieve accurate segmentation of sea ice and water.

[0070] According to the present invention, a computer-readable storage medium storing a computer program is provided, wherein when the computer program is executed by a processor, the steps of the deep learning-based SAR image polar sea ice detection method are implemented.

[0071] Compared with the prior art, the present invention has the following beneficial effects:

[0072] (1) In view of the difficulty of polar sea ice SAR image detection, this invention designs a two-level deep learning network. First, the DeeplabV3 network is used for water area detection, and then the STDC network is used for sea ice detection, which improves the accuracy and efficiency of sea ice detection.

[0073] (2) Data augmentation technology was used to expand the number of samples, which enhanced the generalization ability of the model and enabled the model to have good detection results in different sea ice scenarios.

[0074] (3) As a lightweight base network, MobileNet reduces computational and memory overhead, making it easy to use in satellite or other resource-constrained environments;

[0075] (4) The STDC network design reduces computational complexity by carefully adjusting the convolution kernel size, while retaining rich multi-scale information and improving the performance of sea ice detection. Attached Figure Description

[0076] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0077] Figure 1 The flowchart shows a deep learning-based SAR image polar sea ice detection method.

[0078] Figure 2 This is a set of SAR satellite-borne test data;

[0079] Figure 3These are the test results for sea ice and water areas, with yellow representing seawater and red representing sea ice;

[0080] Figure 4 This is another example of SAR satellite-borne test data;

[0081] Figure 5 These are the test results for sea ice and water areas, with yellow representing seawater and red representing sea ice. Detailed Implementation

[0082] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0083] Example

[0084] like Figure 1 This invention aims to provide a deep learning-based method for detecting polar sea ice in SAR images, addressing the aforementioned problems in existing technologies. This method achieves accurate detection of polar sea ice by designing a specific deep learning network. The specific implementation process is as follows:

[0085] Step 1: The water area detection algorithm based on DeepLabV3 network uses MobileNet with dilated convolution and the Spatial Pyramid Pooling Module ASPP with dilated convolution as the Encoder to introduce multi-scale information. The Decoder module is used to fuse low-level features and high-level features to achieve land and water segmentation.

[0086] To achieve accurate segmentation of land and water, this invention employs the DeepLab V3 network. This network effectively fuses multi-scale information through a unique Encoder-Decoder structure, improving the accuracy of segmentation boundaries. The Encoder consists of a MobileNet module with dilated convolutions and a Spatial Pyramid Pooling module (ASPP) with dilated convolutions, which can incorporate multi-scale information. The Decoder module further fuses low-level and high-level features, improving the accuracy of segmentation boundaries.

[0087] The dilated convolution module, a core technology of DeepLab models, expands the receptive field without changing the feature map size by inserting holes between elements of the standard convolution kernel, thereby effectively extracting multi-scale information. For the input feature map... (H is the height, W is the width, and C is the number of channels), the calculation process of dilated convolution can be expressed as:

[0088]

[0089] in, To output the feature map, K is the convolution kernel (where K is the kernel size), and r is the dilation rate (the rate parameter). Adjusting the r value controls the receptive field size; for example, when r = 1, dilated convolution degenerates into standard convolution; when r > 1, the receptive field expands as r increases. In this algorithm, the rate value of the convolutional layers is set appropriately for different output strides (defined as the ratio of the input image resolution to the output feature map resolution). Specifically, if the output stride is 8, the rate values ​​for the first few convolutional layers are set to 1, 2, and 4 respectively; if the output stride is 16, the rate values ​​are set to 1, 1, and 2 respectively, to balance computational efficiency and feature extraction capability.

[0090] The Spatial Pyramid Pooling Module (ASPP) utilizes dilated convolutions of different rates to extract information at multiple scales. Its structure includes 1×1 convolutional layers, 3×3 dilated convolutional layers of different rates, and a global average pooling layer.

[0091] The specific process is as follows:

[0092] 1×1 convolutional layer: Adjusts the number of channels in the input feature map to reduce computational complexity, and outputs a feature map with dimension C1.

[0093] 3×3 dilated convolutional layers with different rates: Multiple 3×3 dilated convolutions with different rates (such as r1, r2, r3) are used to extract features at different scales, and the output feature maps have dimensions of C2, C3, and C4, respectively.

[0094] Global average pooling layer: Compresses the input feature map into a global feature vector, then adjusts the number of channels to C5 through 1×1 convolution, and upsamples to restore the original feature map size.

[0095] Finally, the features at different scales are concatenated along the channel dimension to obtain a new feature map.

[0096] MobileNet, as a lightweight base network, uses depthwise separable convolution to build deep neural networks, effectively reducing the number of model parameters and making it suitable for on-board algorithm porting. Its core operation is to decompose standard convolution into depthwise convolution (DW convolution) and pointwise convolution (PW convolution).

[0097] Depthwise convolution (DW convolution): For each channel of the input feature map, an independent convolution kernel is used to perform a convolution operation. The calculation process is as follows:

[0098]

[0099] in, The kernel is a depthwise convolution, and the number of channels in the output feature map is the same as that in the input.

[0100] Pointwise convolution (PW convolution): Adjusting the number of channels in the output of DW convolution using a 1×1 convolution. The calculation process is as follows:

[0101]

[0102] in, The kernel is a pointwise convolution kernel, and the number of channels in the output feature map becomes C′.

[0103] In addition, MobileNet uses the h-swish activation function, the formula of which is:

[0104]

[0105] This function maintains nonlinear characteristics while reducing computational complexity and improving model performance. Meanwhile, the Bneck core module (bottleneck module) further optimizes the network structure and reduces parameter redundancy through a dimensionality reduction and subsequent dimensionality increase approach. In this algorithm, the hyperparameters of the MobileNet network are set to: an initial number of channels of 32, 18 layers, and an expansion coefficient of 6 for the Bneck module, achieving a lightweight design while ensuring accuracy.

[0106] The Decoder module borrows from the encoder-decoder structure. This algorithm introduces a new decoder module, aiming to deeply fuse low-level and high-level features to improve the accuracy of segmentation boundaries. The specific process is as follows: The features obtained from the encoder are upsampled using bilinear interpolation to obtain a feature map four times the size. Low-level features of the corresponding size are obtained from the encoder, and dimensionality reduction is first performed on the low-level features using a 1×1 convolution, reducing the number of channels from C... low Reduced to C′ low The calculation formula is:

[0107]

[0108] Where X is the original low-level feature map. Let Y be the dimensionality-reducing convolution kernel, and Y be the dimensionality-reduced feature map. The dimensionality-reduced low-level features are fused with the upsampled high-level features, which can be done through element-wise addition or concat operations to obtain a fused feature map. The fused feature map is then subjected to bilinear interpolation to restore it to the same size as the original image, resulting in the final sea ice segmentation prediction map.

[0109] Step 2: Water Area Segmentation Data Preparation and Augmentation. To better train the water area detection model constructed in Step 1, the SAR grayscale image is segmented into 1K×1K data points, and the ground truth image of artificially labeled water areas is prepared as a marker image with pixels of 0 and 1, also segmented into 1K×1K data points. Data augmentation operations such as rotation, translation, and scaling are performed on the prepared marker images and grayscale images. The marker images and their corresponding grayscale images are augmented at the same ratio to expand the sample size and improve the model's generalization ability.

[0110] Step 3: To address the problem of similar spectral features and difficulty in distinguishing sea ice and water in SAR images, this invention proposes a sea ice detection algorithm based on a short-term dense cascaded STDC network. This algorithm utilizes an encoding-decoding architecture to achieve accurate segmentation of sea ice and water. The specific details are as follows:

[0111] The encoding network is designed with a short-term dense cascaded module (STDC module) and integrated into the U-net architecture. The decoder uses a pre-trained STDC network as the base network and combines the context path encoding context information of BiSeNet to achieve accurate segmentation of sea ice and water.

[0112] Coding network design:

[0113] 1) Short-Term Dense Concatenation Module (STDC Module): This module optimizes the design of convolutional kernel size to obtain a variable and scalable receptive field while maintaining a small number of parameters, effectively preserving feature information of sea ice and water at different scales. This module mainly consists of multiple convolutional branches, each using convolutional kernels of different sizes to achieve multi-scale feature extraction. Assume the input feature map is... (H and W represent the height and width of the feature map, respectively, C) in (Assuming the number of input channels), the calculation process for each branch within the STDC module is as follows:

[0114] Branch 1: Use a 1×1 convolution kernel to output feature map X1, calculated as X1 = Conv 1×1 (X), where Conv 1×1 This represents a 1×1 convolution operation with C1 output channels.

[0115] Branch 2: Uses a 3×3 convolution kernel, outputting feature map X2 = Conv 3×3 (X), with C2 output channels.

[0116] Branch 3: Achieve a similar effect to a 5×5 convolution kernel by using two cascaded 3×3 convolution kernels, outputting feature map X3 = Conv 3×3 (Conv 3×3 (X)), the number of output channels is C3.

[0117] The output feature maps of the three branches are concatenated along the channel dimension to obtain the output Y = [X1, X2, X3] of the STDC module, which has C channels. out =C1+C2+C3.

[0118] In terms of complexity analysis, the number of parameters P of the STDC module is determined by the dimension of the input and output channels, and the calculation formula is as follows:

[0119] P = C in ×(C1+3 2 ×C2+3 2 ×3 2 ×C3)

[0120] In this algorithm, C is set in =32, C1=16, C2=16, C3=16. Calculations show that the number of parameters in this module is significantly reduced compared to traditional modules, making it more suitable for scenarios with limited on-board computing resources.

[0121] 2) STDC Network Integration. The designed STDC modules are integrated into the encoder part of the U-net architecture to form an STDC network. The encoder contains multiple downsampling stages, each consisting of multiple STDC modules and a max-pooling layer. In each downsampling stage, the feature map size is halved and the number of channels is increased through a max-pooling layer, thereby extracting more abstract semantic features. For example, the input to the first downsampling stage is the original image. After processing by two STDC modules, the feature map size changes from H×W through a 2×2 max-pooling layer. The number of channels increases accordingly, and subsequent stages follow the same pattern to gradually achieve multi-level extraction of sea ice and water features.

[0122] Decoder Design:

[0123] 1) Context path encoding based on BiSeNet: A pre-trained STDC network is used as the base network of the encoder, and the context path of BiSeNet is introduced to encode contextual information. During the encoding process, feature maps with different downsampling ratios are obtained from different stages of the encoder. For example, feature maps with larger downsampling ratios are obtained from deeper stages of the encoder, which contain richer semantic information; feature maps with smaller downsampling ratios are obtained from shallower stages, retaining more detailed information. Global average pooling (GAP) is used to generate global contextual information for these feature maps with different downsampling ratios. For feature maps... The global average pooling operation is calculated as follows:

[0124]

[0125] Here, G is the generated global context vector. This operation compresses the feature map into a vector with global information. Subsequently, it undergoes operations such as 1×1 convolution to adjust the number of channels and is upsampled to an appropriate size before being fused with other feature maps, thereby enhancing the expression of the overall features of sea ice and water.

[0126] 2) U-shape structure upsampling and feature combination: The decoder adopts a U-shape structure to combine features from different levels of the encoding stage with the upsampled features. The specific process is as follows: First, the high-level semantic features output by the encoder are progressively upsampled. Upsampling methods can include bilinear interpolation, with each upsampling doubling the feature map size. Simultaneously, feature maps from the corresponding levels of the encoding stage are obtained, and the upsampled feature maps are concatenated with the encoding stage feature maps along the channel dimension to achieve the fusion of high- and low-level features, providing more comprehensive information for subsequent accurate segmentation.

[0127] 3) Attention Refinement Module (ARM): This module introduces an attention mechanism to further process the fused features, enhancing features related to sea ice and suppressing irrelevant background information. The specific calculation process is as follows: For the input feature map... First, global average pooling is used to obtain the global feature vector, then two 1×1 convolutional layers are applied, with the ReLU activation function used in between. The calculation formula is as follows:

[0128] A=σ(W2·ReLU(W1·GAP(F in )))

[0129] Where W1 and W2 are the parameters of a 1×1 convolution kernel, σ is the sigmoid activation function, and A is the generated attention weight map. The attention weight map is then multiplied element-wise with the input feature map to obtain the refined feature map.

[0130]

[0131] in, This indicates an element-wise multiplication operation, which highlights the characteristics of sea ice regions and improves segmentation accuracy.

[0132] 4) Feature Fusion Module (FFM): This module further fuses features from different sources and at different levels, making the final features more effective for sea ice and water segmentation. It fuses the features processed by ARM with other relevant features. Fusion methods can include element-wise addition, channel concatenation followed by convolution, etc. Assuming the inputs are feature maps F1 and F2, the output feature map F is obtained after processing by the feature fusion module. final This module enables deep feature fusion, comprehensively utilizing information from multiple sources to provide higher-quality feature representations for subsequent segmentation prediction.

[0133] 5) Segmentation Prediction and Loss Function Optimization: After the above series of processes, a 1×1 convolutional layer converts the number of channels in the final feature map into the number of categories (2 in this algorithm, corresponding to sea ice and water respectively). Then, a Sigmoid activation function is used to obtain the probability value of each pixel belonging to sea ice or water, thus generating the final segmentation prediction map. To optimize network training and alleviate the imbalance between sea ice and water categories in SAR images, a joint optimization using Binary Cross Entropy (BCE) and Dice loss is employed. The Binary Cross Entropy loss function measures the difference between the predicted probability and the true label, and its calculation formula is:

[0134]

[0135] Where N is the total number of pixels, y i For real labels, p i This is for predicting probabilities. The Dice loss measures the overlap between the predicted results and the true labels, and is calculated using the following formula:

[0136]

[0137] The joint loss function is L = αL BCE +(1-α)L Dice , where α is the balance coefficient, which is set to 0.6 in this algorithm. Through joint optimization, the network can better learn the characteristics of sea ice and water, thereby improving the segmentation performance.

[0138] Figure 2 This is a SAR satellite-borne test data, sourced from the "Tianji" constellation satellite of the Tianyi Research Institute. The data illuminates a glacier, and the sea ice covered is of the types of broken ice and multi-year-old ice.

[0139] Figure 3 The results show the detection results of sea ice and water area (yellow for seawater and red for sea ice). The results show that the proposed algorithm has a good segmentation effect on small ice fragments and old ice areas with weak microwave reflection.

[0140] Figure 4 This is another SAR satellite test data, sourced from the "Tianji" constellation satellite of Tianyi Research Institute. The data illuminates a glacier, and the sea ice covered is mainly composed of broken ice.

[0141] Figure 5 The results show the detection results for sea ice and water area (yellow for seawater and red for sea ice). The results show that the proposed algorithm has a more detailed segmentation effect on the more difficult-to-segment small fragments of sea ice.

[0142] This invention also provides a deep learning-based SAR image polar sea ice detection system, comprising: Module M1: employing a lightweight convolutional neural network architecture with dilated convolutions and a spatial pyramid pooling module with dilated convolutions as the encoder part in the deep learning model, introducing multi-scale information, and using the decoder part in the deep learning model to fuse low-level and high-level features to achieve land and water segmentation; Module M2: cutting the SAR grayscale image into data of a preset size, preparing a marker image with pixels of 0 and 1 from the ground truth image of artificially marked water, and cutting it into the same size, performing rotation, translation, and scaling operations on the prepared marker image and grayscale image, wherein the marker image and the corresponding grayscale image are augmented at the same ratio; Module M3: employing an encoding and decoding method, designing a short-term dense cascaded module in the encoding network and integrating it into the encoder-decoder segmentation network architecture, and using a pre-trained short-term dense cascaded network as the base network in the decoder, combining the context path encoding context information of the real-time semantic segmentation network to achieve accurate segmentation of sea ice and water.

[0143] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the deep learning-based SAR image polar sea ice detection method.

[0144] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0145] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A deep learning-based method for detecting polar sea ice in SAR images, characterized in that, include: Step 1: A lightweight convolutional neural network architecture with dilated convolutions and a spatial pyramid pooling module with dilated convolutions are used as the encoder part in the deep learning model to introduce multi-scale information. The decoder part in the deep learning model is used to fuse low-level features and high-level features to achieve land and water segmentation. Step 2: Cut the SAR grayscale image into data of a preset size, prepare the ground truth image of the artificially marked water area into a marker image with pixels of 0 and 1, and cut it into the same size. Perform rotation, translation and scaling operations on the prepared marker image and grayscale image, wherein the marker image and the corresponding grayscale image are augmented at the same ratio. Step 3: Using an encoding and decoding approach, the encoding network is designed with short-term dense cascaded modules and integrated into the encoder-decoder segmentation network architecture. The decoder uses a pre-trained short-term dense cascaded network as the base network and combines the context path encoding context information of the real-time semantic segmentation network to achieve accurate segmentation of sea ice and water.

2. The deep learning-based SAR image polar sea ice detection method according to claim 1, characterized in that, For the input feature map Where H is the height, W is the width, and C is the number of channels, the calculation process of dilated convolution is expressed as: in, Let H′, W′, and C′ represent the height, width, and number of channels of the output feature map, respectively. Let i, j, and k be the coordinate indices of the output feature map, and m, n, and p be the coordinate indices inside the convolution kernel. K is the convolution kernel, K is the kernel size, and r is the dilation rate parameter. The size of the receptive field is controlled by adjusting the value of r. When r = 1, the dilated convolution degenerates into a standard convolution. When r > 1, the receptive field expands as r increases. For different output strides, the dilatancy parameter of the convolutional layer is set as the ratio of the input image resolution to the output feature map resolution. If the output stride is 8, the dilatancy parameter of the first few convolutional layers is set to 1, 2, 4 in sequence; if the output stride is 16, the dilatancy parameter is set to 1, 1, 2 in sequence.

3. The deep learning-based SAR image polar sea ice detection method according to claim 2, characterized in that, The spatial pyramid pooling module uses dilated convolutions with different dilation parameters to achieve multi-scale information extraction. Its structure includes a 1×1 convolutional layer, a 3×3 dilated convolutional layer with different dilation parameters, and a global average pooling layer. 1×1 convolutional layer: Adjusts the number of channels in the input feature map, and outputs a feature map with dimension C1; 3×3 dilated convolutional layers with different dilation parameters: Multiple 3×3 dilated convolutions with different dilation parameters are used to extract features at different scales, and the output feature maps have dimensions of C2, C3, and C4, respectively. Global average pooling layer: Compresses the input feature map into a global feature vector, then adjusts the number of channels to C5 through 1×1 convolution, and upsamples to restore the original feature map size; Features at different scales are concatenated along the channel dimension, i.e., the feature maps are stacked along the channel dimension to obtain a new feature map.

4. The deep learning-based SAR image polar sea ice detection method according to claim 3, characterized in that, Deep neural networks are constructed using depthwise separable convolution, decomposing standard convolution into depthwise convolution and pointwise convolution; Depthwise convolution: For each channel of the input feature map, an independent convolution kernel is used to perform a convolution operation. The calculation process is as follows: in, The kernel is a depthwise convolution, and the number of channels in the output feature map is the same as that in the input. Pointwise convolution: Adjusting the number of channels in the output of depthwise convolution using 1×1 convolution. The calculation process is as follows: in, The kernel is a pointwise convolution, and the number of channels in the output feature map becomes C′; The h-swish activation function is used, and the formula is: Where x is the input value; the hyperparameters of the lightweight convolutional neural network architecture are set as follows: initial number of channels 32, number of layers 18, and dilation coefficient 6.

5. The deep learning-based SAR image polar sea ice detection method according to claim 4, characterized in that, The features obtained from the encoder are upsampled using bilinear interpolation to obtain a feature map of 4 times the size. Low-level features of the corresponding size are then extracted from the encoder, and dimensionality reduction is performed on these low-level features using a 1×1 convolution, reducing the number of channels from C... low Reduced to C′ low The calculation formula is: Where X is the original low-level feature map. For dimensionality reduction convolution kernels, C low C′ is the number of channels in the original low-level feature map. low Let Y be the number of channels in the dimensionality-reduced feature map. The low-level features after dimensionality reduction are fused with the high-level features after upsampling by element-wise addition or concatenation to obtain the fused feature map. The fused feature map is then bilinearly interpolated again to restore it to the same size as the original image, resulting in the final sea ice segmentation prediction map.

6. The deep learning-based SAR image polar sea ice detection method according to claim 5, characterized in that, The short-term dense cascaded module consists of multiple convolutional branches, each using a different sized convolutional kernel to achieve multi-scale feature extraction. Let the input feature map be... Where H and W are the height and width of the feature map, respectively, and C... in Given the number of input channels, the calculation process for each branch is as follows: Branch 1: Use a 1×1 convolution kernel to output feature map X1, calculated as X1 = Conv 1×1 (X), where Conv 1×1 This represents a 1×1 convolution operation with C1 output channels; Branch 2: Uses a 3×3 convolution kernel, outputting feature map X2 = Conv 3×3 (X), the number of output channels is C2; Branch 3: Achieve the effect of a 5×5 convolution kernel through two cascaded 3×3 convolution kernels, outputting feature map X3 = Conv 3×3 (Conv 3×3 (X)), the number of output channels is C3; The output feature maps of the three branches are concatenated along the channel dimension to obtain the output Y = [X1, X2, X3], which has C channels. out =C1+C2+C3; The number of parameters P in a short-term dense cascaded module is determined by the input / output channel dimension, and the calculation formula is: P=C in ×(C1+3 2 ×C2+3 2 ×3 2 ×C3) Among them, C in =32, C1=16, C2=16, C3=16.

7. The deep learning-based SAR image polar sea ice detection method according to claim 6, characterized in that, The designed short-term dense concatenated modules are integrated into the encoder part of the encoder-decoder segmentation network architecture to form a short-term dense concatenated network. The encoder contains multiple downsampling stages, each consisting of multiple short-term dense concatenated modules and a max pooling layer. In each downsampling stage, the feature map size is halved and the number of channels is increased through the max pooling layer, thereby extracting more abstract semantic features. In the first downsampling stage, the input is the original image. After processing by two short, densely cascaded modules, it passes through a 2×2 max-pooling layer, changing the feature map size from H×W to [the original image size]. The number of channels increases accordingly, gradually realizing multi-level extraction of sea ice and water features.

8. The deep learning-based SAR image polar sea ice detection method according to claim 7, characterized in that, During the encoding process, feature maps with different downsampling ratios are obtained from different stages of the encoder. Global average pooling is then used to generate global context information for these feature maps with different downsampling ratios. The global average pooling operation is calculated as follows: Wherein, G is the generated global context vector. This operation compresses the feature map into a vector with global information. Subsequently, a 1×1 convolution operation is performed to adjust the number of channels, and the vector is upsampled to a preset size and fused with other feature maps to enhance the expression of the overall features of sea ice and water. The decoder adopts a U-shape structure, which combines features from different levels of the encoding stage with the upsampled features. The specific process is as follows: First, the high-level semantic features output by the encoder are upsampled step by step. The upsampling method is bilinear interpolation. Each upsampling doubles the size of the feature map. At the same time, the feature map is obtained from the corresponding level of the encoding stage. The upsampled feature map is then concatenated with the feature map of the encoding stage in the channel dimension to achieve the fusion of high and low level features. By introducing an attention mechanism, the fused features are further processed to enhance features related to sea ice and suppress irrelevant information. The specific calculation process is as follows: The input feature map... Where H ″ W ″ C ″ Given the height, width, and number of channels of the input feature map, a global average pooling process is first used to obtain the global feature vector. Then, the vector is passed through two 1×1 convolutional layers, with the ReLU activation function used in between. The calculation formula is as follows: A=σ(W2·ReLU(W1·GAP(F in ))) Where W1 and W2 are the parameters of a 1×1 convolution kernel, σ is the sigmoid activation function, GAP is the global average pooling operation, and A is the generated attention weight map. The attention weight map is then multiplied element-wise with the input feature map to obtain the refined feature map. in, This indicates an element-wise multiplication operation, which highlights the characteristics of sea ice regions and improves segmentation accuracy. The features processed by the attention mechanism are fused with other relevant features. The fusion method is either element-wise addition or channel concatenation followed by convolution. The final feature map's channel count is converted to the number of categories using a 1×1 convolutional layer. Then, a Sigmoid activation function is applied to obtain the probability value of each pixel belonging to sea ice or water, thus generating the final segmentation prediction map. To optimize network training and alleviate the class imbalance problem of sea ice and water in SAR images, binary cross-entropy and Dice loss are jointly used for optimization. The binary cross-entropy loss function measures the difference between the predicted probability and the true label, and its calculation formula is as follows: Where N is the total number of pixels, y i For real labels, p i To predict probabilities; Dice loss is used to measure the degree of overlap between the predicted result and the true label. The calculation formula is as follows: The joint loss function is: L = αL BCE +(1-α)L Dice , where α is the balance coefficient, and joint optimization enables the network to better learn the characteristics of sea ice and water, thereby improving segmentation performance.

9. A deep learning-based SAR image polar sea ice detection system, characterized in that, The deep learning-based SAR image polar sea ice detection method according to any one of claims 1 to 8 includes: Module M1: It adopts a lightweight convolutional neural network architecture with dilated convolution and a spatial pyramid pooling module with dilated convolution as the encoder part in the deep learning model, introduces multi-scale information, and uses the decoder part in the deep learning model to fuse low-level features and high-level features to achieve land and water segmentation. Module M2: Cuts the grayscale image of SAR into data of a preset size, prepares the ground truth image of artificially marked water area into a marker image with pixels of 0 and 1, and cuts it into the same size. Performs rotation, translation and scaling operations on the prepared marker image and grayscale image, wherein the marker image and the corresponding grayscale image are augmented at the same ratio. Module M3: It adopts an encoding and decoding approach. The encoding network is designed with short-term dense cascaded modules and integrated into the encoder-decoder segmentation network architecture. The decoder uses a pre-trained short-term dense cascaded network as the base network and combines the context path encoding context information of the real-time semantic segmentation network to achieve accurate segmentation of sea ice and water.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the deep learning-based SAR image polar sea ice detection method according to any one of claims 1 to 8.

Images