SAR internal wave automatic detection method based on location adaptive convolutional neural network

Abstract

This invention belongs to the field of marine remote sensing technology, specifically relating to an automatic SAR ocean internal wave detection method based on a position-adaptive convolutional neural network. The method includes collecting and preprocessing SAR image data within a target area over a given time period; drawing internal wave mask images as ground truth labels based on the internal wave characteristics of the images; constructing a position-adaptive convolutional neural network; training the model using remote sensing images and ground truth labels to obtain an internal wave extraction model; and inputting SAR images from other time periods into the internal wave extraction model to extract internal wave information, thereby achieving accurate identification and mapping of the morphology and location of ocean internal waves. This invention is based on deep learning for ocean internal wave recognition, overcoming the difficulty of traditional image detection methods being susceptible to speckle noise and improving the fragmented recognition results caused by the inability of conventional convolutional methods to adapt to internal wave morphology.

Description

Technical Field

[0001] This invention belongs to the field of marine remote sensing technology, specifically relating to an automatic SAR internal wave detection method based on a position-adaptive convolutional neural network. Background Technology

[0002] Internal ocean solitary waves are a typical ocean wave phenomenon widely distributed in marginal seas and continental shelf areas. They often occur in seawater with a stable, stratified density structure, and their waveforms typically exhibit a wave train pattern arranged in an orderly manner according to amplitude and crest length. This phenomenon has significant impacts on marine engineering, underwater acoustic propagation, and submersible navigation. Currently, the main techniques for detecting internal ocean waves are in-situ observation and satellite remote sensing. In-situ observation, as the most direct and highly accurate method, has been widely used to analyze the basic characteristics of internal waves, ocean mixing effects, and their interaction with mesoscale processes. However, this method has limitations such as high cost, high workload, and difficulty in achieving large-scale synchronous observation. Satellite remote sensing technology effectively overcomes these shortcomings. Its imaging principle is based on the smooth and rough sea surface textures formed by the converging and diverging motions of the sea surface caused by internal waves. Remote sensing technologies commonly used for detecting internal ocean waves can be mainly divided into optical sensors and synthetic aperture radar (SAR). Among them, synthetic aperture radar, as a type of active sensor, can effectively capture changes in sea surface roughness caused by internal waves at a spatial resolution scale of one to tens of meters. This technology is unaffected by cloud cover and sunlight conditions, enabling all-weather Earth observation. In SAR-acquired internal wave images, internal waves typically appear as orderly alternating bright and dark stripes. These features can often be identified using classic image detection methods such as wavelet transform, Canny operator, and Fourier transform. However, due to the unique imaging mechanism of SAR systems, these traditional detection methods often struggle to effectively suppress speckle noise and are susceptible to interference from other marine phenomena. Frequency domain analysis methods, in practical applications, heavily rely on the selection of wavelet basis functions and the design of signal decomposition strategies, and often require manual intervention in the computation process, resulting in significant limitations when processing large-scale data and meeting operational requirements.

[0003] The following key issues still need to be addressed in the automatic extraction of ocean internal wave fringe information from current SAR images: 1. Complex background interference. In SAR images, the extraction of internal ocean waves is often severely interfered with by complex sea surface backgrounds (such as wind fields, ocean currents, ship wakes, etc.). Traditional deep convolutional neural networks (such as standard ResNet) usually adopt a stacked structure with a single path, and their feature transformation methods are relatively simple. When facing such complex backgrounds, the model capacity and feature richness may be insufficient, resulting in limited ability to distinguish key features of internal waves.

[0004] 2. Conventional convolution kernels struggle to adapt to the morphology of internal waves. In SAR images, internal waves typically appear as long, continuous stripes, exhibiting significant spatial variability in shape and scale. Conventional convolution kernels, due to weight sharing and a fixed receptive field, are ill-suited to these localized feature variations, leading to incomplete and fragmented internal wave extraction.

[0005] 3. Class imbalance problem. Internal waves account for a very small percentage of pixels in SAR images (less than 10% or even less), resulting in severe class imbalance and hindering the effective learning of internal wave features. Summary of the Invention

[0006] This invention overcomes the above-mentioned defects and provides an automatic SAR internal wave detection method based on a position-adaptive convolutional neural network. It solves the problems in the prior art, such as the limited ability to distinguish key features of internal waves due to complex background interference, the difficulty of conventional convolution to adapt to the internal wave morphology leading to incomplete and broken internal wave extraction, and the serious class imbalance caused by the extremely small proportion of internal waves in the SAR image.

[0007] To achieve the above objectives, the present invention provides an automatic SAR internal wave detection method based on a position-adaptive convolutional neural network, comprising the following steps: S1. Collect SAR ocean internal wave image data within the time range of the target area and perform preprocessing; S2. Select a sub-time period within the time range. For the SAR image within the sub-time period, draw a pixel-level precision internal wave mask image as the real label based on the internal wave performance characteristics. Then, uniformly crop the image and its label into a standard-sized sub-image to construct a dataset. S3. Construct a position-adaptive convolutional neural network that integrates U-Net, ResNeXt, and LSC; S4. Use the dataset described in S2 to train and optimize the network model to obtain the internal wave extraction model; S5. The internal wave extraction model is used to automatically extract internal wave information from SAR images in other time periods and generate internal wave detection results.

[0008] Furthermore, in step S3, the position-adaptive convolutional neural network uses U-Net as the backbone network, embeds a ResNeXt residual module in the encoder, and embeds an LSC module in the decoder.

[0009] Furthermore, the LSC module includes: Large kernel sensing module used to generate dynamic convolutional kernel weights; An adaptive adjustment unit is used to calculate the stability index of the dynamic convolution kernel weights and generate a confidence map based on the index. In addition, a small kernel perception module that performs position-adaptive convolution on the input features using the dynamically adjusted convolutional kernel weights based on the confidence map.

[0010] Furthermore, the adaptive adjustment unit adjusts the dynamic convolutional kernel weights through the following steps: S31. The large kernel perception module generates a dynamic convolutional kernel weight matrix; S32. Calculate the standard deviation S of the kernel weight at each spatial location, which serves as a stability index for the dynamic kernel at that location. S33. Convert the standard deviation S into a confidence level C using a learnable scaling factor, expressed as: , Where α is a learnable parameter; S34. Multiply the confidence level element by element with the dynamic kernel weight matrix to obtain the adjusted kernel weight; S35. The small kernel perception module uses adjusted kernel weights to perform position-adaptive convolution operations on the input features.

[0011] Furthermore, in step S4, a composite loss function is used during the training process to optimize the internal wave class imbalance problem.

[0012] Furthermore, the composite loss function combines the three loss functions Dice, Focal, and MCC, and its expression is as follows: , Where α and β represent the balance factors among the three loss functions: Dice, Focal, and MCC.

[0013] Furthermore, in the composite loss function, α=0.3 and β=0.2.

[0014] Furthermore, in step S5, SAR images from other time periods are input into the internal wave extraction model to automatically extract internal wave information. The network outputs a probability map of each pixel belonging to the internal wave feature, which is then binarized to form the final internal wave extraction result.

[0015] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0016] This invention utilizes high spatial resolution synthetic aperture radar (SAR) images and, through an integrated ResNeXt algorithm employing a grouped convolution strategy, significantly enhances the network's width and feature diversity without substantially increasing the number of parameters. This enables the model to learn richer and more discriminative internal wave features, overcoming the problem that SAR images are susceptible to the influence of ocean background during internal wave stripe extraction from remote sensing images, resulting in limited ability to distinguish key internal wave features. The LSC module actively optimizes the local feature aggregation method, using parallel large and small convolution kernels to better capture multi-scale local contextual information such as internal wave edges and patterns. This is more advantageous for the refined extraction of complex and varied ocean internal waves, solving the problem that conventional convolution cannot adapt to the morphology of internal waves, leading to incomplete and fragmented internal wave extraction. The fused composite loss function of Dice, Focal, and MCC works synergistically to drive model learning from three dimensions: region matching, focusing on difficult samples, and macroscopic statistical stability. This guides features towards correct and robust segmentation decisions, addressing the problem of class imbalance caused by the extremely low proportion of internal wave pixels in SAR images.

[0017] The innovative structure (ResNeXt+LSC+U-Net) and innovative loss function (Dice+Focal+MCC) of this invention, with their collaborative design of "strong feature extractor + refined optimization target", make the model more suitable for the inherent requirements of the complex task of "high-precision and robust recognition of ocean internal wave morphology". Furthermore, this invention is also applicable to satellite SAR images other than Sentinel-1. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the automatic SAR internal wave detection method based on a position-adaptive convolutional neural network of the present invention; Figure 2 This is a schematic diagram of the position-adaptive convolutional neural network structure of the present invention; Figure 3 This is a comparison chart of the internal wave detection results of the position-adaptive convolutional neural network in this embodiment of the invention with four other deep learning models. Detailed Implementation

[0019] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This invention provides an automatic SAR internal wave detection method based on a position-adaptive convolutional neural network, such as... Figure 1 As shown, the details are as follows: 1. Determine the target sea area and time range, collect multi-temporal synthetic aperture radar remote sensing image data within this area, and perform preprocessing such as radiometric calibration, noise filtering, and resampling to enhance image quality and internal wave characteristics. The image data used is Sentinel-1 imagery covering the target sea area, with a sufficient number of images to meet the training and testing requirements of the internal wave extraction model, and a spatial resolution better than 10 meters.

[0021] In a specific implementation, this embodiment of the invention collected 476 Sentinel-1 SAR images of a certain sea area from 2014 to 2023, and performed radiometric calibration, filtering and denoising, and resampling processing.

[0022] Second, for images within a specific sub-time period, based on the characteristics of internal waves in the images, corresponding internal wave mask images were created as ocean internal wave real labels through expert interpretation. All images and their corresponding labels were then uniformly cropped into 256×256 pixel sub-images to further adapt to network input and enhance data utilization. In creating the real labels, professional image processing software was used, with verified historical internal wave data as auxiliary references, and the process was completed through visual interpretation and delineation, followed by cross-validation.

[0023] In specific implementation, this embodiment of the invention uses image processing software in conjunction with stylus operation to extract the inner wave stripe information from 476 SAR images as the true value of the inner wave label, and crops all images and labels into 256×256 sub-images with a step size of 256, obtaining a total of 2246 pairs of sub-image samples.

[0024] III. Constructing a position-adaptive convolutional neural network, such as Figure 2 As shown, the U-net semantic segmentation model is selected as the backbone network, the ResNeXt residual module is used as the core module of the U-Net encoder, and the LSC (Large-Small Convolution) module is embedded in the decoder.

[0025] In SAR images, internal waves typically exhibit weak contrast and elongated stripe patterns, easily confused with the surrounding sea surface texture. To enhance the discriminative representation of such fine structures under complex background conditions, a ResNeXt module is embedded in the encoder. Each residual block introduces grouped convolutions to extend the residual learning framework, thereby enabling multiple parallel feature transformation paths. These paths independently process the input features, and the results are finally aggregated, providing the model with additional feature flexibility and diversity. The transformed features are combined with the input features through identity shortcut connections, which helps to achieve stable optimization and preservation of effective information during deep encoding. This design increases feature diversity without significantly increasing computational cost, enabling the network to capture heterogeneous texture responses in SAR images caused by internal waves and background processes.

[0026] In SAR images, internal wave features typically appear as elongated, curved, and spatially heterogeneous stripe patterns. Their feature scale and orientation can vary significantly across the image. To address this, an LSC module is introduced into the decoder to achieve position-adaptive feature transformation. The LSC module consists of two complementary components: Large Kernel Perception (LKP) and Small Kernel Aggregation (SKA). Large Kernel Perception helps capture the overall orientation and spatial layout of IW stripes over a large area of ​​the sea surface, encoding the global spatial layout of the internal wave stripe structure and generating convolution weights that reflect spatial variations in different contexts within the image. Small Kernel Aggregation uses these generated weights to perform position-specific convolutions on the input feature map, enabling a more refined focus on local details and texture changes at stripe edges. The combination of large and small kernels provides a way to improve the model's representation of IW stripe features while controlling computational complexity.

[0027] However, ResNeXt, through multiple parallel paths and grouped convolutions, tends to learn highly diverse but potentially overly abstract or scattered local features. While its output feature maps are highly discriminative, features extracted from different paths may differ semantically. Furthermore, the core of the LSC convolution module is a dynamic kernel generation and application mechanism, rather than a simple multi-scale convolutional parallel aggregation, aiming to fuse multi-scale context. When directly receiving skip connection features from the encoder (especially deep ResNeXt layers), the scale inconsistency and semantic gap of these input features may be amplified. The Large Kernel Perception (LKP) module uses a 7×7 large kernel depthwise separable convolution to capture contextual information with a wide receptive field. Features are then further processed through another 1×1 convolution layer, and finally, a 1×1 convolution projects the features into dynamic convolution kernel weights. The shape of these weights corresponds to a 3×3 convolution kernel (9 weight values) generated for each spatial location. The Small Kernel Aggregation (SKA) module utilizes the dynamic kernel weights generated by LKP to perform position-adaptive feature transformations on the input features. In the specific implementation, the input features are expanded into local 3×3 neighborhood blocks, and then multiplied element-wise with dynamic kernel weights and summed to simulate the effect of using dynamically generated 3×3 convolution kernels for convolution.

[0028] Specifically, the LKP module relies on 7×7 deep convolutions with large kernels to capture wide receptive field context and generate dynamic kernels. If the input features contain a large number of fragmented or inconsistent local responses, it will interfere with the generation quality of the dynamic kernels, resulting in unstable or inaccurate kernel weights. When the SKA module applies these dynamic kernels, the unstable kernel weights will further amplify the noise and inconsistencies in the input features, interfering with the model's judgment of the overall internal wave direction and the extraction of local details.

[0029] This mismatch can lead to a decrease in the efficiency of the decoder during feature reconstruction and fusion, manifesting as feature fusion noise near jump connection points, ultimately affecting the sharpness of the segmentation boundaries and the continuity of the inner wave fringes. Therefore, this invention introduces a dynamic kernel quality evaluation mechanism in the LSC module, which adaptively adjusts the application intensity based on the stability of the generated kernel, as follows: Dynamic kernel generation: The LKP module processes the input features and generates a dynamic convolutional kernel weight matrix W, with a shape corresponding to a 3×3 convolutional kernel at each position.

[0030] Kernel quality assessment: For the generated dynamic kernel, we calculate the stability index of the kernel weights at each spatial location. A simple and effective method is to calculate the standard deviation S(x,y) of the nine kernel weight values ​​at each location. A higher standard deviation indicates that the dynamic kernel at that location varies greatly, and may be unstable or ambiguous; a lower standard deviation indicates that the kernel weights are relatively consistent and of higher quality.

[0031] Adaptive adjustment: The standard deviation S is converted into a confidence score C using a learnable scaling factor. , Here, α is a learnable parameter that controls the adjustment intensity. Then, the confidence level C is multiplied element-wise by the dynamic kernel weights to obtain the adjusted kernel weights. : , In this way, the kernel weight is enhanced at positions with higher dynamic kernel quality, and suppressed at positions with lower quality.

[0032] Feature Transformation: The SKA module uses adjusted dynamic kernel weights. The input features are subjected to position-adaptive convolution operations, and then the output is obtained through batch normalization and residual connections.

[0033] This approach enables the LSC module to adaptively adjust its application intensity based on the quality of the generated dynamic kernel itself, thereby maintaining robustness when input features are inconsistent, improving feature fusion performance, and ultimately enhancing the clarity of segmentation boundaries and the continuity of inner wavy fringes.

[0034] The position-adaptive convolutional neural network constructed in this invention effectively enhances the model's ability to extract feature diversity of heterogeneous textures in SAR images through the ResNeXt module introduced in the encoder. Combined with the large-kernel-small-kernel collaborative mechanism of the LSC module in the decoder, it realizes adaptive fusion of the global spatial layout and local edge details of internal wave fringes. This design enables the model to accurately capture and reconstruct multi-scale internal wave features from weak to significant and from local to global, ultimately achieving high-precision and robust automatic detection of internal waves in complex sea surface backgrounds.

[0035] Fourth, the constructed SAR image sample dataset is used as input, and its corresponding true internal wave labels are used as ground truth values. The dataset is then divided into training, validation, and test sets according to a set ratio to complete the training and optimization of the internal wave extraction model. During model training, data augmentation strategies are employed to expand the training samples, including rotation, flipping, scaling, translation, and brightness alteration to improve the model's generalization ability. To address the negative impact of internal wave class imbalance on model training, a complex kernel loss function is used, which integrates Dice loss, Focal loss, and Matthews correlation coefficient (MCC) loss, to improve the recognition efficiency of internal wave pixels. The calculation formula is as follows: , Here, α and β represent the balance factors among the three loss functions: Dice, Focal, and MCC. When α=1 and β=0, it is Dice Loss; when α=0 and β=1, it is Focal Loss; and when α=β=0, it is MCC Loss. The three loss functions work together to drive model learning from three dimensions: region matching, hard sample focusing, and macroscopic statistical stability. Theoretically, this can achieve a more robust and accurate convergence effect.

[0036] Although the composite loss function, ResNeXt, and LSC modules are all designed to address the challenges of ISW extraction, their collaborative training faces new challenges: the gradient generated by the composite loss may interfere with the network's stable optimization, leading to a degradation in the network's recognition ability. Therefore, to investigate the impact of the composite loss function on model performance and to explore the optimal configuration of the balance factors in the loss function formula, multiple sets of comparative experiments were designed, and the F1-score was used as the core indicator for quantitative analysis. Verification showed that when α=0.3 and β=0.2, the model achieved the highest performance, with an F1-score of 92.0%. This indicates that the determined balance factor allocation scheme can effectively coordinate various supervision signals and significantly improve the model's overall performance in the internal wave recognition task.

[0037] Meanwhile, to address the problem that the gradient flow becomes complex due to the deep heterogeneous network structure (ResNeXt+LSC+U-net), leading to gradient vanishing, exploding, or instability that makes it difficult for the model to converge to the optimal solution, especially since the embedding of the two modules may disrupt the optimization friendliness of the original U-Net's concise structure, the AdamW (Adam with decoupled weight decay) adaptive optimizer is used during model training, combined with composite learning rate scheduling, which helps the model escape local optima more smoothly in order to find a better solution.

[0038] In specific implementation, this embodiment of the invention uses 476 SAR images of a certain sea area and their corresponding binary map labels to train the established location adaptive neural network. A total of 2246 pairs of sub-map samples participate in the training. Among them, 80% of the training set and 20% of the validation set are divided into training sets and validation sets, and a total of 1000 training rounds are conducted. The model with the highest accuracy on the validation set is selected as the final model.

[0039] 5. Using the trained internal wave automatic extraction model, internal wave information in SAR images from other time periods is automatically extracted. The network outputs a probability map of each pixel belonging to the internal wave feature. The probability map is binarized to form the final internal wave extraction result.

[0040] To verify the contribution of each network component, this invention uses five model configurations for comparative verification. The models used are: (1) the original U-Net as the baseline model; (2) a U-Net with a ResNeXt module embedded in the encoder; (3) a U-Net with an LSC module embedded in the decoder; (4) a U-Net that includes both a ResNeXt embedding block and an LSC module; and (5) an adaptive convolutional neural network model based on this invention. All configurations (1)-(4) consistently use the MCC loss function. All models are trained using the same training dataset, optimization strategy, and evaluation protocol. Figure 3 Representative qualitative results obtained from five model configurations are presented. The baseline U-Net suffers from incomplete extraction and fragmentation of inner wave fringes under complex background conditions; introducing ResNeXt embedding blocks or LSC modules can improve the continuity of fringes and reduce misclassification to varying degrees; at the same time, the model integrating these two components generates a more complete and coherent inner wave fringe structure.

[0041] The embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. An automatic SAR internal wave detection method based on a position-adaptive convolutional neural network, characterized in that, Includes the following steps: S1. Collect SAR ocean internal wave image data within the time range of the target area and perform preprocessing; S2. Select a sub-time period within the time range. For the SAR image within the sub-time period, draw a pixel-level precision internal wave mask image as the real label based on the internal wave performance characteristics. Then, uniformly crop the image and its label into a standard-sized sub-image to construct a dataset. S3. Construct a position-adaptive convolutional neural network that integrates U-Net, ResNeXt, and LSC; S4. Use the dataset described in S2 to train and optimize the network model to obtain the internal wave extraction model; S5. The internal wave extraction model is used to automatically extract internal wave information from SAR images in other time periods and generate internal wave detection results.

2. The automatic SAR internal wave detection method based on position-adaptive convolutional neural network according to claim 1, characterized in that, In step S3, the position-adaptive convolutional neural network uses U-Net as the backbone network, embeds a ResNeXt residual module in the encoder, and embeds an LSC module in the decoder.

3. The automatic SAR internal wave detection method based on a position-adaptive convolutional neural network according to claim 2, characterized in that, The LSC module includes: Large kernel sensing module used to generate dynamic convolutional kernel weights; An adaptive adjustment unit is used to calculate the stability index of the dynamic convolution kernel weights and generate a confidence map based on the index. In addition, a small kernel perception module that performs position-adaptive convolution on the input features using the dynamically adjusted convolutional kernel weights based on the confidence map.

4. The automatic SAR internal wave detection method based on position-adaptive convolutional neural network according to claim 3, characterized in that, The adaptive adjustment unit adjusts the dynamic convolutional kernel weights through the following steps: S31. The large kernel perception module generates a dynamic convolutional kernel weight matrix; S32. Calculate the standard deviation S of the kernel weight at each spatial location, which serves as a stability index for the dynamic kernel at that location. S33. Convert the standard deviation S into a confidence level C using a learnable scaling factor, expressed as: ， Where α is a learnable parameter; S34. Multiply the confidence level element by element with the dynamic kernel weight matrix to obtain the adjusted kernel weight; S35. The small kernel perception module uses adjusted kernel weights to perform position-adaptive convolution operations on the input features.

5. The automatic SAR internal wave detection method based on position-adaptive convolutional neural network according to claim 1, characterized in that, In step S4, a composite loss function is used during training to optimize the internal wave class imbalance problem.

6. The automatic SAR internal wave detection method based on position-adaptive convolutional neural network according to claim 5, characterized in that, The expression for the composite loss function is as follows: ， Where α and β represent the balance factors among the three loss functions: Dice, Focal, and MCC.

7. The automatic SAR internal wave detection method based on a position-adaptive convolutional neural network according to claim 6, characterized in that, In the composite loss function, α=0.3 and β=0.

2.

8. The automatic SAR internal wave detection method based on position-adaptive convolutional neural network according to claim 1, characterized in that, In step S5, SAR images from other time periods are input into the internal wave extraction model to automatically extract internal wave information. The network outputs a probability map of each pixel belonging to the internal wave feature, which is then binarized to form the final internal wave extraction result.

Images