Method for extracting oceanic internal waves from sar remote sensing images based on semantic segmentation

Abstract

The application belongs to the technical field of remote sensing image semantic segmentation, and discloses a method for extracting ocean internal waves from a SAR remote sensing image based on semantic segmentation, which constructs a semantic segmentation model for extracting ocean internal waves, introduces a frequency domain-attention enhancement module in the model feature extraction stage, models the response characteristics of different frequency components in the multi-scale level, so that the network can simultaneously perceive the high-frequency fine and long texture, edge mutation information and low-frequency background structure information of the ocean internal wave, and the attention guide mechanism guides the network to actively focus on the potential internal wave area in the training. In the model morphology matching stage, a morphology enhancement block is introduced for the geometric characteristics of the ocean internal wave, such as the fine and long directionality, the direction adaptive modeling of the characteristics is carried out, through the series connection of multiple morphology enhancement blocks, the model can effectively suppress the background interference in the process of step-by-step upsampling, maintain the continuity and integrity of the internal wave structure, and improve the boundary definition and structure consistency of the segmentation result.

Description

Technical Field

[0001] This invention belongs to the field of semantic segmentation technology of remote sensing images, specifically relating to a method for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation, which is particularly suitable for extracting target scenes such as ocean internal waves in SAR remote sensing images. Background Technology

[0002] With the development of marine observation methods such as satellite remote sensing technology, people can easily acquire marine remote sensing image data (such as SAR or optical remote sensing images) with large coverage and high spatial resolution. While the massive amount of remote sensing images brings rich information to related scientific research, it also puts forward higher requirements for the efficiency and accuracy of image analysis and processing. In SAR remote sensing images, ocean internal waves are usually characterized by low contrast, strong background interference, elongated and tilted shape, and low pixel ratio.

[0003] Existing methods often optimize only one characteristic, making it difficult to simultaneously address multiple complex features. Therefore, there is an urgent need for a technical solution that can efficiently and accurately extract ocean internal waves to improve automation, reduce labor costs, and meet the practical application needs of massive SAR images. Over the past few decades, researchers have proposed various methods for ocean internal wave extraction based on wavelet transform, edge detection, and matrix analysis. However, these traditional methods typically rely on manually setting thresholds and require repeated parameter tuning for different scenarios, resulting in poor versatility and stability, and hindering automated deployment.

[0004] With the development of neural network technology, deep learning-based methods for extracting internal ocean waves have received widespread attention in recent years. These methods leverage the automatic feature learning capabilities of deep models to improve the accuracy of internal wave identification and segmentation. Early studies often used rectangular bounding boxes, which struggled to depict the fine morphology of internal waves. Subsequently, improved semantic segmentation networks such as U-Net and SegNet were introduced, achieving more refined internal wave extraction through multi-scale feature modeling, noise suppression, and loss function optimization. Some studies also combined filtering, feature fusion, and traditional classifiers to improve robustness under complex sea conditions. Recent methods further integrate Transformer structures and customized loss functions to enhance the modeling ability for low-contrast, elongated internal waves.

[0005] Despite significant advancements in accuracy and automation achieved by deep learning, several challenges remain: 1. Difficulty in distinguishing internal wave features from complex ocean backgrounds; 2. Mismatch between the elongated, tilted morphology of internal waves and the receptive field of traditional convolutional kernels; 3. Sparse distribution of internal waves leading to severe class imbalance. Therefore, a semantic segmentation method capable of stably and accurately extracting ocean internal waves from complex ocean SAR remote sensing images is urgently needed to meet practical requirements such as monitoring ocean internal wave propagation processes and scientific research. Summary of the Invention

[0006] The purpose of this invention is to propose a method for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation. This method integrates a frequency domain perception-driven feature modeling mechanism with a morphology alignment-driven decoding and reconstruction mechanism, systematically solving the problem of accurately extracting targets with low contrast, weak texture, and slender structures from two levels: feature extraction and morphology matching. This helps to improve the recognition accuracy and robustness of ocean internal waves under complex sea conditions.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A method for extracting internal ocean waves from SAR remote sensing images based on semantic segmentation includes the following steps:

[0009] Step 1. Acquire raw SAR remote sensing images, perform annotation and preprocessing operations, and construct a training dataset;

[0010] Step 2. Construct a semantic segmentation model including an encoder and a decoder;

[0011] The encoder consists of four feature extraction stages, namely the first, second, third, and fourth feature extraction stages, as well as a frequency domain-attention enhancement module (FAEM). The four feature extraction stages are connected in sequence and are used to extract ocean internal wave features at different scales.

[0012] The output features of each feature extraction stage are input into the FAEM module. The FAEM module introduces frequency domain analysis and combines an attention mechanism to generate a frequency domain saliency probability map to enhance the model's response to low-contrast ocean internal wave regions.

[0013] The generated frequency domain saliency probability map is multiplied by the output features at the pixel level of the first, second, third, and fourth feature extraction stages, and residuals are concatenated to obtain the features of each feature extraction stage after being guided by the probability map.

[0014] The decoder includes three sequentially connected morphological enhancement blocks for morphological alignment modeling of features during the upsampling process, enabling the model to match the geometry of ocean internal waves while gradually restoring spatial resolution.

[0015] Define three morphological enhancement blocks as the first, second, and third morphological enhancement blocks;

[0016] The features guided by the third and fourth feature extraction stages are input into the third morphological enhancement block; the output features of the third morphological enhancement block and the features guided by the second feature extraction stage are input into the second morphological enhancement block;

[0017] The output features of the second morphological enhancement block and the features guided by the first feature extraction stage are input into the first morphological enhancement block; the output results of the three morphological enhancement blocks are concatenated to finally generate a predicted segmentation map corresponding to the model input image;

[0018] Step 3. Train the model based on the training dataset, and use the trained model to extract ocean internal waves from SAR remote sensing images.

[0019] Furthermore, based on the above method, this invention also proposes a corresponding system for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation, which adopts the following technical solution:

[0020] A system for extracting internal ocean waves from SAR remote sensing images based on semantic segmentation includes the following modules:

[0021] The preprocessing module is used to acquire raw SAR remote sensing images, perform annotation and preprocessing operations, and build a training dataset;

[0022] The ocean internal wave extraction module is used to build a semantic segmentation model that includes an encoder and a decoder;

[0023] The encoder consists of four feature extraction stages, namely the first, second, third, and fourth feature extraction stages, as well as a frequency domain-attention enhancement module (FAEM). The four feature extraction stages are connected in sequence and are used to extract ocean internal wave features at different scales.

[0024] The output features of each feature extraction stage are input into the FAEM module; the FAEM module applies frequency domain enhancement blocks, feature pyramids, and attention mechanisms to generate a frequency domain saliency probability map;

[0025] The generated frequency domain saliency probability map is multiplied by the output features at the pixel level of the first, second, third, and fourth feature extraction stages, and residuals are concatenated to obtain the features of each feature extraction stage after being guided by the probability map.

[0026] The decoder includes three sequentially connected morphological enhancement blocks for morphological alignment modeling of features during the upsampling process, enabling the model to match the geometry of ocean internal waves while gradually restoring spatial resolution.

[0027] Define three morphological enhancement blocks as the first, second, and third morphological enhancement blocks;

[0028] The features guided by the third and fourth feature extraction stages are input into the third morphological enhancement block; the output features of the third morphological enhancement block and the features guided by the second feature extraction stage are input into the second morphological enhancement block;

[0029] The output features of the second morphological enhancement block and the features guided by the first feature extraction stage are input into the first morphological enhancement block; the output results of the three morphological enhancement blocks are concatenated to finally generate a predicted segmentation map corresponding to the model input image;

[0030] The model is trained based on the training dataset, and the trained model is used to extract ocean internal waves from SAR remote sensing images.

[0031] The present invention has the following advantages:

[0032] As described above, this invention proposes a method for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation. This method constructs a semantic segmentation network structure for extracting ocean internal waves from SAR remote sensing images. By integrating a frequency-domain perception-driven feature modeling mechanism with a morphology-alignment-driven decoding and reconstruction mechanism, it systematically addresses the difficulty in accurately extracting low-contrast, weak-texture, and elongated structural targets from both feature representation and geometric reconstruction perspectives. In the feature extraction stage, a frequency-domain attention enhancement module is introduced to explicitly model the response characteristics of different frequency components at multiple encoder scales. This enables the network to simultaneously perceive the high-frequency elongated texture, edge abruptness information, and low-frequency background structural information of ocean internal waves, achieving effective complementarity between deep semantic information and shallow spatial details. This significantly improves the consistency and scale robustness of feature representation in internal wave scenarios at different scales and propagation intensities. Simultaneously, an attention-guided mechanism guides the network to actively focus on potential internal wave regions during training, effectively alleviating the background dominance problem caused by low target proportion. In the morphology matching stage, a morphological enhancement block is introduced to adaptively model the orientation of the features, targeting the elongated and directional geometric features of ocean internal waves. This mechanism overcomes the limitations of traditional square convolutions, which have fixed receptive fields and insufficient morphological alignment capabilities. It enables the network to dynamically adjust the importance of different convolutional branches based on feature responses, significantly enhancing the matching accuracy for strip-shaped structures while maintaining the model's generalization ability. Through the concatenation of multiple morphological enhancement blocks, the model effectively suppresses background interference during progressive upsampling, maintaining the continuity and integrity of the internal wave structure, and significantly improving the boundary clarity and structural consistency of the segmentation results. Finally, by introducing a composite loss optimization strategy for sparse, elongated targets, this invention further mitigates the adverse effects of class imbalance on model training, ensuring the complete extraction of the internal wave structure from the optimization objective level. This invention, through the collaborative design of frequency domain awareness, multi-scale fusion, attention guidance, and morphological alignment at the overall network architecture level, not only improves the model's robust modeling ability for ocean internal wave features in low-contrast SAR scenarios but also achieves significant improvements in preserving the structure and refining the boundaries of elongated targets. Compared to existing technologies, this invention exhibits superior comprehensive performance in terms of weak texture background adaptability, scale variation robustness, and the structural integrity of strip-shaped targets, making it highly valuable for engineering applications and widespread adoption. Attached Figure Description

[0033] Figure 1 This is a flowchart illustrating the method for extracting internal ocean waves from SAR remote sensing images in Embodiment 1 of the present invention.

[0034] Figure 2 This is a schematic diagram illustrating the three main problems to be solved in Embodiment 1 of the present invention; wherein Figure 2 (a) in the image represents a low contrast issue. Figure 2 (b) in the text represents a shape mismatch problem. Figure 2 (c) in the table represents a severe class imbalance problem;

[0035] Figure 3 This is an overall block diagram of the semantic segmentation model built in Embodiment 1 of the present invention;

[0036] Figure 4 This is a structural diagram of the frequency domain-attention enhancement module in Embodiment 1 of the present invention;

[0037] Figure 5 This is a structural diagram of the frequency domain enhancement block in the frequency domain-attention enhancement module of Embodiment 1 of the present invention;

[0038] Figure 6 This is a structural diagram of the feature pyramid in the frequency domain-attention enhancement module in Embodiment 1 of the present invention;

[0039] Figure 7 This is a structural diagram of the morphological enhancement block in Embodiment 1 of the present invention;

[0040] Figure 8 This is an example image showing the effect of manual annotation on a dataset in Embodiment 1 of the present invention; wherein Figure 8 In the text, (a) and (b) indicate low contrast, (c) and (d) indicate emphasis on form mismatch, and (e) indicates category imbalance;

[0041] Figure 9 This is a comparison diagram of the segmentation visual effects of the semantic segmentation model in Embodiment 1 of the present invention with other major models; wherein Figure 9 In the figure, (a) is the input image, (b), (c), (d), (e), (f), (g), and (h) are the extraction results of U-Net, VisionTransformer, SegFormer, K-Net, SegNeXt, Mask2Former, and the method of the present invention, respectively; (i) is the dataset annotation. Detailed Implementation

[0042] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0043] Example 1

[0044] Currently, improving the accuracy of ocean internal wave extraction mainly faces the following three challenges:

[0045] like Figure 2 As shown in (a), the internal ocean wave characteristics are difficult to distinguish from the ocean background, and Fourier spectrum analysis shows that the high-frequency proportion is less than 15%, indicating a low contrast problem; Figure 2 As shown in (b), ocean internal waves exhibit a long, slender, and tilted morphological characteristic, which is morphologically mismatched with the receptive field of traditional square convolution; as Figure 2 As shown in (c), due to the small spatial coverage of ocean internal waves, the statistical results of the dataset used in this invention show that its pixels account for only about 2% of all pixels in the image, resulting in a serious class imbalance problem.

[0046] To address the aforementioned problems, and considering the characteristics of ocean internal waves in SAR remote sensing images, such as low contrast, elongated shape, and class imbalance, this invention proposes a method for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation. This method, based on frequency domain analysis and morphological matching, extracts ocean internal waves from SAR remote sensing images to improve extraction accuracy.

[0047] The method of this invention mainly includes three parts: data preprocessing, model building, and training and optimization.

[0048] Data preprocessing is used to filter, crop, augment, and standardize the label format of the raw SAR images to build a dataset suitable for neural network training and to ensure that ocean internal waves are correctly labeled.

[0049] In the model construction section, a semantic segmentation model based on an encoder-decoder structure was built. The preprocessed image was input into a multi-layer encoder of a neural network, and the multi-scale feature maps output by the encoder were input into a multi-layer decoder.

[0050] The encoder extracts feature maps of different scales and semantic levels through structures such as convolution, pooling, feature fusion, and attention modules, thereby simultaneously capturing the texture details of the inner waves and the global strip structure information.

[0051] The decoder learns the morphological features, contour structure and continuity of ocean internal waves through modules such as progressive upsampling, morphological matching and boundary refinement, and restores spatial resolution, thereby generating a predicted segmentation map corresponding to the input image.

[0052] In the training and optimization part, this invention constructs a combined loss function based on the characteristics of internal wave targets such as low contrast, slenderness and low pixel ratio. At the same time, it uses optimization strategies (such as learning rate scheduling, data balancing strategy, class weight adjustment, etc.) to update the network parameters of the model to improve the accuracy of the model in extracting ocean internal waves.

[0053] like Figure 1 As shown, the method for extracting internal ocean waves from SAR remote sensing images based on semantic segmentation includes the following steps:

[0054] Step 1. Acquire raw SAR remote sensing images, perform annotation and preprocessing operations, and construct a training dataset.

[0055] This embodiment performs labeling and preprocessing operations on SAR remote sensing images for ocean internal wave extraction. The preprocessing operations include segmentation, filtering, and random allocation to form a training dataset for training and testing.

[0056] Images acquired by various remote sensing satellites typically contain only raw echo information and lack precise pixel-level annotations for neural network training. Furthermore, there is currently a lack of standardized semantic segmentation datasets for ocean internal wave extraction tasks.

[0057] Therefore, this invention first performs pixel-level precise annotation on relevant SAR remote sensing images to ensure the accuracy of the spatial location and morphological structure of the ocean's internal wave region, providing reliable supervision information for subsequent model learning of slender strip structures.

[0058] After annotation, the image needs to be segmented. Since SAR remote sensing images usually have high spatial resolution and large swath width, directly inputting them into a neural network for training will not only significantly increase memory usage, but also weaken the model's ability to effectively model local elongated structures, leading to problems such as training instability, limited batch size, and difficulty in model convergence.

[0059] Therefore, in this embodiment, the original SAR remote sensing image is uniformly divided into 512×512 pixels, which improves the model's learning efficiency of local morphological features while ensuring the integrity of the local structure of the ocean wave stripes.

[0060] Furthermore, the task of extracting internal ocean waves suffers from a significant class imbalance problem, meaning that internal ocean waves account for an extremely low percentage of pixels in SAR remote sensing images. In the segmented samples, a large number of images contain no internal ocean waves or have a very small percentage of internal wave pixels. Retaining a suitable number of such samples helps improve the model's adaptability to complex ocean backgrounds, but if the proportion is too high, it weakens the model's learning effect on internal wave features. Therefore, this invention performs foreground proportion filtering and ratio control on the segmented images during the data construction stage. This ensures background diversity while avoiding excessive class imbalance, providing a more stable data foundation for subsequent model training.

[0061] Finally, a program was written in Python to randomly allocate the selected samples and construct a training dataset, namely the training set and the test set, for model training and performance evaluation, so as to ensure the objectivity and reproducibility of the experimental results.

[0062] Step 2. Construct a semantic segmentation model including an encoder and a decoder, such as... Figure 3 As shown, the encoder includes four feature extraction stages and a Frequency-Attention Enhanced Module (FAEM).

[0063] The four feature extraction stages are defined as the first, second, third, and fourth feature extraction stages (i.e., ... Figure 3 The process is divided into four stages: Stage 1, Stage 2, Stage 3, and Stage 4. Each feature extraction stage is sequentially connected and used to extract ocean internal wave features at different scales.

[0064] The output features of each feature extraction stage are input into the FAEM module. The FAEM module applies a frequency domain enhancement block, a feature pyramid network, and an attention mechanism to generate a frequency domain saliency probability map.

[0065] The generated frequency domain saliency probability map is multiplied by the pixel-level output features of the first, second, third, and fourth feature extraction stages, and residuals are concatenated to obtain the features of each feature extraction stage guided by the probability map.

[0066] The decoder includes three sequentially connected morphological enhancement blocks for morphological alignment modeling of features during the upsampling process, enabling the model to match the geometry of ocean internal waves as the spatial resolution is gradually restored.

[0067] Three morphological enhancement blocks are defined as the first, second, and third morphological enhancement blocks.

[0068] The features guided by the third and fourth feature extraction stages are input into the third morphological enhancement block; the output features of the third morphological enhancement block and the features guided by the second feature extraction stage are input into the second morphological enhancement block.

[0069] The output features of the second morphological augmentation block are input into the first morphological augmentation block along with the features guided by the first feature extraction stage; the outputs of the three morphological augmentation blocks are then concatenated to generate a predicted segmentation map corresponding to the model input image.

[0070] The following is a detailed description of the components in the semantic segmentation model's encoder and decoder.

[0071] First, at each feature extraction stage, a feature encoder is used to process the segmented dataset to extract ocean internal wave features at different scales. Then, in the encoder's multi-scale feature extraction stage, frequency domain analysis is introduced into the feature maps at each scale, and an attention mechanism is combined to generate a frequency domain saliency probability map to enhance the model's response capability to low-contrast ocean internal wave regions.

[0072] like Figure 2 As shown in (a), internal ocean waves are difficult to distinguish from the ocean background in SAR remote sensing images, and Fourier spectrum analysis reveals that their high-frequency energy proportion is usually less than 15%, exhibiting a significant low-contrast characteristic. This phenomenon indicates that feature extraction methods relying solely on spatial domain texture or pixel intensity are insufficient to fully characterize the discriminative features of internal ocean waves.

[0073] Based on the above analysis, this embodiment constructs a semantic segmentation model for ocean internal wave extraction.

[0074] In this embodiment, the model adopts a four-layer structure in the encoder feature extraction stage, and each layer uses a multi-scale convolutional attention network (MSCAN) for feature extraction.

[0075] MSCAN is a feature extraction module proposed in SegNeXt. Considering both computational efficiency and segmentation performance, this embodiment selects the MSCAN configuration used in SegNeXt-B. This invention does not simply reuse the MSCAN structure, but rather introduces frequency domain decomposition based on its output features, enabling multi-scale semantic features to simultaneously possess frequency response information.

[0076] The MSCAN structure contains multiple parallel branches, each capturing multi-scale contextual features through strip convolutions with different kernel sizes. Because strip convolutions better match the elongated geometry of ocean internal waves in terms of receptive field morphology, they are more conducive to the stable extraction of internal wave features compared to conventional square convolutions.

[0077] The four-layer MSCAN can output feature maps of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively, as shown in the formula:

[0078] .

[0079] Feature maps used in different feature extraction stages It means that, among them The time indicates the input image.

[0080] Next, the feature maps from the four scales are fed into the frequency domain – the attention enhancement module FAEM – for further processing. The overall structure of the FAEM module is as follows: Figure 4 As shown, this FAEM module does not introduce frequency domain processing separately at the input or decoding stage. Instead, it applies frequency domain modeling to the features at each scale simultaneously after the encoder has extracted multi-scale features but before the semantics are fully abstracted, so that the frequency information can continue to play a role in the subsequent multi-scale fusion process.

[0081] Frequency domain clues have a stronger ability to distinguish internal ocean waves under low contrast conditions.

[0082] Therefore, this embodiment sets up four frequency domain enhancement blocks in the frequency domain-attention enhancement module FAEM.

[0083] Each frequency domain enhancement block corresponds to one MSCAN output scale. These frequency domain enhancement blocks perform frequency domain feature modeling of multi-scale features to preserve discriminative information related to ocean internal waves to the maximum extent under low contrast conditions.

[0084] The FAEM module first applies frequency domain analysis to separate high-frequency and low-frequency features in the image, thereby improving the effective learning of internal wave features in the ocean. Next, it combines deep, low-resolution semantic features with shallow, high-resolution frequency domain results using a feature pyramid network to enhance extraction accuracy. Finally, an attention mechanism is applied to multi-scale feature fusion, generating a probability map that guides the model to focus on and learn internal wave features from the training samples.

[0085] This invention addresses the problem that existing methods based on spatial domain pixel or texture features struggle to effectively distinguish ocean internal waves from the background in low-contrast SAR images, and are prone to insufficient response of slender structures. It proposes a frequency domain-semantic-attention collaborative enhancement feature extraction mechanism and designs a FAEM module accordingly. Unlike existing methods that only perform frequency domain enhancement at the input or decoding stage, this invention directly embeds frequency domain analysis into the encoder's multi-scale feature extraction process, enabling frequency domain information to participate in modeling throughout the semantic hierarchical abstraction process, thus improving the representation capability of ocean internal waves under weak texture and low contrast conditions.

[0086] Specifically, the FAEM module includes a four-level frequency domain enhancement block, a four-level feature pyramid, and an attention module.

[0087] Each frequency domain enhancement block corresponds to the output feature of a feature extraction stage, and is used to perform frequency domain feature modeling on the features at the corresponding scale, thereby obtaining the frequency domain enhanced features at each scale.

[0088] The feature map output by the fourth-level frequency domain enhancement block is processed by convolution to obtain the intermediate features of the fourth-level feature pyramid. Then, it is further smoothed and locally enhanced, and then upsampled again to generate the output features of the fourth-level feature pyramid.

[0089] Except for the fourth-level feature pyramid, which is the last-level feature pyramid, each level of feature pyramid obtains its own feature pyramid fusion result by fusing the fusion result of the subsequent level feature pyramids and the output of the current level frequency domain enhancement block, and obtains its own feature pyramid output based on the fusion result of the current level feature pyramid.

[0090] Finally, the output of each feature pyramid level is input into the attention module to generate a probability map.

[0091] Each level of the feature pyramid network combines deep, low-resolution semantic features with shallow, high-resolution frequency domain results to perform multi-scale perceptual enhancement on the frequency-domain-enhanced features at the same level, thereby obtaining the fused features.

[0092] Because ocean internal waves in SAR images have similar textures to the surrounding background, traditional convolutional neural networks mainly rely on local spatial domain responses for feature extraction, which can easily lead to blurred edges, broken elongated structures, or insufficient responses in ocean internal waves. Especially under low-contrast conditions, it is difficult to explicitly characterize the frequency differences inherent in internal waves by simply relying on spatial domain modeling.

[0093] To address this, the present invention incorporates frequency domain analysis units at multiple scales of the encoder, decoupling and modeling different frequency components during the feature extraction stage by decomposing the feature map into high-frequency features. and low frequency characteristics And feature extraction is performed separately to achieve differentiated modeling of information at different frequencies. For example Figure 5 As shown, the processing flow of the frequency domain enhancement block is as follows:

[0094] First, the feature maps at each scale are decomposed into high-frequency features. and low frequency characteristics High frequencies are used to depict local texture details, while low frequencies are used to depict overall structural information, thereby enhancing the contrast between the ocean background and the ocean's internal wave region.

[0095] The feature map is decomposed using the following formula: ; ;

[0096] in Indicates the first Feature maps output from each feature extraction stage Perform a convolution operation. For convolution kernel; and Indicates the feature map Perform average pooling to adjust the resolution; .

[0097] This indicates that a convolution operation is performed on the feature map after average pooling. It is the convolution kernel.

[0098] The high-frequency component mainly includes the elongated texture, edges, and local amplitude abrupt changes of ocean internal waves, exhibiting higher discriminative power against low-contrast backgrounds. The low-frequency component, on the other hand, characterizes the overall background structure and energy distribution, helping to maintain the continuity and integrity of the internal wave morphology. By independently modeling and collaboratively fusing high-frequency and low-frequency features at the same scale, this invention effectively avoids the weakening of frequency information during deep semantic abstraction. Subsequently, by re-fusing the two at each scale, this invention effectively enhances the encoder's sensitivity to the details of internal wave morphology and significantly strengthens its feature representation capability.

[0099] This invention employs two consecutive octave convolutions to decouple high- and low-frequency information. Octave convolution explicitly splits high-frequency and low-frequency features at a spatial scale and performs convolutional modeling and bidirectional information exchange at different resolutions, effectively reducing spatial redundancy and improving feature representation efficiency. This process not only enhances discriminative power in the frequency dimension but also provides a more structured feature foundation for morphological alignment in the subsequent decoding stage.

[0100] The high-frequency components mainly contain information on the elongated texture, abrupt edge changes, and local amplitude variations of internal ocean waves; the low-frequency components reflect the overall background structure and energy distribution characteristics. Given that both high-frequency and low-frequency features are important for the extraction of internal ocean waves, they are fused in the final stage of the frequency domain enhancement block to obtain complete and complementary frequency domain clues at the current scale.

[0101] In the final stage of the frequency domain enhancement block, the high and low frequency information after octave convolution is fused to obtain complete and complementary frequency domain cues at the current scale. The fusion result is denoted as... The formula is expressed as follows: ; Indicates frequency domain enhancement, and These represent the high-frequency and low-frequency features obtained after two octave convolution processes, respectively.

[0102] By separating, modeling, and synergistically fusing different frequency components at multiple scale levels, the model proposed in this invention can more stably perceive the internal wave region of the ocean under low contrast conditions, thereby significantly improving the discriminative ability of feature representation.

[0103] Analysis of the SegNeXt structure shows that multi-scale information interaction is one of the key factors for achieving high-quality semantic segmentation. Even under the same spatial resolution, the scale of ocean internal waves in SAR images can still vary significantly due to differences in propagation intensity and spatial extent. Therefore, this embodiment constructs a multi-scale feature fusion module based on a top-down path, namely a feature pyramid structure, the specific form of which is as follows: Figure 6 As shown.

[0104] A multi-scale feature extraction network with a strip convolutional structure enables the convolutional receptive field to match the elongated shape of ocean internal waves along the long axis. Compared with conventional square convolution, it has higher structural adaptability and response stability for strip-shaped targets.

[0105] Unlike traditional methods that fuse only spatial domain features, the feature pyramid in this embodiment uses frequency domain enhanced features. As input, the multi-scale fusion process simultaneously incorporates semantic information, spatial details, and frequency response features.

[0106] To address the problem that ocean internal waves exhibit large scale variations and fine internal waves are easily overlooked in deep features in SAR images, this invention performs pyramid-style fusion of the frequency domain enhancement features output from each layer of the encoder.

[0107] Unlike traditional feature pyramids built solely based on spatial domain features, the pyramid fusion unit of this invention uses frequency domain enhanced features as the basic input unit, enabling the multi-scale semantic fusion process to simultaneously include spatial, semantic, and frequency domain information.

[0108] First, high-resolution and low-resolution features are added and fused under a unified channel dimension. Then, the upsampled features are convolved and smoothed to reduce the semantic differences between features of different scales. Finally, multi-scale feature sequences are generated step by step through a top-down path to form an effective complementarity between deep semantic information and shallow spatial detail information.

[0109] Specifically, the feature pyramid processing flow is as follows:

[0110] The feature map output from the fourth-level frequency domain enhancement block is convolved with a 1×1 convolution to obtain the intermediate features of the fourth-level feature pyramid. Then, the intermediate features are processed by 3×3 convolution. Smoothing and local enhancement are performed, followed by upsampling to generate output features. ;

[0111] ;

[0112] in This represents the output of the fourth-level feature pyramid. This indicates a 3×3 convolution process.

[0113] Except for the fourth-level feature pyramid, the processing flow for the third-level feature pyramid to the first-level feature pyramid is as follows:

[0114] No. The feature pyramid of the next level first performs a 1×1 convolution on the high-resolution feature map output by the frequency domain enhancement block of the same level to complete channel alignment. Then, it upsamples the intermediate features (fusion features) output by the next level feature pyramid, i.e., the low-resolution feature map, to match the spatial dimension. Finally, it fuses the features by adding elements one by one to obtain the fusion features of this level.

[0115] .

[0116] in Indicates the characteristics of subsequent fusion. This indicates the output result of the frequency domain enhancement block. Indicates the current level. Indicates the subsequent level, , ; Represents a 1×1 convolution. This indicates upsampling processing.

[0117] Subsequently, the fused features are smoothed and locally enhanced using 3×3 convolution, and then upsampled again to generate the output features. :

[0118] ;

[0119] in Indicates the first The output of the feature pyramid at level 1 This indicates a 3×3 convolution process.

[0120] The aforementioned fusion strategy, while maintaining the consistency of high-level abstract semantics, explicitly supplements the edge and texture details of shallow frequency domain enhancement, effectively alleviating the problem of internal waves being submerged by low-frequency background information during downsampling.

[0121] Compared with the traditional encoder-decoder structure, the feature pyramid constructed in this invention exhibits stronger scale robustness when facing ocean internal waves with different propagation intensities, lengths, and widths.

[0122] While maintaining high-level semantic consistency, this invention effectively supplements the edge and texture details of shallow frequency domain enhancement, thereby improving the stability and integrity of ocean internal wave extraction in multi-scale SAR scenarios.

[0123] To further guide the model to gradually increase its focus on the internal wave region of the ocean during training, this invention introduces an attention mechanism based on multi-scale fused features to construct a frequency domain saliency probability map and guide the feature response through the attention mechanism. Unlike existing attention mechanisms that mainly rely on channel statistics or spatial response distribution, the attention weights of this invention are jointly generated by multi-scale frequency domain enhanced features, so that the attention allocation is simultaneously constrained by both spatial location and frequency saliency.

[0124] The generation of this probabilistic map not only relies on spatial location or channel statistics, but also comprehensively considers the response distribution of multi-scale frequency domain enhancement features, enabling the attention weights to reflect the salience of potential internal wave regions in the frequency dimension. By adaptively adjusting the feature response weights across the full resolution range, this attention mechanism effectively enhances the model's ability to perceive the slender structures and weakly textured regions of internal waves, alleviating the accuracy degradation caused by the low proportion of internal wave pixels in the ocean.

[0125] The specific representation of the probability graph is shown in the formula: .

[0126] in, This represents the saliency probability plot used to guide the model. These are the fused features for each scale output by each feature pyramid. This is an attention mechanism. This attention mechanism can assign higher weights to the locations of potential ocean internal waves in areas with weak texture and low contrast, effectively alleviating the training offset problem caused by the low pixel proportion of internal waves in the entire SAR image, and improving the model's sensitivity and stability to sparse and slender targets.

[0127] The FAEM module does not achieve performance improvements by constructing entirely new network structures. Instead, it systematically integrates and customizes frequency domain modeling, multi-scale fusion, and attention-guided strategies to address the typical characteristics of ocean internal waves, such as low contrast, large scale variations, and elongated shapes. This unified design enhances the explicit modeling capability of different frequency components during the encoding stage, providing a stable feature foundation for morphology alignment during subsequent decoding and reconstruction. This results in higher accuracy and stronger robustness of ocean internal wave extraction in low-contrast SAR scenarios, demonstrating good engineering applicability and promotional value.

[0128] In SAR remote sensing images, ocean internal waves exhibit elongated and highly directional morphological characteristics. Traditional decoding methods struggle to effectively align the receptive field with the target geometry during feature reconstruction, often resulting in broken or blurred wave strip structures. This invention, based on the results guided by a probabilistic graph, introduces morphological enhancement blocks to explicitly introduce morphological constraints during the feature decoding stage, thereby achieving adaptive alignment between the model's receptive field and the actual propagation morphology of ocean internal waves.

[0129] This invention utilizes the probability map ω generated in step 2 as an explicit prior clue for the internal wave region of the ocean to positively guide the features. This allows the model to prioritize high-probability internal wave regions during the decoding stage, suppressing background interference and enhancing the internal wave structure response at the feature level, thereby providing stable guiding information for subsequent morphology alignment. The formula is expressed as follows:

[0130] .

[0131] in, This represents the feature map after attention guidance.

[0132] Indicates the first The feature maps output by each feature extraction stage This represents a probability plot of saliency in the frequency domain. This indicates a pixel-level multiplication operation. Indicates will With the Feature maps output from each feature extraction stage Perform residual connection.

[0133] Due to the propagation characteristics of internal ocean waves, they typically appear as thin, alternating bright and dark stripes in SAR remote sensing images. The high resolution of SAR images further enhances these thin stripe features. However, traditional square convolution kernels are not well-suited to the morphological characteristics of internal ocean waves. When modeling targets with significant directionality and constrained geometry, it is difficult to balance structural continuity and directional consistency, thus limiting the model's ability to effectively represent the thin propagation morphology of internal ocean waves.

[0134] To address this issue, this invention employs a Morphological-Enhanced Block (MEB) to model the morphological alignment of features during the upsampling process. This allows the model to match the geometry of ocean internal waves as spatial resolution is progressively restored. Traditional decoding structures based on fixed square convolutional kernels struggle to align with the elongated and highly directional geometry of ocean internal waves, easily introducing structural distortion and boundary blurring during upsampling and feature reconstruction. The MEB is introduced in the decoding stage to explicitly enhance the network's ability to model the morphological features of internal waves. Unlike existing methods that rely solely on multi-scale upsampling or skip connections to recover spatial details, this invention starts with the morphology of the convolutional receptive field and imposes structural constraints on the feature reconstruction process during the decoding stage, enabling the network to actively align with the true geometry of the target during reconstruction.

[0135] like Figure 3 As shown, the feature decoder consists of three identical morphological enhancement blocks, and each morphological enhancement block has the same structure, such as... Figure 7 As shown. Dynamic Snake Convolution (DSConv) simulates the undulating motion of a snake to better align with the shape of slender objects, and is particularly adept at extracting slender and sloping structures like ocean internal waves. This operation is divided into vertical snake convolution and horizontal snake convolution, as shown... Figure 2 As shown in (b) of the diagram.

[0136] However, existing serpentine convolution assumes that convolution branches in different directions have equal importance, failing to consider the fact that ocean internal waves typically have a dominant propagation direction in actual imaging. Under complex background conditions, this equal weight assumption is prone to introducing redundant responses independent of the dominant direction of the internal waves, thus affecting the shape alignment effect.

[0137] To address this issue, this invention constructs a morphological enhancement block that employs a channel attention mechanism to dynamically allocate weights to convolutional branches in different directions. This allows the model to adaptively select the receptive field direction that better matches the current ocean internal wave morphology based on input features. Its three convolutional branches are: horizontal serpentine convolution (H), vertical serpentine convolution (V), and square convolution (S). The square convolution branch, as a supplementary path, helps maintain the stability of feature representation in regions with uncertain orientation or complex morphology, avoiding excessive reliance on a single-direction assumption. Its structure is as follows: Figure 8 As shown.

[0138] like Figure 7 As shown, the morphological enhancement block in this embodiment includes three parallel convolutional branches and one channel attention.

[0139] Three parallel convolutional branches are defined as dynamic serpentine convolution in the horizontal direction, dynamic serpentine convolution in the vertical direction, and standard square convolution, so that the receptive field shape can be adaptively adjusted according to the feature response.

[0140] The processing flowcharts for each convolutional branch are as follows: .

[0141] in For the first The processing results of each branch, This represents the feature map input to the three parallel convolutional branches, i.e., the feature map after the fourth feature extraction stage guided by the probabilistic map or the output feature map of the next morphological enhancement block.

[0142] Indicates to Perform different convolution kernels Convolution operation, The three values ​​correspond to three parallel convolution branches.

[0143] Three morphological enhancement blocks are cascaded in the decoder. Each morphological enhancement block mainly contains three parallel branches: horizontal dynamic serpentine convolution, vertical dynamic serpentine convolution, and standard square convolution. These branches can adaptively adjust the receptive field morphology according to the feature response, thereby improving the matching ability of ocean internal wave geometry. Among them, the receptive field of DSConv's convolution kernel can simulate the spatial offset characteristics generated during serpentine motion, breaking through the limitations of traditional convolution kernel regular sampling. This gives the convolution operation a certain degree of morphological flexibility in space, making it more suitable for characterizing slender and tilted ocean internal wave structures.

[0144] However, the original DSConv model assumes equal importance for the horizontal and vertical directions, failing to fully consider the significant anisotropy of internal wave propagation directions in real SAR scenarios. This can easily lead to redundancy or inadequacy in directional modeling. To address this, this invention introduces a channel attention mechanism into the MEB (Multi-channel Array). Based on the actual distribution of feature responses, it adaptively learns the importance weights of the three convolutional branches, enabling the network to dynamically adjust the contribution levels of different directional components. This maintains the generalization ability of square convolutions while highlighting the response to the main internal wave direction structure.

[0145] Channel attention generates attention weights for the three parallel convolutional branches to dynamically adjust their contributions, thereby enhancing the model's adaptability to ocean internal wave morphology. The formula is expressed as follows:

[0146] .

[0147] in This represents the feature map input to the channel attention block. This represents a multilayer perceptron with one hidden layer. This represents the sigmoid function. This indicates max pooling.

[0148] Each channel can be ranked according to its corresponding attention score. The weight of the channel in the merged result is dynamically adjusted so that the receptive field is aligned as closely as possible with the morphology of the ocean internal waves, while minimizing the noise and negative impact that may be introduced by other channels.

[0149] in This is an array of three values, representing the real-time weights of the three parallel convolutional branches. By dynamically adjusting the receptive field in different directions, this structure enables the model to achieve fine alignment with the true geometry of ocean internal waves during the decoding stage, reducing background noise interference, and maintaining the continuity and integrity of the strip structure during progressive upsampling.

[0150] Next, the results of the three parallel convolutional branches, after dynamically adjusted weights, are concatenated, as expressed in the following formula:

[0151] .

[0152] in This is the result of concatenating the dynamically adjusted weights of the three parallel convolutional branches. This means concatenating the results processed by the three parallel convolutional branches; For the first The weights of each branch.

[0153] The spliced ​​features The result after convolution is concatenated with the result after convolution of the features extracted from the corresponding feature extraction stage guided by the probabilistic graph, to obtain the result of convolution. Output features of each morphological enhancement block :

[0154] .

[0155] in , For the first Features obtained through probabilistic graph guidance during the feature extraction stage.

[0156] Output features in the above manner It can adaptively enhance the directional component consistent with the geometry of ocean internal waves while fusing information from different convolutional morphologies, effectively suppressing interference noise from the background region, and maintaining the continuity and integrity of the strip structure during subsequent step-by-step upsampling and feature reconstruction.

[0157] By introducing morphological enhancement blocks, the decoding stage no longer relies solely on scale recovery, but also possesses the ability to explicitly align the target shape, thereby significantly improving the boundary accuracy and structural consistency of the final segmentation result.

[0158] Specifically, taking the third morphological enhancement block as an example, one input to the third morphological enhancement block is the feature from stage 4 guided by the probabilistic graph. This feature is first processed sequentially through three parallel convolutional branches, and then the results of the three parallel convolutional branches, after dynamically adjusted weights, are concatenated. In addition, the third morphological enhancement module also has another input: the feature from stage 3 guided by the probabilistic graph. This feature, after convolution processing, is further concatenated with the result of concatenated features after convolution processing, thus obtaining the output feature of the first morphological enhancement block.

[0159] Similarly, one input to the second morphological augmentation block is the output feature of the third morphological augmentation block, which is then processed through three parallel convolutional branches and concatenated with the weighted results. Furthermore, the second morphological augmentation block also receives the feature from stage 2 guided by a probabilistic graph. This feature, after convolutional processing, is further concatenated with the concatenated feature to obtain the output feature of the second morphological augmentation block.

[0160] Similarly, one input to the first morphological augmentation block is the output feature of the second morphological augmentation block, which is then processed through three parallel convolutional branches and concatenated with the weighted results. Furthermore, the first morphological augmentation block also receives the feature from stage 1 guided by the probabilistic graph. This feature, after convolutional processing, is further concatenated with the concatenated feature to obtain the output feature of the first morphological augmentation block.

[0161] This invention splices the results of three parallel convolutional branches after dynamically adjusting their weights, enabling the semantic segmentation model to more accurately align with the true geometric shape of ocean internal waves, reduce interference noise from the background region, and maintain the continuity and integrity of the strip structure in subsequent stepwise upsampling and feature reconstruction processes.

[0162] This invention utilizes morphological enhancement blocks to enable the model to restore spatial details while possessing the ability to align the morphology of ocean internal wave features, effectively improving the boundary accuracy and structural consistency of the final segmentation result. Therefore, the morphological enhancement blocks in this invention implement a morphological recognition mechanism completed during the feature decoding stage, which, unlike traditional methods that rely on post-processing or loss correction, improves the stability and accuracy of ocean internal wave extraction at the structural level.

[0163] The outputs of the three morphological enhancement blocks are concatenated to generate a predicted segmentation map corresponding to the model input image.

[0164] Step 3. Train the model based on the training dataset, and use the trained model to extract ocean internal waves from SAR remote sensing images.

[0165] In the task of extracting internal ocean waves, the extraction process can be viewed as a pixel-level binary classification problem. However, due to the extremely low proportion of internal waves in the image, the task faces severe class imbalance, making it difficult for the traditional binary cross-entropy (BCE) loss to effectively optimize model performance. Specifically, in SAR images, the proportion of pixels in the internal ocean wave region is extremely low, and there is a severe imbalance between the foreground and background. In such class imbalance scenarios, the traditional binary cross-entropy loss is easily dominated by a large number of background pixels, resulting in poor model training performance.

[0166] The Dice loss, by directly measuring the overlap between the predicted mask and the real mask, effectively improves the model's sensitivity to the overall structure of the foreground region and prevents the inner wave region from being submerged by the background. The Focal loss, on the other hand, enhances the learning ability of difficult-to-classify samples by reducing the weight of easily classified background samples, making the network pay more attention to fine structures and boundary regions.

[0167] By combining the global structural constraint capability of Dice loss with the hard-sample focusing characteristic of Focal loss, a target optimization mechanism that simultaneously considers structural integrity and local discriminativity is constructed. This mechanism is then implemented through hyperparameters. and By balancing the contribution ratios of Dice loss and Focal loss, robust optimization is achieved for intrinsic wave characteristics such as foreground sparsity, weak contrast, and elongated boundaries.

[0168] Specifically, Dice loss directly optimizes the overlap between the predicted results and the ground truth labels, making the model pay more attention to sparse and elongated inner wave foreground structures; Focal loss further improves the model's ability to identify subtle inner wave structures and their boundaries by reducing the weight of easily classified background pixels and emphasizing difficult-to-classify samples such as low-contrast pixels.

[0169] By weighting and fusing Dice and Focal losses using adjustable hyperparameters, the class imbalance problem is alleviated while achieving more stable and accurate extraction of ocean internal waves. The model's loss function... The definition is as follows:

[0170] .

[0171] Through the above loss design, the semantic segmentation model can not only cope with the class imbalance problem, but also further enhance its ability to distinguish low-contrast inner wave structures and their boundary regions. Moreover, the loss function can form a collaborative optimization mechanism with the FAEM module and morphological enhancement block, providing constraints for the model to learn stable inner wave structure features.

[0172] In addition, to verify the effectiveness of the method proposed in this invention, experimental verification will be carried out below, and the following comparison methods will be introduced: U-Net, Vision Transformer, SegFormer, K-Net, SegNeXt, and Mask2Former.

[0173] In 2022, Tao M. released the IWs dataset for object detection, which contains 390 SAR images clearly showing internal ocean waves, taken by the Sentinel-1 satellite. However, there is currently no publicly available dataset specifically designed for training neural network models to extract internal ocean waves. Therefore, based on the aforementioned dataset, a custom-annotated, segmented, filtered, and randomly distributed dataset was created, called SSIW-SAR.

[0174] SSIW-SAR sample images and annotations are as follows: Figure 8 As shown. Among them Figure 8 In the diagram, (a) and (b) represent low contrast, (c) and (d) represent emphasized shape mismatch, and (e) represents class imbalance. The original images and their corresponding ground truth labels were divided into 512×512 image patches. Based on the object detection bounding boxes (YOLO format) provided in the original dataset, this invention filtered out image patches that mostly did not contain any internal ocean wave features or contained only a very small number of IWs. Ultimately, a total of 8,024 valid image patches were obtained. Of these, 7,000 were used for training, and the remaining 1,024 were allocated to the validation set.

[0175] To evaluate the effectiveness of the proposed method in ocean internal wave extraction, commonly used evaluation metrics were employed. These metrics include mean intersection-over-union ratio (mIoU), F1 score (F1), mean accuracy (mAcc), mean precision (mPrecision), mean recall (mRecall), and mean F1 score (mF1). The proposed method was compared with several classic general semantic segmentation models. The comparison results are shown in Table 1, where all data are percentages.

[0176] Table 1 Comparison of evaluation indicators between the method of the present invention and the comparative method

[0177]

[0178] The results in Table 1 above demonstrate that the method of this invention outperforms general segmentation networks (such as U-Net and SegNeXt) and several ocean internal wave extraction techniques across all evaluation metrics. These results highlight the superior performance of the method of this invention and confirm its state-of-the-art effectiveness in ocean internal wave extraction. Compared to SegNeXt as a benchmark, the method of this invention achieves significant performance improvements across multiple metrics, including an 11% increase in mIoU, a 20% increase in F1 score, and a 12% increase in mAcc, primarily attributed to its superior ability to extract low-contrast ocean internal waves.

[0179] Extraction results of various methods, such as Figure 9 As shown, this invention compares the extraction performance of several representative methods, including classic architectures such as U-Net and ViT, and semantic segmentation models such as SegFormer, K-Net, SegNeXt, and Mask2Former. Differences in extraction results between different methods are indicated by red boxes, while yellow boxes highlight areas ignored in the ground truth label annotations. Figure 9 (a) in the image is the input image. Figure 9 (b) in the image represents the extraction result from U-Net. Figure 9 (c) in the image represents the extraction result from the Vision Transformer. Figure 9 (d) in the table represents the extraction result from SegFormer. Figure 9 (e) in the figure represents the extraction result of K-Net. Figure 9 (f) in the figure represents the extraction result of SegNeXt. Figure 9 In the table, (g) represents the extraction result from Mask2Former. Figure 9 The (h) column in the figure represents the extraction results of this embodiment. Figure 9 In this context, (i) represents the dataset label.

[0180] like Figure 9As shown in the first row, the internal ocean waves exhibit relatively high contrast with the background. Although some areas in the upper left corner are annotated in the ground truth labels, their features are not readily apparent. The method of this invention successfully extracted these weak structures, demonstrating its ability to capture weak signals. The second row shows a particularly elongated internal ocean wave in the lower right corner, which is more difficult to segment compared to other internal ocean waves in the image. The method of this invention successfully extracted this structure, highlighting its robustness in detecting elongated objects. The third row presents a scene similar to the first row, but with the internal ocean waves oriented differently. The method of this invention continues to accurately extract the internal ocean wave region, further validating its generalization ability. In the fourth and fifth rows, the contrast between the internal ocean waves and the background is very low, making them difficult to distinguish. The comparison results show that the method of this invention can extract internal ocean waves to the maximum extent under low contrast conditions. In particular, in the fifth row, the area marked by the yellow box shows the broken internal ocean wave segments in the original annotations.

[0181] The extraction results show that this invention can automatically identify and extract this structure, indicating that it has effectively learned the core features of internal ocean waves. The model proposed in this embodiment can leverage its frequency enhancement and morphological adaptation capabilities to learn more robust or refined internal ocean wave features than those captured by manual annotation. The semantic segmentation model proposed in this invention can detect weak signals easily overlooked in complex human backgrounds, or continuous structures missed during annotation due to subjective judgment. This may help correct incomplete areas in the original labels, thereby facilitating the accurate extraction of internal ocean waves from SAR remote sensing images.

[0182] Example 2

[0183] This embodiment 2 describes a system for extracting internal ocean waves from SAR remote sensing images based on semantic segmentation. This system is based on the same inventive concept as the above-mentioned method for extracting internal ocean waves from SAR remote sensing images based on semantic segmentation.

[0184] The system for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation in this embodiment includes the following modules:

[0185] The preprocessing module is used to acquire raw SAR remote sensing images, perform annotation and preprocessing operations, and build a training dataset;

[0186] The ocean internal wave extraction module is used to build a semantic segmentation model that includes an encoder and a decoder;

[0187] The encoder consists of four feature extraction stages, namely the first, second, third, and fourth feature extraction stages, as well as a frequency domain-attention enhancement module (FAEM). The four feature extraction stages are connected in sequence and are used to extract ocean internal wave features at different scales.

[0188] The output features of each feature extraction stage are input into the FAEM module; the FAEM module applies frequency domain enhancement blocks, feature pyramid networks, and attention mechanisms to generate a frequency domain saliency probability map;

[0189] The generated frequency domain saliency probability map is multiplied by the output features at the pixel level of the first, second, third, and fourth feature extraction stages, and residuals are concatenated to obtain the features of each feature extraction stage after being guided by the probability map.

[0190] The decoder includes three sequentially connected morphological enhancement blocks for morphological alignment modeling of features during the upsampling process, enabling the model to match the geometry of ocean internal waves while gradually restoring spatial resolution.

[0191] Define three morphological enhancement blocks as the first, second, and third morphological enhancement blocks;

[0192] The features guided by the third and fourth feature extraction stages are input into the third morphological enhancement block; the output features of the third morphological enhancement block and the features guided by the second feature extraction stage are input into the second morphological enhancement block;

[0193] The output features of the second morphological enhancement block and the features guided by the first feature extraction stage are input into the first morphological enhancement block; the output results of the three morphological enhancement blocks are concatenated to finally generate a predicted segmentation map corresponding to the model input image;

[0194] The model is trained based on the training dataset, and the trained model is used to extract ocean internal waves from SAR remote sensing images.

[0195] It should be noted that the implementation process of the functions and roles of each functional module in the system described in this embodiment 2 is detailed in the implementation process of the corresponding steps in the method of embodiment 1 above, and will not be repeated here.

[0196] Example 3

[0197] This embodiment 3 describes a computer device including a memory and one or more processors. Executable code is stored in the memory. When the processor executes the executable code, it implements the steps of the method for extracting internal ocean waves from SAR remote sensing images based on semantic segmentation in embodiment 1 above.

[0198] In this embodiment, the computer device can be any device or apparatus with data processing capabilities, and will not be described in detail here.

[0199] Example 4

[0200] This embodiment 4 describes a computer-readable storage medium storing a program that, when executed by a processor, is used to implement the steps of the method for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation in embodiment 1 above.

[0201] The computer-readable storage medium can be an internal storage unit of any device or apparatus with data processing capabilities, such as a hard disk or memory, or an external storage device of any device with data processing capabilities, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc.

[0202] Of course, the above description is only a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. It should be noted that any equivalent substitutions or obvious modifications made by those skilled in the art under the guidance of this specification fall within the scope of this specification and should be protected by the present invention.

Claims

1. A method for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation, characterized in that, Includes the following steps: Step 1. Acquire raw SAR remote sensing images, perform annotation and preprocessing operations, and construct a training dataset; Step 2. Construct a semantic segmentation model including an encoder and a decoder; The encoder includes four feature extraction stages: the first, second, third, and fourth feature extraction stages, and a frequency domain-attention enhancement module (FAEM). The four feature extraction stages are connected in sequence and are used to extract ocean internal wave features at different scales. The output features of each feature extraction stage are input into the FAEM module; The FAEM module introduces frequency domain analysis and combines it with an attention mechanism to generate a frequency domain saliency probability map; The generated frequency domain saliency probability map is multiplied by the output features at the pixel level of the first, second, third, and fourth feature extraction stages, and residuals are concatenated to obtain the features of each feature extraction stage after being guided by the probability map. The decoder includes three sequentially connected morphological enhancement blocks for morphological alignment modeling of features during the upsampling process, enabling the model to match the geometry of ocean internal waves while gradually restoring spatial resolution. Define three morphological enhancement blocks as the first, second, and third morphological enhancement blocks; The features guided by the third and fourth feature extraction stages are input into the third morphological enhancement block; the output features of the third morphological enhancement block and the features guided by the second feature extraction stage are input into the second morphological enhancement block; The output features of the second morphological enhancement block are input into the first morphological enhancement block after the feature extraction stage is guided by the first feature extraction stage. The outputs of the three morphological enhancement blocks are concatenated to generate a predicted segmentation map corresponding to the model input image; Step 3. Train the model based on the training dataset, and use the trained model to extract ocean internal waves from SAR remote sensing images.

2. The method for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation according to claim 1, characterized in that, Step 1 specifically involves: First, pixel-level annotations are performed on the original SAR remote sensing images. After annotation, the images are segmented into 512×512 pixels. Next, the segmented images are filtered and proportioned based on the foreground proportion. Finally, the filtered images are randomly assigned to construct a training dataset, including a training set and a test set.

3. The method for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation according to claim 1, characterized in that, In step 2, the multi-scale convolutional attention network MSCAN is used to extract features in each feature extraction stage.

4. The method for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation according to claim 1, characterized in that, In step 2, the FAEM module includes a four-level frequency domain enhancement block, a four-level feature pyramid, and an attention module; Each frequency domain enhancement block corresponds to the output feature of a feature extraction stage, and is used to perform frequency domain feature modeling on the features at the corresponding scale, thereby obtaining the frequency domain enhanced features at each scale; The feature map output by the fourth-level frequency domain enhancement block is processed by convolution to obtain the intermediate features of the fourth-level feature pyramid. Then, it is further smoothed and locally enhanced, and then upsampled again to generate the output features of the fourth-level feature pyramid. Except for the fourth-level feature pyramid, which is the last-level feature pyramid, each level of feature pyramid obtains its intermediate features by fusing the intermediate features of the subsequent level feature pyramids and the output of the current level frequency domain enhancement block. The intermediate features of the current level feature pyramid are then smoothed, locally enhanced, and upsampled to obtain the output of the current level feature pyramid. Finally, the output of each feature pyramid level is input into the attention module to generate a probability map.

5. The method for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation according to claim 4, characterized in that, The processing flow of the frequency domain enhancement block is as follows: First, the feature maps at each scale are decomposed into high-frequency features. and low frequency characteristics The formula is expressed as follows: ； ； in Indicates the first Feature maps output from each feature extraction stage Perform convolution operations. For convolution kernel; and Indicates the feature map Perform average pooling to adjust the resolution; ; This indicates that a convolution operation is performed on the feature map after average pooling. For convolution kernel; High- and low-frequency information are decoupled and modeled by two consecutive octave convolutions. Octave convolution explicitly splits high-frequency features on a spatial scale. Low-frequency characteristics Convolutional modeling and bidirectional information interaction are performed at different resolutions. In the final stage of the frequency domain enhancement block, the high and low frequency information after octave convolution is fused to obtain complete and complementary frequency domain cues at the current scale. The fusion result is denoted as... The formula is expressed as follows: ; Indicates frequency domain enhancement; and These represent the high-frequency and low-frequency features obtained after two octave convolution processes, respectively.

6. The method for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation according to claim 4, characterized in that, The processing flow of the feature pyramid is as follows: The feature map output from the fourth-level frequency domain enhancement block is convolved with a 1×1 convolution to obtain the intermediate features of the fourth-level feature pyramid. Then, the intermediate features are processed by 3×3 convolution. Smoothing and local enhancement are performed, followed by upsampling to generate output features. : ； in This represents the output of the fourth-level feature pyramid. This indicates a 3×3 convolution process; The fourth level feature pyramid, excluding The processing flow of the feature pyramid at level 1 is as follows: No. The feature pyramid of the first level first performs a 1×1 convolution on the feature map output by the frequency domain enhancement block of the same level to complete channel alignment. Then, it upsamples the intermediate features output by the subsequent feature pyramid to match the spatial dimension. Then, it further fuses the features by adding elements one by one to obtain the intermediate features of the feature pyramid of this level, i.e., the fused features. ； in This indicates the fusion characteristics of the subsequent stage. This indicates the output result of the frequency domain enhancement block. Indicates the current level. Indicates the subsequent level, , ; Represents a 1×1 convolution. Indicates upsampling processing; Subsequently, the fused features are smoothed and locally enhanced using 3×3 convolution, and then upsampled again to generate the output features. : ； in Indicates the first The output of the feature pyramid at level 1 This indicates a 3×3 convolution process.

7. The method for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation according to claim 4, characterized in that, The attention module constructs a frequency domain saliency probability map based on multi-scale fusion features, and its formula is as follows: ； in This represents the significance probability plot used to guide the model. These are the fused features output by each feature pyramid for each scale. This is an attention mechanism.

8. The method for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation according to claim 1, characterized in that, In step 2, the formula for the features after probabilistic graph guidance at each feature extraction stage is as follows: ； in Indicates the first The feature maps output by each feature extraction stage This represents a probability plot of saliency in the frequency domain. This indicates a pixel-level multiplication operation. Indicates will With the Feature maps output from each feature extraction stage Perform residual connection; .

9. The method for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation according to claim 1, characterized in that, In step 2, the morphological enhancement block includes three parallel convolutional branches and one channel attention; Three parallel convolutional branches are defined as dynamic serpentine convolution in the horizontal direction, dynamic serpentine convolution in the vertical direction, and standard square convolution, so as to adaptively adjust the receptive field shape according to the feature response. The processing flowcharts for each convolutional branch are as follows: ; in For the first The processing results of each branch, This represents the feature map input to the three parallel convolutional branches, i.e., the feature map after the fourth feature extraction stage is guided by the probabilistic map or the output feature map of the morphological enhancement block; Indicates to Perform different convolution kernels Convolution operation, The three values ​​correspond to three parallel convolution branches; Channel attention generates attention weights for three parallel convolutional branches, expressed by the following formula: ； in This represents a multilayer perceptron with one hidden layer. This represents the sigmoid function. This indicates max pooling; It is an array of three values, representing the real-time weights of the three parallel convolutional branches; Next, the results of the three parallel convolutional branches, after dynamically adjusted weights, are concatenated, as expressed in the following formula: ； in This is the result of concatenating the dynamically adjusted weights of the three parallel convolutional branches. This means concatenating the results processed by the three parallel convolutional branches; For the first The weights of each branch; The spliced ​​features The result after convolution is concatenated with the result after convolution of the features extracted from the corresponding feature extraction stage guided by the probabilistic graph, to obtain the result of convolution. Output features of each morphological enhancement block : ； in , For the first Features obtained through probabilistic graph guidance during the feature extraction stage.

10. The method for extracting ocean internal waves from SAR remote sensing images based on semantic segmentation according to claim 1, characterized in that, In step 3, during model training, the following loss function is defined: ;in This represents the total loss function of the model. for loss, for Loss, hyperparameters and Used for balance Loss and The proportion of the loss contributed by each party.

Images