Semi-automatic construction method of ocean internal wave segmentation data set based on SAR (Synthetic Aperture Radar) image
By combining traditional image processing and deep learning model iterative optimization methods, the efficiency and accuracy issues in constructing ocean internal wave segmentation datasets were solved, achieving efficient and automated dataset construction that adapts to the diversity of internal wave morphology and background, thereby improving the scale and quality of the dataset.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV
- Filing Date
- 2026-01-19
- Publication Date
- 2026-04-28
AI Technical Summary
Existing technologies are inefficient, inaccurate, reliant on manual labor, and difficult to scale when constructing ocean internal wave segmentation datasets. They also cannot adapt to the diversity of internal wave morphology and background, resulting in inconsistent annotations and high noise levels.
By combining traditional image processing techniques with deep learning models, initial samples are generated using Gaussian matching filters and double threshold segmentation. The U-Net semantic segmentation model is then used for iterative optimization, data augmentation, and sample selection, forming a semi-automated dataset construction process.
It significantly improves the efficiency and accuracy of dataset construction, reduces labor costs, and enhances the scale and generalization ability of datasets, forming a virtuous cycle of self-evolution.
Smart Images
Figure CN121937898A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of remote sensing image processing and artificial intelligence technology, and in particular to a semi-automatic method for constructing an ocean internal wave segmentation dataset based on SAR imagery. Background Technology
[0002] Ocean internal waves are a type of large-scale gravity wave that occurs within the density gradient of seawater. Their amplitude can reach tens or even hundreds of meters, and their crests often extend for tens to hundreds of kilometers. Internal waves have a significant impact on marine engineering construction, maritime navigation safety, underwater target detection, and marine environmental monitoring. Therefore, the rapid and accurate identification and monitoring of ocean internal waves is of great importance.
[0003] Synthetic Aperture Radar (SAR) offers advantages such as all-weather, all-day imaging and cloud penetration, making it the most important and reliable remote sensing method for monitoring internal ocean waves. Internal waves in SAR images typically appear as alternating bright and dark, elongated, continuous stripes with distinct linear characteristics. To automatically extract internal wave information from SAR images, a high-precision pixel-level semantic segmentation model is crucial and forms an important foundation for building an automated internal wave monitoring system.
[0004] However, training high-quality deep learning models relies on a large amount of accurate pixel-level labeled data. Currently available internal wave datasets are mostly in the form of object detection (bounding boxes), which are difficult to describe the true wavefront structure of internal waves and are not suitable for fine segmentation tasks. At present, the construction of pixel-level internal wave datasets mainly relies on manual visual interpretation and delineation, which is inefficient, costly, and inconsistent in quality, and difficult to apply in large-scale scenes. Although traditional image processing methods (such as edge detection and filtering) can assist in extracting linear structures to some extent, they often lack the ability to generalize and cannot adapt to the diversity of internal wave morphology, intensity, and background, resulting in incomplete extraction results or the inclusion of a large amount of noise, still requiring a lot of manual intervention.
[0005] Therefore, existing technologies still suffer from problems such as low efficiency, insufficient accuracy, reliance on manual labor, and difficulty in scaling up when constructing internal wave segmentation datasets. There is an urgent need for an automated or semi-automated pixel-level annotation method that can improve annotation efficiency, enhance annotation consistency, and adapt to the linear structural features of internal waves, so as to meet the demand for high-quality marine internal wave segmentation data for deep learning model training and engineering applications. Summary of the Invention
[0006] This invention aims to address the problems of low annotation efficiency, unstable accuracy, and difficulty in scalable construction in existing technologies by proposing a semi-automatic method for constructing an ocean internal wave segmentation dataset based on SAR imagery. This method effectively solves problems such as limited samples, annotation difficulties, and interference from internal wave analogues by combining traditional image processing techniques with iterative optimization strategies using deep learning models. It can effectively improve the quality of initial samples and continuously improve the dataset size and annotation accuracy through a model-driven data augmentation mechanism.
[0007] To achieve the above technical objectives, the present invention adopts the following technical solution:
[0008] A semi-automatic method for constructing an ocean internal wave segmentation dataset based on SAR imagery, the method comprising:
[0009] S1 acquires SAR images containing ocean internal wave characteristics, and linearly stretches the pixels in the image whose backscattering coefficient is within a preset range in the grayscale space. The stretched image is used as the original image.
[0010] S2 uses Gaussian matched filters in different directions to enhance the original image; the enhanced image is segmented by double thresholds to extract internal waves as labels and construct initial internal wave samples;
[0011] S3 performs augmentation processing on the initial internal wave samples to construct an initial internal wave segmentation dataset;
[0012] S4 uses the initial inner wave segmentation dataset to train a deep learning model and obtain the initial inner wave segmentation model;
[0013] S5 performs inference prediction on the new SAR original image and existing samples based on the initial internal wave segmentation model; the correctly predicted internal wave areas of the new SAR original image are used as new positive samples to supplement the incorrectly predicted non-internal wave areas as negative samples to supplement the original samples; and the internal wave areas predicted by the existing samples with higher accuracy than the original labels are used to replace the corresponding labels of the original samples.
[0014] S6 repeats S3-S5 until the quality and size of the dataset meet the preset standards, thus obtaining the final ocean internal wave segmentation dataset.
[0015] In some embodiments of the present invention, the pixel values of pixels with backscattering coefficients in the image range of -35 dB to 0 dB are linearly stretched in the grayscale value range of 0–255 to enhance the contrast between the internal waves and the background seawater, which facilitates subsequent feature enhancement, threshold segmentation and deep learning model training.
[0016] In some embodiments of the present invention, the enhancement process includes: performing convolution operations on the original image using two-dimensional Gaussian matched filters in different directions, taking the maximum response value of each pixel in all directions, and obtaining an enhanced image.
[0017] In some embodiments of the invention, the enhancement process uses 18 two-dimensional Gaussian matched filters in different directions, spaced 10° apart.
[0018] In some embodiments of the present invention, the dual threshold segmentation includes:
[0019] Based on the statistical characteristics of the enhanced image, two thresholds, high and low, are set to perform dual-threshold segmentation;
[0020] Pixels above the high threshold are marked as inner waves, and pixels below the low threshold are marked as background;
[0021] For pixels between high and low thresholds, a connectivity test is performed between them and the inner wave pixels. If a pixel is connected to an inner wave pixel, it is marked as an inner wave pixel; otherwise, it is marked as background.
[0022] In some embodiments of the present invention, the augmentation process includes:
[0023] The sample images were linearly fused with pure seawater images without internal waves at different scales to generate multiple new samples; and,
[0024] The samples are randomly rotated and flipped to generate multiple new samples.
[0025] In some embodiments of the present invention, the deep learning model is selected from the U-Net semantic segmentation model.
[0026] In some embodiments of the present invention, after extracting the internal wave as a label from the enhanced image through dual threshold segmentation, the original image and the corresponding label are visually inspected and corrected to form an initial internal wave sample.
[0027] In some embodiments of the present invention, the SAR image is selected as a VV polarization image in the interferometric wide-swath imaging mode.
[0028] In some embodiments of the present invention, the SAR images are uniformly resampled to a preset resolution.
[0029] The present invention has the following beneficial effects:
[0030] (1) This invention transforms the tedious pixel-level annotation work into a mode of “preliminary algorithm extraction + manual correction + model iterative optimization”, which significantly improves the efficiency of dataset construction and reduces the cost and subjective error of pure manual annotation.
[0031] (2) Inner waves are accurately located through multi-directional GMF and double threshold segmentation, and the initial quality is ensured by manual inspection. More importantly, the powerful feature learning capability of deep learning models is used to continuously optimize the fineness of existing labels during the iteration process and to discover weak feature inner waves that are difficult to find through traditional methods, thereby continuously improving the overall accuracy of the dataset.
[0032] (3) By data augmentation and model reasoning in multiple samples and multiple scenarios, new positive samples and representative negative samples (false alarms) are continuously introduced, so that the dataset covers richer internal wave morphology, intensity changes and ocean background, which is conducive to training a segmentation model with stronger generalization ability.
[0033] (4) The iterative framework designed in this invention enables the dataset construction process to have self-evolution capabilities. The improvement of model performance directly drives the improvement of dataset quality and the expansion of its scale, while a better dataset further promotes the improvement of model performance, forming a virtuous cycle. Attached Figure Description
[0034] Figure 1 The overall flowchart of the semi-automatic construction method for ocean internal wave segmentation dataset based on SAR imagery provided by this invention is shown.
[0035] Figure 2 This is a schematic diagram of the semi-automatic generation process of the initial positive sample in this invention (corresponding to step S2).
[0036] Figure 3 This is a schematic diagram of the dataset augmentation method (corresponding to step S3) in this invention.
[0037] Figure 4 This is a schematic diagram of the dataset optimization and expansion process based on the model iteration strategy in this invention (corresponding to steps S4-S6).
[0038] Figure 5 This is a display of the Sulu Sea internal wave segmentation results and partial sample results of the dataset in this invention. Detailed Implementation
[0039] To facilitate understanding and implementation of this invention by those skilled in the art, the following describes the implementation of this invention in further detail, using Suluhai as an example, to semi-automatically construct an inner wave segmentation dataset:
[0040] S1. Data Preparation and Preprocessing:
[0041] From publicly available ocean internal wave SAR target detection datasets, 138 Sentinel-1 SAR imagery images covering the Sulu Sea were selected for the study area. All selected images are VV polarization data acquired in Sentinel-1 satellite interferometric wide swath (IW) mode, with spatial resolution resampled to 100 meters.
[0042] Pixels in the image with backscattering coefficients ranging from -35 dB to 0 dB are subjected to grayscale stretching with pixel values (grayscale values) ranging from 0 to 255. Pixel values below -35 dB are uniformly mapped to 0, and pixel values above 0 dB are uniformly mapped to 255. This enhances the grayscale contrast between the inner wave stripes and the background seawater, providing standardized input for subsequent feature enhancement and pixel-level segmentation. The spatial resolution is 100 meters.
[0043] S2. Semi-automatic generation of initial positive samples:
[0044] This step aims to efficiently generate high-quality initial inner-wave pixel-level labels.
[0045] S21. Gaussian Matched Filter (GMF): A two-dimensional Gaussian matched filter is constructed to address the characteristic of internal waves appearing as bright and dark stripes in SAR images.
[0046] The Gaussian matched filter is a one-dimensional Gaussian second derivative along the x-axis, repeated along the y-axis in the neighborhood:
[0047]
[0048] Where σ is the standard deviation of the second derivative of the Gaussian function, and its magnitude is related to the width of the inner wave fringes, and L is the neighborhood length of the GMF.
[0049] The parameters of the Gaussian matched filter are set based on the image characteristics and width of the inner wavy lines. In this embodiment, the standard deviation of the Gaussian function σ is set to 3, and the filter size is 80 pixels × 17 pixels.
[0050] S22. Multi-directional convolution: To match the various possible orientations of the internal waves, this invention rotates the GMF along the 0 to π direction. The resulting coordinate transformation is:
[0051]
[0052] The filter for the direction of rotation is:
[0053]
[0054] in, It is a GMF rotated counterclockwise by θ°, which has a peak response at θ±π / 2.
[0055] In this embodiment, the GMF is rotated from 0° to 180° at 10° intervals, generating a total of 18 filters in different directions. The original SAR image is then convolved with each of these 18 filters:
[0056]
[0057] S23. Obtain the maximum response image: For each pixel, take the maximum value among the convolution results in 18 directions to form an enhanced maximum response image.
[0058]
[0059] The image significantly highlights the internal wave fringe information while suppressing some background noise.
[0060] S24. Double threshold segmentation: To avoid the breakage of weak signal internal waves caused by single threshold segmentation, the double threshold method is adopted.
[0061] Calculate the mean m and standard deviation σ of the image with the maximum response, and set a high threshold. low threshold Based on experience, k1=1.3 and k2=1.5 are set. Pixel areas above the high threshold are designated as inner wave regions, pixel areas below the low threshold are designated as background regions, and pixel areas between the high and low thresholds are designated as potential inner wave regions. It is determined whether potential inner wave regions are connected to inner wave regions. If they are connected, they are designated as inner wave regions; otherwise, they are designated as background regions.
[0062]
[0063] Where N(x,y) represents the neighboring pixels of (x,y), and S(x',y')=1 indicates that it is connected to the existing inner wave pixels.
[0064] S25. Manual Correction: Perform manual visual inspection on the inner wave mask obtained by automatic segmentation, and manually remove obvious erroneous detection results caused by coastlines, islands, ships, etc., to ensure the accuracy of the initial positive sample labels.
[0065] S3, Data Augmentation:
[0066] Data augmentation was performed on the internal wave positive samples (images and corresponding labels) obtained from S2 to expand the data volume and improve model robustness. The methods employed included:
[0067] Mixup: This method merges image patches containing positive internal wave samples (as the foreground) with pure seawater areas without internal waves (as the background) at different scales.
[0068]
[0069] Where λ is the mixing ratio.
[0070] In this embodiment, image blocks are linearly fused at ratios of 7:3 and 3:7 to generate new training samples. This simulates the performance of internal waves under different background brightness levels.
[0071] Geometric transformations: Randomly rotate the sample (e.g., at 45° intervals) and flip it horizontally / vertically.
[0072] The enhanced data are combined to form the initial internal wave segmentation dataset D0.
[0073] S4. Initial Model Training:
[0074] The U-Net semantic segmentation model was selected as the basic architecture and trained using the dataset D0 to obtain the initial inner wave segmentation model M0.
[0075] S5. Iterative optimization and expansion of the dataset:
[0076] This step iteratively optimizes the dataset through model inference.
[0077] S51, Model Inference: Use the current best model (initially M0) to perform two aspects of inference: Inference A: Perform full-scene inference on new SAR images (378 scenes from 2014 to 2022) that have not participated in training to find new internal waves.
[0078] Inference B: Re-infer the original SAR image corresponding to the labeled positive sample to obtain the model-predicted, potentially more refined, internal wave region.
[0079] S52. Sample Screening and Storage:
[0080] For the results of inference A, after rapid manual verification, correctly predicted internal wave regions are added to the dataset as new positive samples. Regions that are incorrectly predicted as internal waves (such as special sea conditions or cloud edges) are false positives, which are negative samples that the model has difficulty distinguishing. These are added to the dataset as new negative samples.
[0081] For the results of inference B, the predicted inner wave labels are visually judged, and the more accurate inner wave regions are selected and segmented as the optimized positive sample labels, replacing or correcting the original corresponding labels.
[0082] S53. Form a new dataset: merge the newly added positive and negative samples with the optimized samples to supplement and update the original dataset D0.
[0083] S6, Iterative Loop:
[0084] Repeat steps S3-S5. After augmenting the dataset with the new data, we obtain an updated and expanded dataset D1. We then retrain the internal wave segmentation model to obtain a more powerful model M1. We use M1 for inference optimization to obtain D2, and then train D2 to obtain M2, repeating this process. Each iteration allows the dataset and model to mutually improve. After five iterations, the dataset contains 500 positive internal wave samples and 500 negative samples of various types. The overall model accuracy reaches 0.89, and the temporal coverage is relatively uniform. It achieves a high level in terms of scale, quality, and diversity, at which point the iteration terminates. Finally, we obtain a high-precision pixel-level SAR internal wave segmentation dataset for the Sulu Sea containing multiple types of samples.
[0085] This invention effectively overcomes the bottleneck of manually constructing internal wave segmentation datasets through the aforementioned semi-automatic, iterative framework, providing a solid data foundation for the accurate extraction of internal wave information based on deep learning.
[0086] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A semi-automatic method for constructing an ocean internal wave segmentation dataset based on SAR imagery, characterized in that, The method includes: S1 acquires SAR images containing ocean internal wave characteristics, and linearly stretches the pixels in the image whose backscattering coefficient is within a preset range in the grayscale space. The stretched image is used as the original image. S2 uses Gaussian matched filters in different directions to enhance the original image; the enhanced image is segmented by double thresholds to extract internal waves as labels and construct initial internal wave samples; S3 performs augmentation processing on the initial internal wave samples to construct an initial internal wave segmentation dataset; S4 uses the initial inner wave segmentation dataset to train a deep learning model and obtain the initial inner wave segmentation model; S5 performs inference prediction on the new SAR original image and existing samples based on the initial internal wave segmentation model; the correctly predicted internal wave areas of the new SAR original image are used as new positive samples to supplement the incorrectly predicted non-internal wave areas as negative samples to supplement the original samples; and the internal wave areas predicted by the existing samples with higher accuracy than the original labels are used to replace the corresponding labels of the original samples. S6 repeats S3-S5 until the quality and size of the dataset meet the preset standards, thus obtaining the final ocean internal wave segmentation dataset.
2. The method according to claim 1, characterized in that, The pixel values of pixels in the image with backscattering coefficients in the range of -35 dB to 0 dB are linearly stretched in the grayscale value range of 0–255.
3. The method according to claim 1, characterized in that, The enhancement process includes: performing convolution operations on the original image using two-dimensional Gaussian matched filters in different directions, and taking the maximum response value of each pixel in all directions to obtain the enhanced image.
4. The method according to claim 1, characterized in that, The enhancement process uses 18 two-dimensional Gaussian matched filters in different directions, spaced 10° apart.
5. The method according to claim 1, characterized in that, The dual threshold segmentation includes: Based on the statistical characteristics of the enhanced image, two thresholds, high and low, are set to perform dual-threshold segmentation; Pixels above the high threshold are marked as inner waves, and pixels below the low threshold are marked as background; For pixels between high and low thresholds, a connectivity test is performed between them and the inner wave pixels. If a pixel is connected to an inner wave pixel, it is marked as an inner wave pixel; otherwise, it is marked as background.
6. The method according to claim 1, characterized in that, The augmentation process includes: The sample images were linearly fused with pure seawater images without internal waves at different scales to generate multiple new samples; and, The samples are randomly rotated and flipped to generate multiple new samples.
7. The method according to claim 1, characterized in that, The deep learning model used is the U-Net semantic segmentation model.
8. The method according to claim 1, characterized in that, After extracting internal waves as labels from the enhanced image through dual-threshold segmentation, the original image and corresponding labels are visually inspected and corrected to form an initial internal wave sample.
9. The method according to claim 1, characterized in that, The SAR image used is a VV polarization image in the interferometric wide-swath imaging mode.
10. The method according to claim 1, characterized in that, The SAR images are uniformly resampled to a preset resolution.