Marine scene segmentation method based on condition invariant semantic segmentation

Abstract

The invention discloses an offshore scene segmentation method based on condition invariant semantic segmentation, and belongs to the field of computer vision and semantic segmentation. According to the method, style mutual conversion of a source domain image and a target domain image is realized through a data loading stage, and a multi-style view pair is generated; a CISS network model is constructed, the CISS network model is composed of a MiT-B5 encoder and a context sensing decoder, the encoder extracts high-level features insensitive to visual conditions by means of discrete wavelet transform, and the decoder achieves feature reconstruction without information loss through inverse discrete wavelet transform and generates a category probability graph; designing a joint loss function including pixel-level cross entropy loss and feature-level invariance loss, and balancing loss weight in combination with a dynamic weighting strategy; and training and optimizing the model by adopting an AdamW optimizer in combination with a linear learning rate warm-up strategy. Experiments show that the method can effectively cope with visual condition changes in a real marine environment, and the robustness of the semantic segmentation model in different marine environments is improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and semantic segmentation technology, and in particular to a method for segmenting maritime scenes based on conditional invariant semantic segmentation. Background Technology

[0002] In maritime scenarios, semantic segmentation is a core technology for achieving autonomous ship operation and environmental perception. However, different visual conditions can lead to significant changes in image style, lighting, and contrast. Traditional semantic segmentation models have insufficient generalization ability under adverse conditions, resulting in problems such as decreased segmentation accuracy and poor robustness.

[0003] In existing semantic segmentation techniques, unsupervised domain adaptation methods improve model adaptability by aligning the feature or pixel distribution of the source domain (normal conditions) and the target domain (adverse conditions). For example, adversarial training-based methods align features between domains through generative adversarial networks, but these methods have high computational complexity and limited robustness to specific disturbances such as lighting and fog in maritime scenes. Pixel-level stylization-based methods achieve domain alignment through image style transfer, but this only stays at the pixel level and does not fully utilize the invariance of high-level semantic features.

[0004] Therefore, there is an urgent need for a semantic segmentation method for maritime scenes that can extract high-level semantic features that are invariant to visual conditions, while also balancing efficiency and accuracy, in order to improve the robustness of semantic segmentation models under different maritime environments. Summary of the Invention

[0005] The problem to be solved by this invention is to provide a marine scene segmentation method based on conditional invariant semantic segmentation, which aims to handle the problem of visual condition changes in real marine environments and improve the robustness of semantic segmentation models in different marine environments.

[0006] This invention adopts the following technical solution: a maritime scene segmentation method based on conditional invariant semantic segmentation, characterized by comprising the following steps:

[0007] S1. Construct a marine dataset and training batch containing ships, waves, coastlines, and clouds. Obtain the source domain image set and the target domain image set. The number of source domain images and target domain images in the training batch is equal.

[0008] S2. Generate a stylized view: Using shallow stylization mapping, the source domain image and the target domain image are converted into each other in style, preserving key semantic information such as ship outline (low-frequency structure) and wave texture (high-frequency details) to obtain a stylized view;

[0009] S3. Construct a CISS (Condition-Invariant Semantic Segmentation) network model: Extract input image features and generate semantic segmentation results through an encoder-decoder network. A matched wavelet pooling layer is introduced into the encoder, and the feature map is decomposed into multiple resolutions through Discrete Wavelet Transform (DWT) to preserve low-frequency structures and high-frequency details related to the semantics of the maritime scene. The output is a high-level feature containing spectral information, which extracts spectral features including low-frequency contours of ships (LL sub-band), high-frequency textures of ocean waves (HH sub-band), and edge details of the coastline (HL / LH sub-band). For the characteristics of the maritime scene, a sub-band attention module is added after the DWT layer to dynamically adjust the sub-band weights. The decoder performs reconstruction without information loss and generates a category probability map based on the high-level features.

[0010] S4. Define the loss function, including pixel-level semantic supervision loss and feature-level invariant loss. Introduce a dynamic weighting strategy to construct a joint loss function and automatically adjust the weight balance between pixel-level semantic supervision loss and feature-level invariant loss.

[0011] Training and optimization of S5 and CISS network models: The optimizer is used in conjunction with a linear learning rate warm-up strategy to optimize the model; during training iterations, pixel-level cross-entropy loss and feature-level invariance loss are optimized simultaneously to enable the encoder to extract style-invariant high-level features and generate semantic segmentation results through the decoder.

[0012] Preferably, in step S1, the marine dataset is collected in an environment with unsupervised domain adaptive semantic segmentation settings. Images of the marine environment under different weather, lighting, and seasonal conditions are acquired through a camera. Various objects in the images (such as ships, ocean, clouds, etc.) are labeled, and data augmentation methods (such as rotation, scaling, flipping, color change, etc.) are used to enhance the diversity of the dataset and divide it into training set, validation set, and test set.

[0013] The source domain image corresponds to normal visual conditions (such as sunny daytime) and is equipped with a pixel-level true semantic label Y. s ;

[0014] The target domain image, corresponding to adverse visual conditions (such as night or fog), is used to generate pseudo-labels through self-training using the DACs framework.

[0015] Preferably, in step S2, the shallow stylization mapping is used to retain the content information of the input image and fuse the style features of the reference image to obtain a stylized view pair;

[0016] Convert the source domain image into a target domain style view. s→t The formula is:

[0017] I s→t =g(I s ,I t );

[0018] Convert the target domain image into a source domain style view. t→s The formula is:

[0019] I t→s =g(I t ,I s );

[0020] Where g is the shallow stylization mapping function, I s For the source domain image, I t For the target domain image.

[0021] Preferably, the shallow stylization mapping includes, but is not limited to: Fourier domain adaptation (FDA), color transfer algorithm, and color dithering;

[0022] The Fourier domain adaptation achieves style transfer by replacing the low-frequency Fourier amplitudes in the source domain image with the low-frequency Fourier amplitudes in the target domain image.

[0023] Preferably, in step S3, the encoder-decoder network adopts a high-resolution domain adaptation architecture (HRDA), decomposing the segmentation network F into an encoder φ and a decoder ω:

[0024]

[0025] in, The function F represents the composite operation of the function, i.e., F = ω(φ()), which means that the segmentation network F is a composite function composed of the decoder ω and the encoder φ.

[0026] In the feature extraction process of the encoder-decoder network, a matching wavelet pooling layer is introduced. By preserving the spectral information through matching wavelet pooling, high-frequency textures (such as wave details) and low-frequency structures (such as ship outlines) under different visual conditions (such as fog and night) in the marine scene are effectively captured.

[0027] The encoder φ is a MiT-B5, composed of DWT layers. It applies four DWT filters to the input feature map via two-dimensional discrete wavelet transform for convolution. Each DWT filter f... mn After convolution with the input data D, a 2x downsampling is performed to obtain 4 subbands, as shown in the following formula corresponding to the convolutional layer + downsampling operation:

[0028] D mn =Conv2D(D,f mn stride=2)

[0029] Among them, D mn Represents the feature map of the m-th row and n-th column sub-band obtained after DWT decomposition, Conv2D represents the two-dimensional convolution operation, and stride represents the stride of the convolution operation;

[0030] Form a feature representation φ(I) that includes spectral information:

[0031] φ(I)∈R C×M×N

[0032] Where C is the number of channels and M×N is the size of the input feature map.

[0033] The decoder ω is a context-aware feature fusion module, composed of an IDWT layer. It upsamples the sub-bands output by the encoder (e.g., through bilinear interpolation) to the original size and passes them through a conjugate filter. Deconvolution reconstructs the feature map and generates a class probability map. The following formula corresponds to the upsampling layer + deconvolution operation:

[0034]

[0035] in, For IDWT filters, and DWT filters f mn Conjugate, scale, upsampling factor, UpSample indicates the upsampling operation, which is used to increase the spatial resolution of the feature map.

[0036] The encoder decomposes the input feature map into one low-frequency sub-band (LL, preserving structural information) and three high-frequency sub-bands (LH, HL, HH, preserving texture and edge details) through two-dimensional discrete wavelet transform. The size of each sub-band is 1 / 2 of the input feature map.

[0037] Preferably, for representing high-frequency textures of ocean waves (such as the HH subband) and representing low-frequency contours of ships (such as the LL subband), a subband attention module is added after the DWT layer. The weights of the four subbands are dynamically adjusted through channel attention. The weights of the HL and HH subbands are increased in nighttime scenes, and the weights of the LL subband are increased in foggy scenes, thereby enhancing the ability to express key spectral features.

[0038] Preferably, in step S4, the pixel-level semantic supervision loss applies weighted cross-entropy loss to both the original source domain image and the target domain image to learn source domain semantic knowledge and adapt it to the target domain through pseudo-labels, as shown below:

[0039]

[0040]

[0041] Where C is the number of categories, H×W is the image size, and α cThe weight parameter for category c is used to emphasize the learning of key categories such as ships and coastlines.

[0042] The feature-level invariant loss applies to stylized view pairs (I) s ,I s→t ) and (I t ,I t→s Normalization based on Frobenius norm loss is applied to reduce the impact of brightness differences on feature amplitude, as shown below:

[0043]

[0044] Where ε is a constant to prevent division by zero, and φ(I′) represents the feature representation extracted by encoder φ from the style-transformed image I′.

[0045] Here, I′ can be either I s→t (Source domain image I) s (Convert to target domain style), or it can be I t→s (Target domain image I) t (Convert to source domain style), ||·|| F It is the Frobenius norm.

[0046] Preferably, a joint loss function is constructed, and a dynamic weighting strategy is introduced to minimize the overall loss, as follows:

[0047]

[0048] Where, λ s , λ t The weighting parameter for feature invariance loss is set as a dynamic weighting function based on the average brightness or contrast of the image, to adapt the invariance constraint strength under complex lighting conditions at sea (such as reflections and shadows) and non-uniform weather disturbances (such as local sea fog), including sunny days, foggy days, rainy days, and night.

[0049] Preferably, in step S5, the optimizer is the AdamW optimizer, with preset encoder and decoder learning rates, and iterative training is performed.

[0050] In the initial stage of training, a linear warm-up is performed to gradually increase the learning rate from 0 to the preset value;

[0051] During training, each batch of inputs consists of the original source domain image, the target domain image, and a stylized view. After calculating the loss, backpropagation is performed, and the encoder and decoder parameters are updated alternately until convergence.

[0052] Preferably, in step S5, the training process includes the following sub-steps:

[0053] S5.1 Data Loading: Each batch contains B=2 pairs of source-target domain images, dynamically generating stylized view I. s→t I t→s This forms four sets of image inputs (I s ,I t ,I s→t ,I t→s );

[0054] S5.2 Forward Propagation: Extracting features from four sets of images using the encoder. The corresponding segmentation prediction is generated through the decoder;

[0055] S5.3 Loss Calculation: Calculate pixel-level semantic supervision loss, including: source domain ground truth label supervision. Target domain pseudo-label adaptation Calculate the feature-level invariant loss, including: source domain feature alignment. Target domain feature alignment The overall loss was incurred;

[0056] S5.4 Backpropagation: The parameters are updated through the AdamW optimizer. The encoder parameters are updated with a low learning rate to stabilize feature extraction, and the decoder optimizes segmentation details with a high learning rate.

[0057] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0058] 1. The marine scene segmentation method of the present invention forces the encoder to ignore visual condition differences through feature invariance loss, extracts general semantic features, ensures that the features are invariant to the input style, and significantly improves the segmentation accuracy of the target domain.

[0059] 2. The marine scene segmentation method of the present invention adopts shallow stylization technology, including: FDA and color transfer. During the data loading stage, the source domain and target domain images are mutually converted in style to generate multi-style views of the same scene, avoiding complex generation models, maintaining computational efficiency, and is suitable for real-time marine scene processing.

[0060] 3. The marine scene segmentation method of the present invention introduces a matching wavelet pooling layer during the feature extraction process. By preserving spectral information through matching wavelet pooling, it effectively captures high-frequency textures and low-frequency structures under different visual conditions in marine scenes. Attached Figure Description

[0061] Figure 1 This is a flowchart illustrating the steps of the marine scene segmentation method based on conditional invariant semantic segmentation of the present invention.

[0062] Figure 2 This is a flowchart illustrating the structure of the conditional invariant semantic segmentation method in this embodiment of the invention. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the application will be further described in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments involved in this invention. All non-innovative embodiments based on these embodiments by other researchers in the art are within the protection scope of this invention. Furthermore, the step numbers in the embodiments of this invention are only set for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0064] This invention discloses a maritime scene segmentation method based on condition-invariant semantic segmentation. The method uses cameras on ships to capture maritime scenes and performs unsupervised semantic segmentation of maritime scenes using a condition-invariant semantic segmentation method, thereby improving the robustness of the model in segmentation under adverse conditions such as day and night and fog.

[0065] In one embodiment of the present invention, a maritime scene segmentation method based on conditionally invariant semantic segmentation, such as... Figure 1 As shown, the steps are as follows:

[0066] Step 1: Collect marine images under various real-world marine conditions, including clear labels for the sea surface, ships, and coastline, to create a marine scene dataset. The dataset includes a source domain and a target domain. The source domain dataset contains multiple classes of pixel-level real-world labels Y. s The target domain dataset generates pseudo-labels through self-training.

[0067] In this embodiment, the target domain pseudo-labels are generated using the DACs framework, with a confidence threshold set to 0.9. Hybrid sampling and data augmentation are performed on the target domain image to improve the quality of the pseudo-labels and select highly reliable labels.

[0068] Step 2: Generate a stylized view by using shallow stylization mapping g, such as Fourier domain adaptation (FDA) or color transfer algorithms, to preserve the content information of the input image and fuse the stylistic features of the reference image.

[0069] Source domain image I s Convert to target domain style view I s→t =g(I s ,I t Preserve the source domain content and integrate the target domain style; transform the target domain image I t Convert to source domain style view I t→s =g(I t ,I s (), retain the content of the target domain and integrate the style of the source domain.

[0070] Specifically, in this embodiment, Fourier domain adaptation is used to generate stylized views, transforming the source domain style into the target domain style:

[0071]

[0072] in, Represents the Fourier amplitude spectrum. denoted by , ⊙ represents the Fourier phase spectrum, ⊙ represents the Hadamard product (element-wise multiplication of matrices), and M is a low-pass filter (cutoff frequency β = 0.01), which preserves the low-frequency style of the target domain and the high-frequency content of the source domain.

[0073] Step 3: Build the CISS network model. The model architecture and configuration are built using a maritime scene segmentation method based on conditionally invariant semantic segmentation, such as... Figure 2 As shown.

[0074] The CISS network structure encoder uses MiT-B5 (SegFormer backbone network), and the decoder is a context-aware feature fusion module that integrates the multi-scale features output by the encoder to generate a segmentation probability map.

[0075] The encoder introduces a matched wavelet pooling layer during feature extraction, and performs multi-resolution decomposition of the feature map through Discrete Wavelet Transform (DWT). This preserves high-frequency detail coefficients and low-frequency approximation coefficients in the horizontal, vertical, and diagonal directions, effectively capturing high-frequency textures (such as wave details) and low-frequency structures (such as ship outlines) under different visual conditions (such as fog and night) in marine scenes, forming a feature representation containing spectral information. Wavelet pooling is implemented through a learnable wavelet filter bank, which avoids the loss of high-frequency details caused by traditional pooling during downsampling, adapts to spectral changes caused by factors such as lighting and fog in marine scenes, and enhances the ability of features to represent complex visual conditions.

[0076] In this embodiment, the input image size is 512×512, and multi-scale high-level features are extracted. The number of channels D = 768, and the feature map size M × N = 16 × 16.

[0077] In this embodiment, during the encoder stage, four learnable or fixed wavelet filters (such as Haar wavelet filters) are applied to the input feature map through the DWT layer. LL ,f LH ,f HL ,f HH Convolution is performed. Downsampling decomposes the input feature map into four sub-bands: one low-frequency approximation sub-band (LL, preserving structural information) and three high-frequency detail sub-bands (LH / HL / HH, preserving texture and edge details), outputting a multi-resolution spectral feature φ(I)∈R. D×M×N .

[0078] The filter matrix is ​​as follows:

[0079]

[0080] Each filter is convolved with the input data D, and then downsampled by a factor of 2 to obtain 4 subbands:

[0081]

[0082] in, This represents a convolution operation with 4 subbands D. DWT The dimensions are half of the original data (width and height are each halved), that is...

[0083] Furthermore, the decoder ω consists of an IDWT layer, which upsamples the subband features output by the encoder to their original size and passes them through a conjugate filter. Deconvolution reconstructs the feature map and generates a class probability map.

[0084] In this embodiment, IDWT reconstructs the original data D using four subbands. Taking Haar wavelet as an example, the reconstruction formula is:

[0085]

[0086] Among them, D LL For low-frequency subband, D LH D HLL D HH For high-frequency subband

[0087] In particular, the core operation of the CISS network structure is to restore the subband features to their original size through upsampling (2x) and filter deconvolution, thereby achieving reconstruction without information loss.

[0088] Furthermore, for high-frequency textures of ocean waves (such as the HH subband) and low-frequency contours of ships (LL subband), a subband attention module is added after the DWT layer. The weights of LL / LH / HL / HH are dynamically adjusted through channel attention (e.g., the weights of HL / HH are increased to 0.8 in nighttime scenes and the weights of LL are increased to 0.6 in foggy scenes) to enhance the expressive power of key spectral features.

[0089] By selecting and retaining LL+HL+HH subbands through learnable subband selection to handle edge detection in nighttime scenes, robust feature extraction of lighting and weather changes in maritime scenes is achieved.

[0090] Step 4: Define the loss function, including pixel-level semantic supervision loss and feature-level invariant loss.

[0091] Pixel-level semantic supervision loss: for the original source domain image Is and target domain image I t Cross-entropy loss is applied to ensure that the model learns semantic knowledge of the source domain and adapts to pseudo-labels of the target domain. This is to improve the model's segmentation robustness in complex maritime scenes, especially when segmenting target objects such as coastlines and ship hulls under low contrast, blurry, or hazy conditions.

[0092] This embodiment introduces a class weight mechanism in the pixel-level semantic supervision loss. For each pixel in the image, a corresponding weight is assigned according to its true class. Common classes (such as sea surface) are assigned low weight, while rare or misclassified classes (such as ship outline and coastline) are assigned high weight, thereby enhancing the model's ability to learn key semantics.

[0093] The weighted cross-entropy loss is defined as follows:

[0094]

[0095] Where C is the number of categories, H×W is the image size, and α c These are the weight parameters for category c.

[0096] Based on the shallow stylization mapping in step 2, stylized view pairs are obtained: (I s ,I s→t ) and (I t ,I t→s ), because I s→t It is presented in the style of the target domain, and therefore aligned with the target domain.

[0097] In this embodiment, the basic settings are modified to change the original source image I. s Replace with the stylized source image I s→t The stylized image is applied to the cross-entropy loss associated with the source domain, where stylization can be achieved through any shallow mapping:

[0098]

[0099] Furthermore, feature invariance loss is used to minimize the difference between the encoder output of the original and stylized views, ensuring that features are invariant to the input style and combined with shallow stylization to achieve cross-domain robustness.

[0100] Feature-level invariance loss for stylized view pairs (I s ,I s→t ) and (I t ,I t→sThe original Frobenius norm loss is used for normalization. By dividing the difference between the two input feature maps by the sum of their respective amplitude norms, the interference of brightness and scale changes on feature comparison is effectively reduced, allowing the model to focus more on the real differences in structure and texture, and forcing the encoder to output visually invariant features.

[0101] The improved feature invariant loss is expressed as follows:

[0102]

[0103] Where D is the number of feature channels, M×N is the feature map size; ||·||F is the Frobenius norm, used to calculate the differences between features; ε is a constant to prevent division by zero, usually taken as 1e -6 This loss minimizes the difference in features before and after style changes, prompting the encoder to output features that are insensitive to changes in visual conditions.

[0104] Adding a normalized denominator reduces errors caused by brightness differences due to sea surface reflection in marine scenes, making it more suitable for handling the "overall brightness shift" problem in nighttime and foggy images.

[0105] Joint loss function: In the specific training process, feature invariance loss and cross-entropy loss are used together. Cross-entropy loss is used for supervision of source domain and target domain images respectively, while cross-entropy loss on stylized images is replaced by feature invariance loss to ensure the invariance of features to style changes.

[0106] In this embodiment, returning to the UDA settings, in the image pair (I s ,I s→t ) and (I t ,I t→s Feature invariant loss is applied to align the internal features of each pair of image views. A dynamic weighting strategy is introduced so that the loss function can automatically adjust the weight balance between semantic loss and feature invariant loss according to the visual characteristics of the input image (such as brightness and contrast).

[0107] The two generated feature invariance losses are combined with the cross-entropy loss in the basic settings, and the final loss formula in the maritime scene segmentation method based on conditional invariant semantic segmentation is as follows:

[0108]

[0109] Wherein, the hyperparameter λ s , λ t Adjust the feature invariance weights, such as setting them to 50 and 20 for sunny day → foggy day, and 100 and 10 for sunny day → night.

[0110] λ can be mapped using functions: in, This represents the average brightness of the image.

[0111] Step 5: Training and optimization of the CISS network model.

[0112] In this embodiment, the optimizer AdamW parameters are set with a weight decay of 0.01 and a preset encoder learning rate of 6×10⁻⁶. -5 The decoder learning rate is 6×10. -4 Linear warm-up with 500 iterations; for the weights of the loss function, considering day / night domain adaptation, let λ... s =100 (strong source region characteristic invariance in response to drastic changes in illumination), λ t =10.

[0113] The training process includes: data loading, forward propagation, loss calculation, and backpropagation. Each batch of input is the original and stylized views. After calculating the loss, backpropagation is performed, and the encoder and decoder parameters are updated alternately until convergence.

[0114] Specifically, the steps are as follows:

[0115] The data loading section reads the source domain image I. s (with Y) s ) and target domain image I t (bring ), generating stylized views I through FDA s→t ,I t→s .

[0116] The forward propagation part extracts four sets of image features from the encoder: φ(I s ),φ(I t ),φ(I s→t ),φ(I t→s The decoder generates four sets of segmentation predictions: F(I) s ),F(I t ),F(I s→t ),F(I t→s ).

[0117] The pixel-level loss calculation part includes: (Source Domain Real Tag Supervision) (Target domain pseudo-label adaptation), the feature-level loss includes: (Source domain feature alignment) (Target domain feature alignment), therefore the overall loss is:

[0118] The backpropagation process updates parameters using the AdamW optimizer, the encoder uses a low learning rate to stabilize feature extraction, and the decoder uses a high learning rate to optimize segmentation details.

[0119] Furthermore, in this embodiment, a virtual environment for training the model is built on a GPU server using Anaconda. The training set is input into the network structure of the maritime scene segmentation method based on conditional invariant semantic segmentation for semantic segmentation model training. After training, a deep learning model for maritime scene segmentation is obtained. The validation set is input into the deep learning model of the maritime scene segmentation method based on conditional invariant semantic segmentation for validation. Based on the validation results, the deep learning model and training parameters are optimized, and finally the deep learning model for maritime scene segmentation with the best performance is obtained.

[0120] The model training parameters in this embodiment are set as follows: The model is based on the HRDA architecture, using a MiT-B5 encoder (SegFormer backbone) and a context-aware feature fusion decoder. The input size is 512×512. DeepLabv2 (ResNet101 backbone) is also supported for comparison experiments. The optimizer is AdamW (weight decay 0.01), and the encoder learning rate is set to 6×10. -5 Stable high-level feature extraction, decoder learning rate 6×10. -4 To accelerate semantic prediction optimization, a linear learning rate warm-up strategy is employed (gradually increasing the learning rate over the first 500 iterations). The default training batch size is 2, which can be increased as needed, containing an equal number of source and target domain images. A stylized view is generated using shallow stylization techniques (default FDA, β = 0.01, which can be replaced with color transfer or color dithering). The target domain data is enhanced using a DACs framework (mixed sampling, pseudo-label filtering with confidence ≥ 0.9). In the loss function, the feature invariance loss weights are adjusted according to the task, such as λ under multiple adverse conditions. s =50, λ t =20, λ when adapting to day and night conditions s =100, λ t =10 to enhance source domain feature alignment.

[0121] After verification, the semantic segmentation performance of the maritime scene segmentation method based on conditional invariant semantic segmentation was compared with that of the current state-of-the-art method MIC under four adverse condition subsets. The core indicator was the mean intersection-union ratio (mIoU). The adaptability of the model under different visual conditions was verified. The specific results are compared in Table 1 below:

[0122] Table 1 Comparison of results between the method of the present invention and the MIC method:

[0123]

[0124] As can be seen, the marine scene segmentation method based on conditional invariant semantic segmentation of this invention approaches or surpasses the MIC method in rainy, snowy, and nighttime conditions, and only slightly underperforms in foggy conditions, but performs excellently in key categories, and is in the first tier overall.

[0125] Based on three typical domain adaptation scenarios, the semantic segmentation performance of the maritime scene segmentation method based on conditional invariant semantic segmentation is compared with the baseline methods HRDA and MIC. The core metric is the mean intersection-union ratio (mIoU).

[0126] Zero-shot generalization: The model is trained only in the day / night domain and tested in the unseen night domain (where geographical / lighting differences are significant);

[0127] Adaptation to multiple adverse conditions: The source domain is under normal conditions, and the target domain is a dataset of comprehensive adverse conditions including night, fog, rain, and snow;

[0128] Day / night domain adaptation: The source domain is daytime and the target domain is nighttime, directly assessing the domain adaptation capability under drastic changes in day and night illumination.

[0129]

[0130] Experiments show that the "shallow stylization + feature invariance loss" architecture of the marine scene segmentation method based on conditional invariant semantic segmentation of this invention exhibits robust performance under different domain adaptation difficulties, especially in zero-shot generalization and extreme lighting change scenarios. It provides an efficient solution for fields such as marine ship navigation that need to cope with complex visual conditions, proving the key role of feature-level constraints in improving model robustness.

[0131] This invention innovatively integrates wavelet transform for semantic segmentation in maritime scenarios. The DWT layer decomposes the feature map into low-frequency structure (LL) and high-frequency details (LH / HL / HH), avoiding the loss of spectral information due to averaging in traditional pooling. For example, in foggy maritime scenarios, the LL sub-band preserves the overall outline of ships and coastlines, while the HL / HH sub-band captures high-frequency variations at fog edges, solving the problem of target blurring under low contrast. The IDWT layer achieves distortion-free reconstruction of the feature map (reconstruction error ≤0.01%) through conjugate filter deconvolution, ensuring that the spectral information extracted by the encoder is completely transmitted to the decoder, improving the semantic boundary localization accuracy under complex lighting conditions (e.g., reducing the segmentation error of ship light edges by 30% at night). It features a dynamic filter adaptation mechanism, supporting learnable wavelet filters (e.g., training dedicated filter banks for different sea areas), enabling the model to adapt to differences in spectral distribution under different water qualities (e.g., turbid / clear) and lighting conditions (e.g., strong reflection / shadow), enhancing cross-domain generalization ability (e.g., improving mIoU by 15% from nearshore ports to open ocean).

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for segmenting maritime scenes based on conditional invariant semantic segmentation, characterized in that, Includes the following steps: S1. Construct a marine dataset and training batch containing ships, waves, coastlines, and clouds. Obtain the source domain image set and the target domain image set. The number of source domain images and target domain images in the training batch is equal. S2. Generate stylized view: Using shallow stylization mapping, the source domain image and the target domain image are converted into each other in style, while retaining key semantic information, including: ship outline and wave texture, to obtain a stylized view; S3. Constructing the CISS network model: Extracting input image features and generating semantic segmentation results through an encoder-decoder network. A matching wavelet pooling layer is introduced into the encoder, and the feature map is decomposed into multiple resolutions through discrete wavelet transform, preserving low-frequency structures and high-frequency details related to the semantics of the maritime scene, and outputting high-level features containing spectral information. Spectral features containing low-frequency ship contours, high-frequency wave textures, and coastline edge details are extracted. For the characteristics of the maritime scene, a sub-band attention module is added after the DWT layer to dynamically adjust the sub-band weights. The decoder performs reconstruction without information loss and generates a category probability map based on the high-level features. S4. Define the loss function, including pixel-level semantic supervision loss and feature-level invariant loss. Introduce a dynamic weighting strategy to construct a joint loss function and automatically adjust the weight balance between pixel-level semantic supervision loss and feature-level invariant loss. S5, CISS network model training and optimization: The optimizer is used in conjunction with a linear learning rate warm-up strategy to optimize the model; during training iterations, pixel-level cross-entropy loss and feature-level invariance loss are optimized simultaneously, enabling the encoder to extract style-invariant high-level features, and the decoder to generate semantic segmentation results.

2. The maritime scene segmentation method based on conditional invariant semantic segmentation according to claim 1, characterized in that, In step S1, the marine dataset is collected in an environment with unsupervised domain adaptive semantic segmentation settings. Images of the marine environment under different weather, lighting, and seasonal conditions are acquired through a camera. Various objects in the images are labeled and data augmentation processing is performed to divide the dataset into training set, validation set, and test set. The source domain image, corresponding to normal visual conditions, is equipped with a pixel-level realistic semantic label Y. s ; The target domain image, corresponding to adverse visual conditions, is used to generate pseudo-labels through self-training using a DACs framework.

3. The maritime scene segmentation method based on conditional invariant semantic segmentation according to claim 2, characterized in that, In step S2, the shallow stylization mapping is used to preserve the content information of the input image and fuse the style features of the reference image; Convert the source domain image into a target domain style view. s→t The formula is: I s→t =g(I s ,I t ); Convert the target domain image into a source domain style view. t→s The formula is: I t→s =g(I t ,I s ); Where g is the shallow stylization mapping function, I s For the source domain image, I t For the target domain image; The stylized view pair is obtained and represented as: (I s ,I s→t ) and (I t ,I t→s ).

4. The maritime scene segmentation method based on conditional invariant semantic segmentation according to claim 3, characterized in that, The shallow stylization mapping includes, but is not limited to: Fourier domain adaptation, color transfer algorithm, and color dithering; The Fourier domain adaptation achieves style transfer by replacing the low-frequency Fourier amplitudes in the source domain image with the low-frequency Fourier amplitudes in the target domain image.

5. The maritime scene segmentation method based on conditional invariant semantic segmentation according to claim 3, characterized in that, In step S3, the encoder-decoder network adopts a high-resolution domain adaptation architecture, decomposing the segmentation network F into encoder φ and decoder ω: in, This represents the composition of functions; The encoder φ is a MiT-B5, composed of DWT layers. It applies four DWT filters to the input feature map via two-dimensional discrete wavelet transform for convolution. Each DWT filter f... mn After convolving with the input data D, a 2x downsampling is performed to obtain 4 subbands: D mn =Conv2D(D,f mn ,stride=2) Among them, D mn Represents the feature map of the m-th row and n-th column sub-band obtained after DWT decomposition, Conv2D represents the two-dimensional convolution operation, and stride represents the stride of the convolution operation; Form a feature representation φ(I) that includes spectral information: φ(I)∈R C×M×N Where C is the number of channels, M×N is the size of the input feature map, and φ(I) represents the feature representation obtained by encoder φ after extracting the input image I; The decoder ω is a context-aware feature fusion module, consisting of an IDWT layer, which upsamples the sub-bands output by the encoder to their original size and passes them through a conjugate filter. Deconvolution reconstructs the feature map and generates a class probability map; in, For IDWT filters, and DWT filters f mn Conjugate, scale represents the upsampling magnification factor, and UpSample represents the upsampling operation.

6. The maritime scene segmentation method based on conditional invariant semantic segmentation according to claim 5, characterized in that, The four sub-bands include: one low-frequency sub-band LL, used to preserve structural information, and three high-frequency sub-bands LH, HL, and HH, each sub-band being half the size of the input feature map; Subband LL is used to describe the low-frequency profile information of ships, subband HH is used to describe the high-frequency texture information of ocean waves, and subband LH / HL is used to describe the edge detail information of coastlines. For the HH subband representing the high-frequency texture of ocean waves and the LL subband representing the low-frequency contour of ships, a subband attention module is added after the DWT layer. The weights of the four subbands are dynamically adjusted through channel attention. The weights of the HL and HH subbands are increased in nighttime scenes, and the weights of the LL subbands are increased in foggy scenes, thereby enhancing the ability to express key spectral features.

7. The maritime scene segmentation method based on conditional invariant semantic segmentation according to claim 5, characterized in that, In step S4, the pixel-level semantic supervision loss applies a weighted cross-entropy loss to both the original source domain image and the target domain image to learn source domain semantic knowledge and adapt it to the target domain using pseudo-labels, as shown below: Where C is the number of categories, H×W is the image size, and α c The weight parameter for category c is used to emphasize the learning of key categories such as ships and coastlines; The feature-level invariance loss, which normalizes the stylized view pairs using the Frobenius norm-based loss to reduce the impact of brightness differences on feature amplitudes, is expressed as follows: Where ε is a constant to prevent division by zero, φ(I′) represents the feature representation extracted by encoder φ from the style-transformed image I′, ||·|| F It is the Frobenius norm.

8. The maritime scene segmentation method based on conditional invariant semantic segmentation according to claim 7, characterized in that, In step S4, a dynamic weighting strategy is introduced to obtain the joint loss function, which is expressed as follows: Where, λ s , λ t The weighting parameter for the feature invariance loss is set as a dynamic weighting function based on the average brightness or contrast of the image, to adapt to the invariance constraint strength under complex lighting and non-uniform weather interference at sea.

9. The maritime scene segmentation method based on conditional invariant semantic segmentation according to claim 3, characterized in that, In step S5, the optimizer is the AdamW optimizer, with preset encoder and decoder learning rates, and iterative training is performed. In the initial stage of training, a linear warm-up is performed to gradually increase the learning rate from 0 to the preset value; During training, each batch of inputs consists of the original source domain image, the target domain image, and a stylized view. After calculating the loss, backpropagation is performed, and the encoder and decoder parameters are updated alternately until convergence.

10. The maritime scene segmentation method based on conditional invariant semantic segmentation according to claim 9, characterized in that, In step S5, the training process includes the following sub-steps: S5.1 Data Loading: Each batch contains B=2 pairs of source-target domain images, dynamically generating stylized view I. s→t I t→s This forms four sets of image inputs (I s ,I t ,I s→t ,I t→s ); S5.2 Forward Propagation: Extracting features from four sets of images using the encoder. The corresponding segmentation prediction is generated through the decoder; S5.3 Loss Calculation: Calculate pixel-level semantic supervision loss, including: source domain ground truth label supervision. Target domain pseudo-label adaptation Calculate the feature-level invariant loss, including: source domain feature alignment. Target domain feature alignment The overall loss was incurred; S5.4 Backpropagation: The parameters are updated through the AdamW optimizer. The encoder parameters are updated with a low learning rate to stabilize feature extraction, and the decoder optimizes segmentation details with a high learning rate.

Images