An adaptive ship instance segmentation method based on feature decoupling
By using an adaptive ship instance segmentation network, features are extracted and reconstructed using a semantic backbone branch and a weather-aware auxiliary branch. This solves the problems of accuracy and robustness in ship instance segmentation under complex marine environments and achieves efficient ship instance segmentation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- WUXI UNIV
- Filing Date
- 2026-04-29
- Publication Date
- 2026-06-19
AI Technical Summary
Existing mainstream instance segmentation algorithms struggle to accurately detect and segment ship instances in complex marine environments, especially under adverse conditions such as sea fog, heavy rain, and waves. This results in suboptimal segmentation accuracy and robustness, making them prone to false detections and missed detections.
An adaptive ship instance segmentation method based on feature decoupling is adopted. Through an adaptive ship instance segmentation network, multi-scale ship geometric features and global environmental semantic features are extracted using a semantic backbone branch and a weather perception auxiliary branch. The feature reconstruction mapping is then performed in conjunction with a weather feature adaptive modulation module to generate ship instance segmentation results.
It improves the accuracy and robustness of instance segmentation under complex sea conditions, reduces boundary blurring and missed detection, enhances end-to-end inference efficiency, and can adjust in real time according to meteorological conditions, reducing reliance on image preprocessing.
Smart Images
Figure CN122244452A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image segmentation technology, and in particular to an adaptive ship instance segmentation method based on feature decoupling. Background Technology
[0002] With the deepening of intelligent water transportation, unmanned and intelligent assisted driving vessels are experiencing rapid development. As a core technology in ship environmental perception, ship instance segmentation not only needs to detect the presence of ships in an image, but also needs to determine the pixel-level contours of each ship. This capability is of decisive significance for maritime collision avoidance decision-making, real-time channel monitoring, and autonomous navigation safety.
[0003] Current mainstream instance segmentation algorithms mainly fall into two categories: Mask R-CNN and the YOLO series. These mainstream instance segmentation algorithms perform excellently in clear, controlled, and typical visual scenes, becoming an important foundation for computer vision tasks. However, real marine weather environments are extremely complex and variable: sea fog can cause severe loss of high-frequency information and a sharp drop in contrast; heavy rain can superimpose random striped pseudo-textures on images; and sea surges and strong light can easily produce local overexposure and false edges. These harsh conditions pose a severe challenge to mainstream instance segmentation algorithms. Mask R-CNN is a two-stage segmentation algorithm that relies on a Region Proposal Network (RPN) to generate candidate bounding boxes. Each candidate bounding box is then classified, regressed, and segmented. However, under conditions of sea fog or strong light interference, image contrast decreases and edges become blurred, making it difficult for the RPN to accurately anchor the ship's position, easily leading to candidate bounding box shifts or omissions. Furthermore, when noise such as wave foam and rain streaks overlap with the ship's area, the segmentation mask often exhibits holes, burrs, or even breaks, failing to output a complete ship outline.
[0004] The YOLO series is a single-stage segmentation algorithm that directly predicts bounding boxes and mask coefficients based on grid partitioning, pursuing real-time performance while maintaining relatively compact feature representation. However, in scenarios like heavy rain or surges, random striped pseudo-textures and false edges in the image can strongly compete with the ship's features. Because YOLO lacks explicit modeling and decoupling capabilities for meteorological noise, its shallow feature extraction network is prone to misallocating attention to high-frequency noise such as rain streaks and waves, resulting in the suppression of ship features and leading to missed detections or misidentification of non-target areas as ships.
[0005] As can be seen from the above introduction, the complex sea conditions in the real ocean environment can cause significant interference to ship instance segmentation. As a result, the segmentation accuracy and robustness of mainstream instance segmentation algorithms such as Mask R-CNN and YOLO are not ideal, and false positives and false negatives are prone to occur. Summary of the Invention
[0006] To address the aforementioned problems and technical requirements, this application proposes an adaptive ship instance segmentation method based on feature decoupling. The technical solution of this application is as follows: An adaptive ship instance segmentation method based on feature decoupling, comprising: The image of the sea area to be segmented is acquired and input into the adaptive ship instance segmentation network. The adaptive ship instance segmentation network includes a semantic backbone branch, a weather perception auxiliary branch, a weather feature adaptive modulation module, and an instance segmentation head. The semantic backbone branch is used to extract multi-scale spatial geometric features of the sea area image to be segmented and output ship geometric feature maps of various scales to the weather feature adaptive modulation module. The weather perception auxiliary branch is used to extract global environmental semantic feature maps from the image of the sea area to be segmented and generate environmental perception vectors. Output to the weather feature adaptive modulation module; The weather feature adaptive modulation module is used to convert the environmental perception vector Dynamic mapping to any number of Ship geometric feature diagrams at each feature level Corresponding mapping parameters and using mapping parameters Ship geometric feature diagram The reconstructed mapping yields the mapped ship geometric feature map. , The parameter is an integer. The instance segmentation head is used to output the ship instance segmentation results of the sea area image to be segmented based on the ship geometric feature map after mapping multiple feature levels.
[0007] A further technical solution is that the adaptive ship instance segmentation method based on feature decoupling also includes: The weather perception auxiliary branch is also used to output the global environmental semantic feature map to the weather feature adaptive modulation module; The weather feature adaptive modulation module is also used to generate ship geometric feature maps based on the global environmental semantic feature map. Matching two-dimensional spatial weather mask and using mapping parameters and two-dimensional space weather mask Collaborative analysis of ship geometry features The reconstructed mapping yields the mapped ship geometric feature map. .
[0008] A further technical solution involves an adaptive modulation module for weather features that modulates environmental perception vectors. Dynamic mapping to ship geometry feature map Corresponding mapping parameters and using mapping parameters Ship geometric feature diagram The reconstructed mapping yields the mapped ship geometric feature map. include: Environmental perception vector Dynamic mapping to ship geometry feature map Corresponding high-frequency mapping parameters and low-frequency mapping parameters ; Based on the ship's geometric characteristics diagram Spatial variance along the channel dimension will affect the ship's geometric feature map. Divided into high-frequency channel groups and low frequency channel group ; Utilizing high-frequency mapping parameters For high frequency channel groups Reconstructing the mapping yields the mapped high-frequency channel group. Using low-frequency mapping parameters For low frequency channel group Reconstruction mapping is performed to obtain the mapped low-frequency channel group. ; Mapped high-frequency channel group and the mapped low-frequency channel group By stitching together the channel dimensions, a mapped ship geometric feature map is obtained. .
[0009] Its further technical solution is to utilize each set of mapping parameters and two-dimensional space weather mask Collaborative analysis of ship geometry features The reconstructed mapping yields the mapped ship geometric feature map. This includes reconstructing the mapping according to the following formula:
[0010] Among them, each set of mapping parameters , Representing the geometric characteristics of a ship The corresponding scaling factor, Representing the geometric characteristics of a ship The corresponding offset.
[0011] A further technical solution involves generating a ship geometric feature map based on a global environmental semantic feature map. Matching spatial weather mask include: The global environmental semantic feature map is sampled to match the ship geometric feature map using bilinear interpolation. Same spatial resolution; The sampled global environment semantic feature map is processed through a Sigmoid activation function to generate a value range in... Two-dimensional spatial weather mask .
[0012] A further technical solution involves a weather perception auxiliary branch comprising cascaded convolutional layers, global average pooling layers, and fully connected layers. The convolutional layers extract global environmental semantic feature maps from the sea area image to be segmented. These global environmental semantic feature maps are then passed through the global average pooling layer and the fully connected layer to output an environmental perception vector of a predetermined dimension. .
[0013] A further technical solution is that the adaptive ship instance segmentation method based on feature decoupling also includes: Construct a sample dataset, which includes multiple sample sea area images carrying real labels with pixel-level masking of ships; In the process of training the adaptive ship instance segmentation network using the sample dataset, each sample sea area image is input into the adaptive ship instance segmentation network, and the global environmental semantic feature map extracted by the weather perception auxiliary branch is input into an additional ship target classifier to obtain the ship detection result. The classification loss of the ship target classifier is calculated based on the ship detection result and the real label of the ship pixel-level mask. The gradient inversion layer reverses the gradient returned by the ship target classifier and then backpropagates it. The network parameters of the convolutional layer in the weather perception auxiliary branch are adjusted in the direction of maximizing the classification loss.
[0014] A further technical solution is that the adaptive ship instance segmentation method based on feature decoupling also includes: Construct a sample dataset, which includes multiple sample sea area images carrying real labels with pixel-level masking of ships; Each sample sea area image is input into an adaptive ship instance segmentation network to obtain the ship instance segmentation results of the sample sea area image. The ship instance segmentation results include the ship pixel-level mask prediction value, the predicted bounding box, and the confidence score of the ship category. The segmentation loss is calculated based on the ship instance segmentation results and their pixel-level mask true labels from the sample sea area images. ; Calculate the confidence score for each ship category relative to the mapped ship geometry. For each channel feature map, the feature gradient map is used. Global average pooling is then performed on the feature gradient map to obtain the channel weights of the corresponding channel feature map. The mapped ship geometric feature map is then mapped according to these channel weights. The feature maps of each channel are weighted and summed, and then activated by ReLU to obtain the current ship category in the [number]th channel. CAM heatmaps at each feature level The interpretive heatmap loss for the current ship category is obtained based on the CAM heatmaps of the current ship category at each feature level and the true labels of the ship's pixel-level mask. ; Comprehensive segmentation loss and interpretive heatmap losses for each ship category Obtain the total loss function and according to minimizing the total loss function The adaptive ship instance segmentation network is trained using a sample dataset.
[0015] A further technical solution involves obtaining an interpretive heatmap loss for the current ship category based on the CAM heatmaps of the current ship category at various feature levels and the actual pixel-level mask labels of the ships. Including calculations according to the following formula:
[0016] in, Is the current ship category in the number The loss weight coefficients for each feature level. Is the current ship category in the number CAM heatmaps at each feature level Total number of pixels included. The image of the sample sea area is downsampled to the pixel-level mask of the ship with respect to the current ship category at the 1st level. CAM heatmaps at each feature level Label images with consistent spatial resolution; This refers to the element-wise multiplication operation of a matrix. Indicates calculation The mean of the L2 norm.
[0017] The further technical solution is that the semantic backbone branch adopts the YOLOv11 network and outputs the ship geometric feature maps of P3, P4 and P5 layers to the weather feature adaptive modulation module, and the instance segmentation head adopts the YOLO11-seg instance segmentation head.
[0018] The beneficial technical effects of this application are: This application discloses an adaptive ship instance segmentation method based on feature decoupling. This method uses an adaptive ship instance segmentation network to segment ship instances from an image of the sea area to be segmented. In addition to extracting multi-scale ship geometric feature maps using a semantic backbone branch, the adaptive ship instance segmentation network also utilizes a weather perception auxiliary branch parallel to the semantic backbone branch to actively perceive the global environment in the image of the sea area to be segmented and generate an environment perception vector, thereby achieving dual-path feature decoupling. A weather feature adaptive modulation module dynamically generates mapping parameters based on the environment perception vector to reconstruct and map the ship geometric feature maps at the corresponding feature levels. This allows for dynamic adaptive adjustment of feature distribution according to the meteorological environment in the image of the sea area to be segmented. This enables the adaptive ship instance segmentation network to adjust in real time according to fluctuations in the meteorological environment of the input image of the sea area to be segmented, without relying on cumbersome image defogging or deraining preprocessing. Finally, the instance segmentation head outputs the ship instance segmentation result based on the mapped ship geometric feature maps at multiple feature levels, improving end-to-end inference efficiency, ship instance segmentation accuracy, and multi-weather robustness.
[0019] The weather feature adaptive modulation module also generates a two-dimensional spatial weather mask based on the global environmental semantic feature map, and works with the mapping parameters to reconstruct the spatial location of the ship geometric feature map in a non-uniform way to deal with the spatial heterogeneity of marine weather, such as the depth dependence of sea fog and the regionality of surge, thereby further improving the instance segmentation accuracy and robustness of the method in complex sea conditions.
[0020] The weather feature adaptive modulation module also divides each ship's geometric feature map into a high-frequency channel group and a low-frequency channel group for reconstruction mapping, in order to deal with high-frequency noise such as raindrops and low-frequency degradation caused by sea fog, thereby further improving the instance segmentation accuracy and robustness of the method in complex sea conditions.
[0021] This method also introduces a physically meaningful interpretability mechanism. Unlike existing techniques that directly fit masks, this method transforms the feature gradient map into an interpretable heatmap loss and adds it to the total loss function. This fundamentally quantifies and suppresses the interference of meteorological noise on neurons, forcing the network's visual attention to the ship's body outline, and significantly reducing boundary blurring and ship missed detection.
[0022] To prevent ship ontological semantics from being mixed in during the extraction of the global environment semantic feature map, the weather perception auxiliary branch also embeds an adversarial discriminant structure with a gradient inversion layer. By maximizing the ship classification loss during network training, the ship geometric features in the global environment semantic feature map can be forcibly stripped, making the decoupling of the two-path features more thorough and thus further improving the segmentation effect. Attached Figure Description
[0023] Figure 1This is a network architecture diagram of the adaptive ship instance segmentation network in Embodiment 1 of this application.
[0024] Figure 2 This is a network architecture diagram of the adaptive ship instance segmentation network in Embodiment 2 of this application.
[0025] Figure 3 This is a network architecture diagram of the adaptive ship instance segmentation network in Embodiment 3 of this application.
[0026] Figure 4 This is a schematic diagram of the network training of the adaptive ship instance segmentation network in Embodiment 4 of this application.
[0027] Figure 5 This is a comparison of the ship instance segmentation results of Model 1 and Model 3 trained in the same sea area image. Detailed Implementation
[0028] The specific embodiments of this application will be further described below with reference to the accompanying drawings.
[0029] Example 1 discloses an adaptive ship instance segmentation method based on feature decoupling. This method uses a pre-trained adaptive ship instance segmentation network for adaptive ship instance segmentation. Please refer to... Figure 1 The diagram shows the network architecture of the adaptive ship instance segmentation network, which includes a semantic backbone branch, a weather perception auxiliary branch, a weather feature adaptive modulation module, and an instance segmentation head. The semantic backbone branch and the weather perception auxiliary branch are two parallel branches.
[0030] After acquiring the image of the sea area to be segmented, it is input into the semantic backbone branch and the weather perception auxiliary branch of the adaptive ship instance segmentation network. In the adaptive ship instance segmentation network: (1) The semantic backbone branch is used to extract multi-scale spatial geometric features of the sea area image to be segmented and output ship geometric feature maps of different scales to the weather feature adaptive modulation module. The semantic backbone branch uses a deep residual convolutional network to output ship geometric feature maps of different sizes to fuse high-level and low-level features to form a feature pyramid, focusing on extracting multi-scale spatial geometric features such as ship shape and edges.
[0031] In the first embodiment, the semantic backbone branch employs a YOLOv11 network. This application will use any... The geometric feature diagram of a ship at each feature level is denoted as follows: , For integer parameters, in one embodiment, the multi-scale ship geometry feature map output by the semantic backbone branch includes the ship geometry feature map of the P3 layer of the YOLOv11 network. Ship geometry feature diagram at P4 layer of YOLOv11 network Ship geometry feature diagram at P5 layer of YOLOv11 network Ship geometry diagram It boasts the highest spatial resolution and largest feature map, preserving rich detail information, making it suitable for identifying small vessel targets. (Vessel geometric feature map) Responsible for identifying medium-sized ship targets. Ship geometry diagram. It has the lowest spatial resolution and the smallest feature map, but contains powerful semantic information and is responsible for locating large ship targets.
[0032] (2) The weather perception auxiliary branch is used to extract the global environmental semantic feature map in the image of the sea area to be segmented. And generate environment perception vectors Output to the weather feature adaptive modulation module.
[0033] This adaptive ship instance segmentation network abandons the traditional single-backbone network feature extraction mode and adopts a dual-feature decoupled branch. In addition to the semantic backbone branch, it also includes a weather perception auxiliary branch parallel to the semantic backbone branch. This weather perception auxiliary branch extracts meteorological information such as transmittance and illuminance, as well as various background information from the sea area image to be segmented through a lightweight encoder to obtain a global environmental semantic feature map. And based on the global environment semantic feature map Generate a fixed-dimensional environment perception vector Output to the weather feature adaptive modulation module.
[0034] In one embodiment, such as Figure 1 As shown, the weather perception auxiliary branch consists of cascaded convolutional layers, global average pooling layers, and fully connected layers. The convolutional layers are used to extract global environmental semantic feature maps from the sea area image to be segmented. The obtained global environment semantic feature map After passing through a global average pooling layer and a fully connected layer, the output is an environment-aware vector of a predetermined dimension. In one instance, the output environment-aware vector The dimension is 128.
[0035] (3) The weather feature adaptive modulation module is used to convert the environmental perception vector Dynamic mapping to any number of Ship geometric feature diagrams at each feature level Corresponding mapping parameters and using mapping parameters Ship geometric feature diagram The reconstructed mapping yields the mapped ship geometric feature map. .
[0036] In one embodiment, a linear reconstruction method is used to reconstruct the ship's geometric feature map. By reconstructing the mapping, the environment perception vector is then... Each set of mapping parameters obtained by dynamic mapping , Representing the geometric characteristics of a ship The corresponding scaling factor, Representing the geometric characteristics of a ship The corresponding offset. Then, the ship's geometric feature map is processed according to the following formula. The reconstructed mapping yields the mapped ship geometric feature map. :
[0037] The weather feature adaptive modulation module reconstructs and maps the ship geometric feature map at each feature level according to the above method, thereby obtaining the mapped ship geometric feature map at each feature level. .like Figure 1 Ship geometry feature map output from P3 layer of YOLOv11 network The reconstructed mapping yields the mapped ship geometric feature map. Ship geometry feature map output from layer P4 of the YOLOv11 network The reconstructed mapping yields the mapped ship geometric feature map. Ship geometric feature map output from P5 layer of YOLOv11 network The reconstructed mapping yields the mapped ship geometric feature map. .
[0038] (4) The instance segmentation head is used to output the ship instance segmentation result of the sea area image to be segmented based on the ship geometric feature map after mapping at multiple feature levels. The obtained ship instance segmentation result includes the ship pixel-level mask prediction value, predicted bounding box, and confidence score of ship category in the sea area image to be segmented. In one embodiment, the YOLO11-seg instance segmentation head is used. The YOLO11-seg instance segmentation head uses the prototype module to solve and output the ship instance segmentation result based on the ship geometric feature map after mapping at multiple feature levels. This part adopts the existing method and will not be described in detail here.
[0039] The adaptive ship instance segmentation network used in this application achieves dual-path feature decoupling through parallel semantic backbone branches and weather-aware auxiliary branches. The weather-aware auxiliary branch can actively perceive the global environment in the sea area image to be segmented and generate an environment-aware vector. Then based on the environment perception vector Dynamically generate mapping parameters By reconstructing and mapping the ship geometric feature maps at the corresponding feature levels, the feature distribution can be dynamically and adaptively adjusted according to the meteorological environment. This allows the adaptive ship instance segmentation network to adjust in real time according to the fluctuating meteorological environment in the input sea area image to be segmented, without relying on cumbersome image defogging or deraining preprocessing. This improves end-to-end inference efficiency, instance segmentation accuracy, and multi-weather robustness.
[0040] Example 2: Based on Example 1, to further improve the ship instance segmentation effect under complex weather conditions, in this example, the weather perception auxiliary branch is also used to integrate the global environmental semantic feature map. For output to the weather feature adaptive modulation module, please refer to [link / reference]. Figure 2 The network architecture diagram shows that the weather feature adaptive modulation module is also used to adjust the global environmental semantic feature map. Generation and ship geometry feature map Matching two-dimensional spatial weather mask Then use the mapping parameters and two-dimensional space weather mask Collaborative analysis of ship geometry features The reconstructed mapping yields the mapped ship geometric feature map. .
[0041] The obtained two-dimensional spatial weather mask This is used to characterize the spatial distribution features of environmental disturbances such as meteorological factors. In one embodiment, the weather feature adaptive modulation module modulates the global environmental semantic feature map. Sampling to ship geometry map Using the same spatial resolution ensures that the spatial distribution characteristics of environmental disturbances coincide with the spatial location of the ship's geometric features. Then, the sampled global environmental semantic feature map is... After the Sigmoid activation function, the generated value range is... Two-dimensional spatial weather mask Typically, the global environment semantic feature map Since the size is smaller than the ship's geometric feature map, bilinear interpolation is used for the global environment semantic feature map. Upsampling to ship geometry map Same spatial resolution.
[0042] In this embodiment, a two-dimensional spatial weather mask is added. Then, using the mapping parameters and two-dimensional space weather mask Collaborative analysis of ship geometry features The reconstructed mapping yields the mapped ship geometric feature map. This includes reconstructing the mapping according to the following formula:
[0043] This embodiment further utilizes a global environment semantic feature map. Extracting two-dimensional spatial weather masks To characterize the spatial heterogeneity of different environmental disturbances (e.g., surges are concentrated at the water surface, sea fog is denser at a distance and lighter near a distance), mapping parameters are used. and two-dimensional space weather mask Cooperative mapping can achieve non-uniform feature distribution correction and linear reconstruction, and automatically perform spatial targeted suppression to filter out noise, thereby completing adaptive purification processing of ship geometric feature maps, such as under surge interference, based on two-dimensional spatial weather masks. Lock the water surface area below the image.
[0044] Example 3: Based on Example 1, in order to further improve the ship instance segmentation effect under complex weather conditions, in this example, the environmental perception vector is... Dynamic mapping to any number of Ship geometric feature diagrams at each feature level Corresponding mapping parameters At that time, the environmental perception vector Dynamic mapping to ship geometry feature map Corresponding high-frequency mapping parameters and low-frequency mapping parameters Please refer to Figure 3 Network architecture diagram, Figure 3 For simplicity, only the geometric features of the ship are shown in detail. The reconstruction mapping process, ship geometric feature map Ship geometry diagram The reconstruction mapping process and ship geometric feature map Similarly, high-frequency mapping parameters... Including ship geometry diagrams Corresponding high-frequency scaling factor and high frequency offset Low-frequency mapping parameters Including ship geometry diagrams Corresponding low-frequency scaling factor and low-frequency offset .
[0045] On the other hand, according to the ship's geometric feature diagram Spatial variance along the channel dimension will affect the ship's geometric feature map. Divided into high-frequency channel groups and low frequency channel group This includes calculating ship geometry. Based on the spatial variance of each channel, channels with spatial variance reaching a variance threshold are grouped into high-frequency channel groups. Channels whose spatial variance does not reach the variance threshold are classified into low-frequency channel groups. .
[0046] Using mapping parameters Ship geometric feature diagram The reconstructed mapping yields the mapped ship geometric feature map. At the same time, including for high-frequency channel groups and low frequency channel group The respective mappings include: Utilizing high-frequency mapping parameters For high frequency channel groups Reconstructing the mapping yields the mapped high-frequency channel group. Using low-frequency mapping parameters For low frequency channel group Reconstruction mapping is performed to obtain the mapped low-frequency channel group. Finally, the mapped high-frequency channel group and the mapped low-frequency channel group By stitching together the channel dimensions, a mapped ship geometric feature map is obtained. .
[0047] Using high-frequency mapping parameters For high frequency channel groups Perform reconstruction mapping and utilize low-frequency mapping parameters. For low frequency channel group When performing the reconstruction mapping, the following formula is used to reconstruct the mapping using only the mapping parameters, following the method in Example 1:
[0048] In this embodiment, the weather feature adaptive modulation module maps the high-frequency channel group and the low-frequency channel group respectively, and can perform reconstruction of high-frequency features (such as texture details) and low-frequency features (such as background brightness) respectively. This can automatically deal with the noise degradation problem caused by frequency band conflict, including dealing with high-frequency noise such as raindrops and low-frequency degradation caused by sea fog, thereby improving the accuracy and precision of ship instance segmentation.
[0049] Or further, by utilizing high-frequency mapping parameters For high frequency channel groups Perform reconstruction mapping and utilize low-frequency mapping parameters. For low frequency channel group When performing the reconstruction mapping, the mapping parameters and the two-dimensional spatial weather mask are used according to the method in Example 2 using the following formula. Co-mapping:
[0050] By further combining the sub-channel mapping with a two-dimensional spatial weather mask, the adaptive filtering effect on ship geometric feature maps can be further improved, such as... Figure 3 The network architecture diagram is illustrated in this example. For instance, under surge interference, the water surface area below the image can be locked based on the two-dimensional spatial weather mask, and the high-frequency scaling factor of the high-frequency channel group sensitive to reflection can be automatically reduced. Value. However, in sea fog scenarios, the high-frequency offset of the high-frequency channel group can be automatically increased. To compensate for local contrast, adaptive purification of ship geometric feature maps is achieved.
[0051] Regardless of which network structure is used in the above embodiments, the adaptive ship instance segmentation network in this application needs to construct a sample dataset before performing ship instance segmentation on the sea area image to be segmented. This sample dataset includes multiple sample sea area images carrying real labels of ship pixel-level masks. Then, the adaptive ship instance segmentation network is pre-trained using this sample dataset.
[0052] Example 4: In this example, a physically meaningful interpretability mechanism is introduced by designing a novel loss function, including: During network training, each sample sea area image is input into the adaptive ship instance segmentation network to obtain ship instance segmentation results for the sample sea area image. These results include predicted ship pixel-level masks, predicted bounding boxes, and confidence scores for ship categories. Then, the segmentation loss is calculated based on the ship instance segmentation results and the ground truth ship pixel-level mask labels for the sample sea area image. This includes classification loss, bounding box loss, and mask loss, as well as segmentation loss. Calculated using existing classical methods.
[0053] In addition to calculating the segmentation loss according to the classic approach In addition, this embodiment also introduces interpretive heatmap loss. It includes the following steps: Calculate the confidence score for each ship category relative to the mapped ship geometry. The feature gradient map of each channel feature map is obtained, and then global average pooling is performed on the obtained feature gradient map to obtain the channel weights of the corresponding channel feature map. The mapped ship geometric feature map is then mapped according to each channel weight. The feature maps of each channel are weighted and summed, and then activated by ReLU to obtain the current ship category in the [number]th channel. CAM heatmaps at each feature level Based on the CAM heatmaps of the current ship category at each feature level and the true labels of the ship's pixel-level mask, an interpretive heatmap loss for the current ship category is obtained. Finally, the overall segmentation loss... and interpretive heatmap losses for each ship category Obtain the total loss function These can be directly added or weighted summed. Finally, the sum is determined by minimizing the total loss function. The adaptive ship instance segmentation network is trained using a sample dataset.
[0054] This embodiment transforms the feature gradient map into an interpretable heatmap loss with physical meaning. The higher the loss of the interpretative heatmap, the more the surface network is distracted by meteorological disturbances such as sea fog or waves. The resulting large gradients will be preferentially fed back to the network layers, forcing the network to relearn and generate mapping rules to complete self-correction. This method fundamentally quantifies and suppresses the interference of meteorological noise on neurons, forcing the network's visual attention to the ship's outline, significantly reducing boundary blurring and ship omissions.
[0055] Because the weather feature adaptive modulation module outputs a multi-scale mapped ship geometry feature map Therefore, in one embodiment, a multi-scale supervision and weighted fusion mechanism is employed to calculate the interpretive heatmap loss. Explanatory heatmap loss for current ship categories Including calculations according to the following formula:
[0056] in, Is the current ship category in the number CAM heatmaps at each feature level Total number of pixels included. The image of the sample sea area is downsampled to the pixel-level mask of the ship with respect to the current ship category at the 1st level. CAM heatmaps at each feature level Label images with consistent spatial resolution. This refers to the element-wise multiplication operation of a matrix. Indicates will and Element-wise multiplication yields Then calculate the mean of the L2 norm. Is the current ship category in the number The loss weight coefficients for each feature level can be customized. Because sea fog severely obscures small distant targets, in practical applications, the loss weight coefficients for each feature level... The shallower the hierarchical structure of each feature level, the smaller the scale of the target ship used for perception, and the larger the corresponding loss weight coefficient. For example, when the YOLOv11 network is used in the semantic backbone branch, the loss weight coefficient corresponding to the P3 layer is relatively high.
[0057] The interpretive heatmap loss calculation mechanism provided in this embodiment ensures that anomalous activation noise falling outside the true label of the ship's pixel-level mask, whether it is a large cargo ship nearby or a small fishing boat obscured by dense fog in the distance, can be accurately quantified and penalized within their respective feature receptive fields.
[0058] Example 5: In this application, the adaptive ship instance segmentation network uses a weather perception auxiliary branch to extract the global environmental semantic feature map to obtain the environmental perception vector. To achieve physical feature decoupling, adaptive modulation is performed by forcibly stripping ship geometric semantics from the global environmental semantic feature map. During the training of the adaptive ship instance segmentation network using a sample dataset, an adversarial discriminant structure with a gradient inversion layer is embedded in the weather perception auxiliary branch. This adversarial discriminant structure includes an additional ship target classifier, such as... Figure 4 As shown, the global environmental semantic feature map extracted by the weather perception auxiliary branch is sequentially connected to the gradient inversion layer and the ship target classifier.
[0059] During the training of the adaptive ship instance segmentation network using the sample dataset, after inputting each sample sea area image into the adaptive ship instance segmentation network, the global environmental semantic feature map extracted by the weather perception auxiliary branch... During forward propagation, the gradient reversal layer does not modify the global context semantic feature map. Function, this global environmental semantic feature map An additional ship target classifier is input to obtain ship detection results, indicating the presence of ships in the sample sea area image. Then, based on the ship detection results and the ship pixel-level mask ground truth labels, the classification loss of the ship target classifier is calculated, and gradient backpropagation is performed. During gradient backpropagation, a gradient inversion layer inverts the gradient returned by the ship target classifier before backpropagation, adjusting the network parameters of the convolutional layers in the weather perception auxiliary branch in the direction that maximizes the classification loss. After adversarial training, the global environmental semantic feature map extracted by this weather perception auxiliary branch forgets geometric features such as ship edges and shapes, ensuring that the output global environmental semantic feature map contains only pure background and weather noise information. This fundamentally avoids feature entanglement during subsequent modulation, achieving purified feature decoupling.
[0060] To demonstrate the robustness and accuracy of the proposed method in complex marine environments, this embodiment constructs a mixed weather dataset containing 15,000 high-resolution marine images acquired under various meteorological conditions, specifically covering clear skies without fog, dusk, heavy rain, and dense sea fog. This mixed weather dataset is randomly divided into a training set and a test set in an 8:2 ratio, and the following control groups are set up for comparative experiments on this mixed weather dataset: (1) Using the training set based on the classic YOLOv11n-seg model structure, the network is trained according to the segmentation loss to obtain Model 1.
[0061] (2) Using the training set based on this application, as shown in the example Figure 3 The adaptive ship instance segmentation network structure shown is used to train the network according to the segmentation loss, resulting in Model 2.
[0062] (3) Using the training set based on this application, as shown in the example Figure 3 The adaptive ship instance segmentation network structure shown is trained according to the network training method provided in this application. Interpretive heatmap loss and adversarial training of the weather perception auxiliary branch are introduced to obtain Model 3.
[0063] The three trained models were then tested on the test set, with mAP50 (mean Average Precision at IoU=0.5) used as the evaluation metric for ship instance segmentation accuracy. The mAP50 and single-image inference time results for the three experimental methods are compared as follows:
[0064] The comparison results above show that Model 2, by adding a weather perception auxiliary branch to decouple features and reconstructing and mapping ship geometric features before instance segmentation, can suppress local meteorological noise such as sea fog and surges. Without losing the ship's geometric edges, it significantly improves the purity of ship geometric features, thereby increasing the ship instance segmentation accuracy of Model 2 to 93.1%.
[0065] Furthermore, after introducing interpretive heatmap loss and adversarial training of the weather perception auxiliary branch during the training process of the adaptive ship instance segmentation network of this application, the ship instance segmentation accuracy of Model 3 was further improved to 93.6%.
[0066] The ship instance segmentation results of Model 1 and Model 3 trained on a portion of the sea area images in the test set are shown in the following figures. Figure 5 As shown, by Figure 5It can be seen that Model 1 is prone to missing some small targets or having blurred edges when faced with dense fog or reflections from ocean waves. In contrast, Model 3 can detect ships very well, reducing the number of missed ships and improving the accuracy and robustness of ship instance segmentation.
[0067] More notably, although the adaptive ship instance segmentation network of this application adds a weather perception auxiliary branch and a weather feature adaptive modulation module compared to the YOLOv11n-seg model, the model inference time of Model 3 has not increased; on the contrary, it has decreased to 0.5ms. This counterintuitive high efficiency is attributed to the feature decoupling and purification mechanism of the adaptive ship instance segmentation network of this application. By accurately removing a large number of redundant noise features in the early stage of feature extraction, the computational burden of the subsequent instance segmentation head is greatly reduced, enabling the entire network to achieve extremely high segmentation accuracy while possessing superior real-time inference capabilities, fully meeting the engineering deployment requirements under real and complex sea conditions.
[0068] The above descriptions are merely preferred embodiments of this application, and this application is not limited to the above embodiments. It is understood that other improvements and variations that can be directly derived or conceived by those skilled in the art without departing from the spirit and concept of this application should be considered to be included within the protection scope of this application.
Claims
1. An adaptive ship instance segmentation method based on feature decoupling, characterized in that, The adaptive ship instance segmentation method based on feature decoupling includes: The image of the sea area to be segmented is acquired and input into the adaptive ship instance segmentation network, which includes a semantic backbone branch, a weather perception auxiliary branch, a weather feature adaptive modulation module, and an instance segmentation head. The semantic backbone branch is used to extract multi-scale spatial geometric features of the sea area image to be segmented and output ship geometric feature maps of various scales to the weather feature adaptive modulation module. The weather perception auxiliary branch is used to extract the global environmental semantic feature map from the image of the sea area to be segmented and generate an environmental perception vector. Output to the weather feature adaptive modulation module; The weather feature adaptive modulation module is used to convert the environmental perception vector Dynamic mapping to any number of Ship geometric feature diagrams at each feature level Corresponding mapping parameters and using mapping parameters Ship geometric feature diagram The reconstructed mapping yields the mapped ship geometric feature map. , The parameter is an integer. The instance segmentation head is used to output the ship instance segmentation result of the sea area image to be segmented based on the ship geometric feature map after mapping multiple feature levels.
2. The adaptive ship instance segmentation method based on feature decoupling according to claim 1, characterized in that, The adaptive ship instance segmentation method based on feature decoupling also includes: The weather perception auxiliary branch is also used to output the global environmental semantic feature map to the weather feature adaptive modulation module; The weather feature adaptive modulation module is also used to generate ship geometric feature maps based on the global environmental semantic feature map. Matching two-dimensional spatial weather mask and using mapping parameters and two-dimensional space weather mask Collaborative analysis of ship geometry features The reconstructed mapping yields the mapped ship geometric feature map. .
3. The adaptive ship instance segmentation method based on feature decoupling according to claim 1, characterized in that, The weather feature adaptive modulation module will use the environmental perception vector Dynamic mapping to ship geometry feature map Corresponding mapping parameters and using mapping parameters Ship geometric feature diagram The reconstructed mapping yields the mapped ship geometric feature map. include: Environmental perception vector Dynamic mapping to ship geometry feature map Corresponding high-frequency mapping parameters and low-frequency mapping parameters ; Based on the ship's geometric characteristics diagram Spatial variance along the channel dimension will affect the ship's geometric feature map. Divided into high-frequency channel groups and low frequency channel group ; Utilizing high-frequency mapping parameters For high frequency channel groups Reconstructing the mapping yields the mapped high-frequency channel group. Using low-frequency mapping parameters For low frequency channel group Reconstruction mapping is performed to obtain the mapped low-frequency channel group. ; Mapped high-frequency channel group and the mapped low-frequency channel group By stitching together the channel dimensions, a mapped ship geometric feature map is obtained. .
4. The adaptive ship instance segmentation method based on feature decoupling according to claim 2, characterized in that, Using each set of mapping parameters and two-dimensional space weather mask Collaborative analysis of ship geometry features The reconstructed mapping yields the mapped ship geometric feature map. This includes reconstructing the mapping according to the following formula: Among them, each set of mapping parameters , Representing the geometric characteristics of a ship The corresponding scaling factor, Representing the geometric characteristics of a ship The corresponding offset.
5. The adaptive ship instance segmentation method based on feature decoupling according to claim 2, characterized in that, Generate ship geometric feature map based on global environmental semantic feature map. Matching spatial weather mask include: The global environmental semantic feature map is sampled to match the ship geometric feature map using bilinear interpolation. Same spatial resolution; The sampled global environment semantic feature map is processed through a Sigmoid activation function to generate a value range in... Two-dimensional spatial weather mask .
6. The adaptive ship instance segmentation method based on feature decoupling according to claim 1, characterized in that, The weather perception auxiliary branch includes cascaded convolutional layers, global average pooling layers, and fully connected layers. The convolutional layers are used to extract global environmental semantic feature maps from the sea area image to be segmented. The global environmental semantic feature maps are then passed through the global average pooling layer and the fully connected layer to output an environmental perception vector of a predetermined dimension. .
7. The adaptive ship instance segmentation method based on feature decoupling according to claim 6, characterized in that, The adaptive ship instance segmentation method based on feature decoupling also includes: Construct a sample dataset, which includes multiple sample sea area images carrying real labels of ship pixel-level masks; In the process of training the adaptive ship instance segmentation network using the sample dataset, each sample sea area image is input into the adaptive ship instance segmentation network, and the global environmental semantic feature map extracted by the weather perception auxiliary branch is input into an additional ship target classifier to obtain the ship detection result. The classification loss of the ship target classifier is calculated based on the ship detection result and the real label of the ship pixel-level mask. The gradient inversion layer reverses the gradient returned by the ship target classifier and then backpropagates it. The network parameters of the convolutional layer in the weather perception auxiliary branch are adjusted in the direction of maximizing the classification loss.
8. The adaptive ship instance segmentation method based on feature decoupling according to claim 1, characterized in that, The adaptive ship instance segmentation method based on feature decoupling also includes: Construct a sample dataset, which includes multiple sample sea area images carrying real labels of ship pixel-level masks; Each sample sea area image is input into an adaptive ship instance segmentation network to obtain the ship instance segmentation result of the sample sea area image. The ship instance segmentation result includes the ship pixel-level mask prediction value, the predicted bounding box, and the confidence score of the ship category. The segmentation loss is calculated based on the ship instance segmentation results and their pixel-level mask true labels from the sample sea area images. ; Calculate the confidence score for each ship category relative to the mapped ship geometry. For each channel feature map, the feature gradient map is used. Global average pooling is then performed on the feature gradient map to obtain the channel weights of the corresponding channel feature map. The mapped ship geometric feature map is then mapped according to these channel weights. The feature maps of each channel are weighted and summed, and then activated by ReLU to obtain the current ship category in the [number]th channel. CAM heatmaps at each feature level The interpretive heatmap loss for the current ship category is obtained based on the CAM heatmaps of the current ship category at each feature level and the true labels of the ship's pixel-level mask. ; Comprehensive segmentation loss and interpretive heatmap losses for each ship category Obtain the total loss function and according to minimizing the total loss function The adaptive ship instance segmentation network is trained using a sample dataset.
9. The adaptive ship instance segmentation method based on feature decoupling according to claim 8, characterized in that, The interpretive heatmap loss for the current ship category is obtained based on the CAM heatmaps of the current ship category at each feature level and the true labels of the ship's pixel-level mask. Including calculations according to the following formula: in, Is the current ship category in the number The loss weight coefficients for each feature level. Is the current ship category in the number CAM heatmaps at each feature level Total number of pixels included. The image of the sample sea area is downsampled to the pixel-level mask of the ship with respect to the current ship category. CAM heatmaps at each feature level Label images with consistent spatial resolution; This refers to the element-wise multiplication operation of a matrix. Indicates calculation The mean of the L2 norm.
10. The adaptive ship instance segmentation method based on feature decoupling according to claim 1, characterized in that, The semantic backbone branch uses the YOLOv11 network and outputs ship geometric feature maps of layers P3, P4 and P5 to the weather feature adaptive modulation module. The instance segmentation head uses the YOLO11-seg instance segmentation head.