Nearshore visual ship target detection method based on directly guided mask detection network

Abstract

The present invention relates to the technical field of nearshore visual target detection. Specifically, it is a nearshore visual ship target detection method based on a direct-guided mask detection network that can meet real-time monitoring requirements and effectively improve detection rate and accuracy. By predicting the foreground mask of the ship target end-to-end, the model focuses directly on the overall characteristics of the ship target, distinguishing highly similar local features, thereby reducing the interference of similar background features. Under the weak supervision condition of no semantic segmentation labels but only bounding box labels, the GMN combines the target mask prediction task with the target detection task. Through weak supervision, it performs more fine-grained semantic segmentation of ship targets based on bounding box detection. The method is robust to changes in target scale, orientation, background, and position, achieving more accurate nearshore visual ship detection, effectively improving the detection rate of the visual ship detection system, and meeting the system's real-time detection requirements.

Description

Technical field:

[0001] The present invention relates to the technical field of nearshore visual target detection, and in particular to a nearshore visual ship target detection method based on a direct guided mask detection network, which can meet real-time monitoring requirements and effectively improve detection rate and accuracy. Background technology:

[0002] With the development of the Maritime Silk Road, maritime transport volume is increasing, and its importance in international trade is growing. The development of the marine economy has led to increasingly complex maritime traffic and increasingly severe challenges to maritime security. Dynamic, real-time, and accurate ship detection plays a vital role in improving the efficiency of maritime traffic management and ensuring maritime safety. Visual image ship detection systems are favored for applications such as safe navigation and intelligent unmanned vessels due to their low video image acquisition costs, short acquisition and processing steps, high image resolution, rich detail and color features, and the ability to perform fine-grained, multi-category ship detection.

[0003] However, in nearshore visual ship detection tasks, ship targets are located in open environments, and the background is often more complex and changeable than for general target detection, making visual ship detection more challenging. Similar features in the background, such as buildings, can easily be confused with ship targets, leading to false detections. Interference features such as undulating water and waves, irregular clouds in the sky, and landmarks, buildings, and vegetation on land significantly impact ship target detection, resulting in poor ship target positioning accuracy. Under the interference of complex backgrounds, the positioning accuracy of ship targets is significantly affected.

[0004] In the field of target detection, convolutional neural network models based on deep learning methods have become a hot topic in target detection research since R-CNN achieved detection accuracy far exceeding that of traditional methods on the VOC dataset. In the field of nearshore visual ship target detection, ship target detection based on convolutional neural networks has become a mainstream research direction, offering higher detection efficiency and significantly improved detection accuracy compared to traditional methods. However, in open environments, nearshore visual ship target detection is prone to missed detections and false detections due to the influence of complex background interference features, resulting in a decrease in detection accuracy. Existing literature methods typically use indirect preprocessing to reduce background interference, fail to fully utilize the semantic features of ship targets, and have certain limitations. Furthermore, existing literature has made limited explorations into the application of anchor-free detection models to visual ship target detection. Summary of the invention:

[0005] In response to the shortcomings and deficiencies in the existing technology, the present invention proposes a nearshore visual ship target detection method based on a direct-guided mask detection network, which is particularly suitable for complex open real visual image scenes. The method directly utilizes the semantic features of ships through an end-to-end anchor-free detection model to suppress background feature interference, and can effectively improve the detection rate and accuracy of nearshore visual ship target detection.

[0006] The present invention is achieved by the following measures:

[0007] A nearshore visual ship target detection method based on a direct guided mask detection network is characterized by comprising the following steps:

[0008] Step 1: Establish a direct guided mask network model. Specifically, the anchor-free YOLOX type is used as the baseline. After the input image is extracted by the backbone network, in order to perform mask prediction to enhance the model's ability to distinguish similar features between the target and the background, the second-level downsampling rate of 16 is input into the GuidedMask module for target mask and background segmentation. In order to suppress the interference of background-irrelevant features and enhance the ability to locate the target, the obtained mask prediction and the other two-level features extracted by the backbone network are input into the Refining module for further feature optimization. The optimized features are semantically fused through the FPN (Feature Pyramid Network). The position and classification of the target are regressed through the anchor-free detection head to output the bounding box prediction result of the ship target.

[0009] Step 2: Train the model built in step 1, specifically:

[0010] Step 2-1: Input an image to the model. To expand the amount of training data and improve the size distribution imbalance in the training samples, the original image input in the training set is first subjected to real-time data augmentation. During training, random data augmentation preprocessing is performed on each epoch. The input image is subjected to a combination of random aspect ratio transformation and random size transformation to improve the problem of uneven target size distribution. Random translation is performed to diversify the target position distribution to improve the model's positioning ability. The robustness and generalization ability of the model are improved by combining random color gamut transformation and random horizontal flipping.

[0011] Step 2-2: Input the data-enhanced image into the GMN backbone network for basic feature extraction. The extracted features are then sequentially predicted using the GuidedMask module to enable the model to distinguish between the overall features of the ship target and the local features of similar backgrounds.

[0012] Step 2-3: Use the guided mask to weight the features through the Refining module to block the interference of irrelevant background features, and enhance the key features of the ship target through the attention mechanism;

[0013] Step 2-4: After multi-scale fusion of the optimized features in the feature pyramid, perform anchor-free bounding box detection, perform gradient backpropagation and parameter optimization on the mask prediction loss and detection loss, and judge the result based on whether the loss converges. If convergence, the training is completed; if not, repeat the training process;

[0014] Step 3: Use the model trained in step 2 to detect ship targets.

[0015] Step 3 of the present invention specifically includes the following steps: inputting data into the trained model, using the backbone network for feature extraction, then performing guided mask prediction processing and background interference suppression processing, performing key feature enhancement processing on the processed data, and then performing bounding box detection processing. Thereafter, dynamic threshold semantic mask segmentation processing is performed, and the bounding box and semantic segmentation results are output. After obtaining the bounding box detection results, the category information of the bounding box detection is combined with the mask information in the guided mask result. The semantic segmentation result of the ship target is calculated within each prediction box using a dynamic threshold method, which is more adaptable than the fixed threshold method. By combining the mask prediction task with the target detection task, the GMN directly utilizes the semantic information of the ship target under the weak supervision signal without the need for semantic mask annotation, achieving both the suppression of background interference features and the more fine-grained and accurate visual ship target detection based on the bounding box detection.

[0016] In the present invention, the ship target area mask foreground segmentation and the ship target bounding box detection are carried out simultaneously. The ship target area mask prediction is carried out in the low-level features with richer position information, while the ship target bounding box detection is carried out in the high-level features with more semantic information after the multi-scale feature fusion of different receptive fields is completed by FPN. During training, the loss function of the guided mask prediction is the BCE (Binary Cross Entropy) loss function of the weak mask label generated by the mask prediction result and the target box annotation. The predicted guided mask allows the model to focus on the overall characteristics of the target and improves the model's ability to recognize local similar features of the background. In the figure, the probability difference between the target area and the background area in the ship target foreground prediction is significant. The ship target mask can be extracted from the complex background by a simple binarization method. The ship target mask is combined with the category information and bounding box information in the subsequent ship target detection task prediction results to achieve semantic segmentation of the ship target and achieve more accurate recognition of the ship target.

[0017] The present invention performs dynamic threshold semantic segmentation on each detected ship target, and the calculation of the semantic segmentation mask is shown in Formula 1, P mask is the target mask probability, P conf is the confidence of the target detection result, λ is the coefficient set to 0.8, and for each location in the bounding box, if the product of the target detection confidence and the target mask prediction probability is greater than the λ-weighted mask probability mean, then the location is retained as the semantic segmentation mask.

[0018]

[0019] The training loss function is shown in Equation 2 and is composed of the loss of the mask prediction task and the center confidence, regression, and classification losses of the target detection task:

[0020] Loss = L mask +L det (2)

[0021] In order to effectively suppress the interference of irrelevant background features and enhance the key features of the target, the Refining module uses the predicted guide mask to constrain the features, and combines the SimAM attention mechanism to further optimize the features, so as to improve the positioning ability of the model and reduce the interference of irrelevant background features. The features with a large amount of background interference are adjusted by Conv3×3, and then multiplied with the foreground mask generated by GuidedMaskModule for weighting to shield the interference of irrelevant background features. The features are further enhanced through the SimAM attention mechanism to enhance the key features of the ship target, and the residual structure is used to ensure the gradient backpropagation. Finally, the optimized features are output after adjustment through Conv1×1.

[0022] This paper proposes a novel direct guided mask detection network (GMN) for ship target detection. By predicting the foreground mask of the ship target end-to-end, the model focuses directly on the overall characteristics of the ship target and distinguishes highly similar local features, thereby reducing the interference of similar background features. Under the weak supervision condition of no semantic segmentation labels but only bounding box labels, the GMN combines the target mask prediction task with the target detection task. Through weak supervision, it performs a more fine-grained semantic segmentation of ship targets based on bounding box detection. It is robust to changes in target scale, orientation, background, and position, achieving more accurate near-shore visual ship detection, effectively improving the detection rate of the visual ship detection system and meeting the system's real-time detection requirements. Description of the drawings:

[0023] Attachment Figure 1 This is a flow chart of the direct guide mask network GuideMaskNetwork detection in the present invention.

[0024] Attachment Figure 2 This is a structural diagram of the direct-guided mask network model of the present invention.

[0025] Attachment Figure 3 This is a structural diagram of the GuidedMask module in the present invention.

[0026] Attachment Figure 4 Schematic diagram of the foreground mask to semantic segmentation mask in the present invention, where Figure 4 (a) is the mask label; 4(b) is the mask prediction; 4(c) is the mask binarization; 4(d) is the semantic segmentation and target detection; 4(e) is the semantic segmentation mask.

[0027] Attachment Figure 5 In the present invention Figure 5 Schematic diagram of comparison of mask learning methods, where Figure 5 (a) Masked weak labels; 5(b) Reconstructed mask prediction; 5(c) Reconstructed mask visualization; 5(d) Guided mask prediction; 5(e) Guided mask visualization.

[0028] Attachment Figure 6 is a schematic diagram of the model detection results in the present invention, wherein Figure 6 (a) Relatively accurate mask; (b) truncated scene; (c) orientation change; (d) multi-target scene; (e) small target scene.

[0029] Attachment Figure 7 It is a schematic diagram of the Refining module structure in the present invention.

[0030] Attachment Figure 8 It is a schematic diagram of the feature optimization effect of the Refining module in the present invention.

[0031] Attachment Figure 9 This is a flowchart of network model training and testing in the present invention, where Figure 9 (a) is the model training flow chart, Figure 9 (b) is the flow chart of model testing after training. Specific implementation method:

[0032] In response to the existing detection methods that use prior information about the ship target's environment to reduce interference, but do not use the semantic information of the ship itself, resulting in the inability to effectively suppress water interference such as sea clutter and reefs. In order to directly use the semantic information of the ship target itself to efficiently solve problems such as near-shore background interference, a direct guided mask network (GMN) is proposed to predict the foreground of the ship target.

[0033] like Figure 1As shown, the GMN network adds the task of predicting a ship target region mask and uses the extracted features to identify the region within the image to which the ship target belongs. This allows the model to focus more on distinguishing the overall ship target features from similar background features during feature extraction, rather than solely focusing on local background features that are highly similar to the target. This reduces the interference of similar background features on detection, enhancing the model's feature extraction and target localization capabilities. Compared to remote sensing ship detection sea-land segmentation and visual ship detection coastline extraction, which indirectly utilize prior information about the ship target's environment to reduce background interference by narrowing the detection area, the proposed guided mask approach directly and effectively utilizes the semantic features of the ship target in an end-to-end manner to segment the foreground and background, achieving more accurate distinction between the ship target and similar background features. To suppress the influence of irrelevant background features, the proposed GMN network uses the predicted guided mask to weight the extracted features to block out interference from other irrelevant features in the target region. Furthermore, the SimAM attention mechanism is combined to further enhance the key ship features, thereby strengthening the model's target localization capability in open scenes. GMN combines the target mask prediction task with the target detection task to achieve relatively accurate semantic segmentation of ship targets without semantic segmentation labels, and performs more fine-grained visual ship target detection based on the target bounding box.

[0034] GMN uses anchor-free YOLOX as its baseline. The specific structure is as follows: Figure 4 As shown. After the input image is extracted by the backbone network, in order to perform mask prediction to enhance the model's ability to distinguish similar features between the target and the background, the second-level features with a downsampling rate of 16 are input into the GuidedMask module for segmentation of the target mask and the background. In order to suppress the interference of background-irrelevant features and enhance the ability to locate the target, the obtained mask prediction and the other two-level features extracted by the backbone network are input into the Refining module for further feature optimization. The optimized features are semantically fused through the FPN (Feature Pyramid Network), and the position and classification of the target are regressed through the anchor-free detection head to output the bounding box prediction result of the ship target. Combining the model mask prediction and bounding box output, GMN achieves the parallel detection of ship targets and semantic segmentation under the condition of only weak supervision signal of the bounding box label. Compared with the target bounding box, the semantic segmentation result reduces the background area pixels that do not belong to the target, achieving a more fine-grained positioning of the detected target.

[0035] The Guided Mask module proposed in GMN enables the model to focus on distinguishing the overall target features of the ship from similar features in the background, thereby reducing the interference of similar background features on detection and enhancing the model's feature extraction capabilities. The mask prediction process is insensitive to background changes and has good adaptability. The Refining module in the model enables the model to extract the key features of the ship target and effectively suppress the intensity of complex and irrelevant background features. The feature optimization process is robust to changes in the orientation and position of the ship target. Compared with methods that use indirect prior information from coastline segmentation to reduce interference or only optimize the network structure and post-processing, GMN uses the semantic information of the ship target more directly and effectively in an end-to-end manner to reduce the interference of complex backgrounds, improving important issues in ship target detection. On the SeaShips dataset, it achieved a high accuracy (mAP.5) of 98.06% and maintained a high detection rate of 55FPS.

[0036] In order to distinguish the ship target features from the highly similar local features in the complex background, the GuidedMask module is designed in GMN to predict the area where the ship target is located, so that the model can focus on the main features of the target. Figure 3 The GuidedMaskModule shown in the figure takes the backbone network features as input and then performs channel compression on them in the upper branch using Conv1×1 and Conv3×3, gradually reducing the dimensionality to two channels. This process extracts the overall semantic information of the ship target. The two-channel features are separated into foreground and background of the ship target using a softmax function in the channel dimension. The output prediction results correspond to the probability of the feature belonging to the background and the probability of belonging to the ship target at a given spatial position, respectively. Unlike the detailed annotation of conventional semantic segmentation, the supervisory signal for the prediction result is only a weak label composed of the area within the target box. After obtaining the target mask prediction, the features at this level are optimized through a structure similar to that of the Refining module to improve the performance of subsequent ship target detection.

[0037] In GMN, the foreground segmentation of the ship target area mask and the ship target bounding box detection are performed simultaneously. The ship target area mask prediction is performed in the low-level features with richer position information, while the ship target bounding box detection is performed in the high-level features with more semantic information after the multi-scale feature fusion of different receptive fields completed by FPN. During training, the loss function guiding the mask prediction is the BCE (Binary Cross Entropy) loss function of the weak mask label generated by the mask prediction result and the target box annotation. Figure 4(a) is the generated foreground channel label of the ship target. The white area is the area where the ship target is located, which means that the probability of the area being the ship target is 1. The prediction guide mask allows the model to focus on the overall characteristics of the target and improves the model's ability to recognize local similar features of the background. Figure 4 In (b), the probability difference between the target area and the background area in the ship target foreground prediction is significant, such as Figure 4 As shown in (c), the ship target mask can be extracted from the complex background by a simple binarization method. Figure 4 The ship target mask shown in (d) is combined with the category information and bounding box information in the subsequent ship target detection task prediction results to achieve semantic segmentation of the ship target and achieve more accurate recognition of the ship target. Figure 4 As shown in (e), the semantic segmentation result output by GMN is relatively more accurate than the original weak label in locating the ship target part from the bounding box. Dynamic threshold semantic segmentation is performed for each detected ship target, and the calculation of the semantic segmentation mask is shown in Formula 1, P mask is the target mask probability, P conf is the confidence of the target detection result, λ is the coefficient set to 0.8, and for each location in the bounding box, if the product of the target detection confidence and the target mask prediction probability is greater than the mask probability mean weighted by λ, then the location is retained as the semantic segmentation mask.

[0038]

[0039] The object detection task aids in learning the GMN ship mask foreground prediction task. The semantic information extracted by the model during object detection constrains the mask, so that the predicted mask no longer simply reconstructs the mask's weak labels but instead mines the semantics of the ship itself, enabling the model to learn a more accurate target mask from the weak labels. The training loss function, as shown in Equation 2, is composed of the mask prediction loss and the target detection center confidence, regression, and classification losses. Figure 5 (b) with Figure 5 (d) The mask prediction results of directly reconstructing weak labels and guided mask prediction results under the constraints of the target detection task. The mask prediction results under the target detection task constraints converge better and show a significant difference from the background area. However, direct reconstruction cannot extract more semantic information, resulting in poor prediction convergence and difficulty in distinguishing the background. The model-predicted mask under the target detection task constraints can obtain relatively accurate target mask results from weak supervision signals, demonstrating that the model can distinguish the overall characteristics of the target entity from similar background features, making the model detection process more interpretable.

[0040] Loss = Lmask +L det (2)

[0041] In the GMN model, target semantic segmentation and target detection are performed in parallel under weak label conditions, such as Figure 6 The model achieves both accurate object detection and relatively accurate semantic segmentation. It learns to predict more accurate semantic segmentation masks from weaker mask labels and performs well in truncated scenes, with varying object orientations, in multi-object scenes, and in scenes with small objects. It is also robust to environment, scale, orientation, and position.

[0042] In order to effectively suppress the interference of irrelevant background features and enhance the key features of the target, the Refining module uses the predicted guidance mask to constrain the features and further optimizes the features in combination with the SimAM attention mechanism to improve the model's positioning ability and reduce the interference of irrelevant background features. Figure 7 The structure of the Refining module is as follows: the input features containing a large amount of background interference are adjusted through Conv3×3, and then multiplied with the foreground mask generated by the GuidedMaskModule for weighting to shield the interference of irrelevant background features. The features are further enhanced with the key features of the ship target through the SimAM attention mechanism, and the residual structure is used to ensure the gradient backpropagation. Finally, the optimized features are output after adjustment through Conv1×1.

[0043] The Refining module effectively suppresses background irrelevant features and enhances the key features of the ship target, such as Figure 8 As shown in (c). Overall, the proposed feature optimization method is insensitive to changes in ship position and background, and the target features of ships in different positions and backgrounds are significantly improved. In addition, the method is robust to changes in orientation, such as Figure 8 The oblique ship target features in the last row are also well enhanced.

[0044] The experiments were conducted on an Ubuntu 20.04 platform, using an Intel Core i7-10700F @ 2.90GHz CPU and an Nvidia GeForce RTX 3070 GPU. The CUDA 11.2 acceleration library was used for parallel acceleration, and the model was implemented using the PyTorch deep learning framework in Python 3.7. To ensure fairness in the comparison experiments, all models used the same 416×416 input image size and data augmentation preprocessing. Parameters were initialized using COCO pretrained weights, and trained for 100 epochs on a SeaShips7000, with parameter optimization using the Adam optimizer. The GMN was trained for 100 epochs. For the first 50 epochs, the backbone network weights were frozen for transfer learning. The minibatch size was set to 7, and the learning rate was 0.0001. For the next 50 epochs, the minibatch size was set to 3, and the learning rate was set to 0.00001.

[0045] The training process is as follows Figure 9 As shown in (a), to expand the amount of training data and improve the size distribution imbalance in the training samples, the original images in the training set are first subjected to real-time data augmentation. During each training epoch, random data augmentation preprocessing is performed. The input images are subjected to a combination of random aspect ratio and scale transformations to improve the uneven size distribution of objects. Random translations are performed to diversify the object position distribution to enhance the model's localization capability. A combination of random color gamut transformation and random horizontal flipping improves the model's robustness and generalization. Next, the augmented images are fed into the GMN backbone network for basic feature extraction. The extracted features are then subjected to guided mask prediction by the GuidedMask module, allowing the model to focus on the overall characteristics of the ship target and distinguish between similar local background features. The guided mask is then used to weight the features in the Refining module to suppress interference from irrelevant background features. Key features of the ship target are then enhanced through an attention mechanism. Finally, the optimized features are fused at multiple scales within the feature pyramid for anchor-free bounding box detection. Gradient backpropagation and parameter optimization are performed on the mask prediction loss and detection loss. The result is judged based on whether the loss converges. If it converges, the training is completed. If not, the training process is repeated.

[0046] The test process is as follows Figure 9As shown in (b), after removing data augmentation, the bounding box detection step is similar to the training process. After obtaining the bounding box detection results, the category information of the bounding box detection is combined with the mask information in the guided mask result. The semantic segmentation result of the ship target is calculated within each predicted box using a dynamic threshold method, which is more adaptive than the fixed threshold method. By combining the mask prediction task with the target detection task, GMN directly utilizes the semantic information of the ship target under the weak supervision signal without semantic mask annotation. This not only suppresses background interference features, but also performs more fine-grained and accurate visual ship target detection based on bounding box detection.

[0047] The test results and analysis are as follows:

[0048] The GMN uses the GuidedMask module to focus on the entire ship target, enhancing the model's ability to distinguish between ship target features and similar local background features. In the Refining module, mask predictions are used to weight input features, suppressing irrelevant background features. The SimAM attention mechanism is used to enhance key ship target features, improving the model's feature extraction and localization capabilities. As shown in Table 1, this improves GMN's mAP.5 from 97.37% to 98.06% and mAP.75 from 83.39% to 86.28% on the SeaShips dataset compared to its baseline model, YOLOX. This improvement in object detection performance demonstrates that the proposed improved model effectively addresses the challenges of ship target localization due to interference from complex background features and large variations in target position and orientation in open environments. Compared to the baseline model's conventional bounding box detection, GMN further achieves finer-grained semantic segmentation of ship targets.

[0049] Table 1. Detection performance analysis of GuidedMaskNetwork on SeaShips

[0050]

[0051] To further explore the role of the Guided Mask module and the Refining module in the GMN and the impact of convolutional semantic information on localization, comparative ablation experiments were conducted. As shown in Table 4-2, introducing only the Guided Mask module to reduce the interference of similar local background features, allowing the model to focus on the global characteristics of the target, or using only the Refining module to enhance key features and suppress irrelevant background interference, both resulted in a slight improvement in mAP.5 and a significant improvement in mAP.75, demonstrating a significant improvement in localization capability. In contrast, the GMN, which combines the two modules, achieves a significant improvement in mAP.5 to 98.06%, thanks to its ability to directly and accurately weight the ship target area features with the relatively accurate guide mask predicted by weak supervision. This more effectively enhances the key ship features and suppresses interference from similar and irrelevant background features. This improvement is even greater in mAP.75. To further enhance the model's localization capability, the Guided Mask module and the Refining module are adjusted to the FPN, and mask prediction is performed on the C1 features fused with the FPN, resulting in GMN II. Mask prediction for features at lower layers incorporates more positional information while achieving higher feature resolution, further improving the model's localization capabilities. Consequently, GMNⅡ achieved improvements of 3.64% and 0.75% in mAP.75 compared to the baseline models YOLOX and GMN, respectively, further enhancing the model's localization capabilities. In terms of detection efficiency, the designed GuidedMask and Refining modules are concise and efficient, requiring minimal computational effort. This allows the improved model to maintain high efficiency, with only a slight reduction in detection rate compared to the baseline model.

[0052] Table 2. Comparative ablation analysis of GuidedMaskNetwork

[0053]

[0054]

[0055] To test the generalization capability of the GMN model, we validated its generalization on the SMD dataset. As shown in Table 3, on the SMD dataset, YOLOX and GMN employ an anchor-free detection approach, resulting in better matching of positive and negative samples for ship targets. Consequently, their detection accuracy (mAP.5) surpasses that of the anchor-free YOLOv3. Compared to the baseline model YOLOX, GMN demonstrates stronger feature extraction capabilities and suppresses background interference, improving detection accuracy for small ship classes such as boats, sailboats, and speedboats, which are susceptible to background interference. These experimental results demonstrate the GMN's strong generalization capabilities for visual ship detection tasks.

[0056] Table 3 Generalization performance analysis of GuidedMaskNetwork on SMD

[0057]

[0058] The detection models on the SeaShips dataset can be divided into anchor-based and anchor-free detection models based on the anchor setting. The anchor-based detection model is more dependent on the setting of the prior frame and the decoding post-processing is more complicated, while the anchor-free detection model does not rely on the setting of the prior frame and is more friendly to small target detection and has a simple post-processing process. In order to compare the performance differences between the two types of detection models in visual ship detection, the present invention uses the anchor-based type YOLOv3 and the anchor-free type YOLOX as baseline models for improvement. Since the anchor-free type does not rely on the design of the prior frame, it is more conducive to target matching, making the detection positioning accuracy higher than the anchor-based model. As shown in Table 4, the anchor-based model YOLOv3 is significantly lower than the center point method anchor-free type Centernet, YOLOX general target detection model and GMN improved model in terms of mAP.75 indicators. Among anchor-free models, Centernet, with a ResNet50 backbone network, has lower mAP.5 and mAP.75 scores than other similar models due to its weak feature extraction capabilities. However, the simplified backbone network design makes detection more efficient. Centernet, with an Hourglass backbone network, enhances the backbone network's feature extraction capabilities, resulting in a mAP.75 score similar to the proposed GMN, but its mAP.5 score is weaker than the GMN. The overly complex backbone network structure makes the model's detection efficiency unable to meet the 30 FPS real-time detection requirement.

[0059] Table 4 Comparative analysis of detection models on SeaShips

[0060]

[0061]

[0062] From the perspective of model improvement, the GMN improves on the underlying challenges of visual ship detection, such as complex background interference and the difficulty of localization in open scenes due to large variations in position, orientation, and size. Consequently, it achieves superior detection performance even with a relatively small amount of training data in anchor-free models. Unlike saliency-aware CNNs, which indirectly utilize prior information about the ship's environment to segment coastlines and reduce background interference, and eYOLO, which utilizes post-processing of position uncertainty to enhance robustness to interference, the proposed improved model effectively addresses the underlying issues by directly optimizing features, further improving detection accuracy. Compared to general object detection models such as shipYOLO, RDSCYOLOv4, YOLOv3, YOLOX, and CenterNet, which only optimize at the model structure level, the improved model addresses these challenges in visual ship detection, resulting in a more significant performance improvement.

[0063] Visual ship detection models for the SeaShips dataset can be categorized based on the detection stage, including two-stage models such as Faster R-CNN and one-stage models such as YOLO. As shown in Table 4-4, the two-stage Faster R-CNN model requires candidate region prediction before performing object regression and classification, resulting in low detection efficiency and failing to achieve the 30 FPS performance standard for real-time detection. However, models such as the one-stage YOLO model meet real-time detection requirements. Furthermore, with continuous optimization of models based on YOLO, they have gradually surpassed Faster R-CNN in detection accuracy. Saliency-aware CNN relies on preprocessing for coastline extraction and saliency map calculation, which introduces a high computational load and prevents end-to-end detection, thus limiting detection speed. RDSCYOLOv4 uses depthwise separable convolutions to reduce the model's parameter count and computational complexity, achieving a high detection rate of 68 FPS on the SeaShips dataset. In order to maintain high detection efficiency, the proposed GMN chose to improve on the YOLO framework and also took the efficiency of the module into consideration during model design. Therefore, the proposed improved model can maintain a high detection efficiency of 55FPS.

[0064] This paper introduces an anchor-free GMN that directly utilizes semantic features of nearshore visual ships based on a guided mask approach. Under weak supervision using bounding box annotations, the model achieves parallel bounding box detection and semantic segmentation of ship targets, achieving a good balance between detection accuracy and detection rate. The model uses the Guided Mask module to predict the foreground mask of ship targets within the constraints of the object detection task, enabling the model to distinguish between global ship target features and interference from local features that are highly similar to the background. In the Refining module, the model weights features using the target guided mask to reduce interference from irrelevant background features. The SimAM attention mechanism is used to enhance key ship target features, improving the model's feature extraction and localization capabilities in open environments. The model's detection performance is analyzed on the SeaShips dataset. Comparative ablation experiments demonstrate the effectiveness of each GMN module. Generalization tests on the SMD dataset verify the improved model's good generalization performance. A comprehensive comparative analysis of the proposed improved model on the SeaShips dataset is conducted based on model type, detection accuracy, improvement method, and detection rate. The results demonstrate that the GMN achieves high detection accuracy while maintaining a good detection rate using relatively little training data.

Claims

1. A nearshore visual ship target detection method based on a direct guided mask detection network, characterized in that: The following steps are involved: Step 1: Establish a direct guided mask network model. Specifically, the anchor-free YOLOX type is used as the baseline. After the input image is extracted by the backbone network, in order to perform mask prediction to enhance the model's ability to distinguish similar features between the target and the background, the second-level downsampling rate of 16 is input into the GuidedMask module for target mask and background segmentation. In order to suppress the interference of background-irrelevant features and enhance the ability to locate the target, the obtained mask prediction and the other two-level features extracted by the backbone network are input into the Refining module for further feature optimization. The optimized features are semantically fused through the FPN (Feature Pyramid Network). The position and classification of the target are regressed through the anchor-free detection head to output the bounding box prediction result of the ship target. Step 2: Train the model built in step 1, specifically: Step 2-1: Input an image to the model. To expand the amount of training data and improve the size distribution imbalance in the training samples, the original image input in the training set is first subjected to real-time data augmentation. During training, random data augmentation preprocessing is performed on each epoch. The input image is subjected to a combination of random aspect ratio transformation and random size transformation to improve the problem of uneven target size distribution. Random translation is performed to diversify the target position distribution to improve the model's positioning ability. The robustness and generalization ability of the model are improved by combining random color gamut transformation and random horizontal flipping. Step 2-2: Input the data-enhanced image into the GMN backbone network for basic feature extraction. The extracted features are sequentially predicted by the GuidedMask module to make the model focus on the overall features of the ship target and distinguish between the local features of similar backgrounds; Step 2-3: Use the guided mask to weight the features through the Refining module to block the interference of irrelevant background features, and enhance the key features of the ship target through the attention mechanism; Step 2-4: After multi-scale fusion of the optimized features in the feature pyramid, anchor-free bounding box detection is performed. Gradient backpropagation and parameter optimization are performed on the mask prediction loss and detection loss. The result is judged based on whether the loss converges. If convergence occurs, the training is completed. If not, the training process is repeated. Step 3: Use the model trained in step 2 to detect ship targets.

2. The method for detecting nearshore visual ship targets based on a direct guided mask detection network according to claim 1, characterized in that: Step 3 specifically includes the following steps: inputting the data into the trained model, using the backbone network to extract features, then performing guided mask prediction processing and background interference suppression processing, performing key feature enhancement processing on the processed data, and then performing bounding box detection processing, and then performing dynamic threshold semantic mask segmentation processing, outputting the bounding box and semantic segmentation results, wherein after obtaining the bounding box detection result, combining the category information of the bounding box detection with the mask information in the guided mask result, and calculating the semantic segmentation result of the ship target in each prediction box by the dynamic threshold method, which is more adaptable than the fixed threshold method; by combining the mask prediction task with the target detection task, GMN directly utilizes the semantic information of the ship target under the weak supervision signal without the need for semantic mask annotation, which not only suppresses the background interference features, but also performs more fine-grained and accurate visual ship target detection based on the bounding box detection.

3. The method for detecting nearshore visual ship targets based on a direct guided mask detection network according to claim 1, characterized in that: The ship target area mask foreground segmentation and ship target bounding box detection are performed simultaneously. The ship target area mask prediction is performed in the low-level features with richer position information, while the ship target bounding box detection is performed in the high-level features with more semantic information after the FPN completes the fusion of multi-scale features with different receptive fields. During training, the loss function of the guided mask prediction is the BCE (Binary Cross Entropy) loss function of the weak mask label generated by the mask prediction result and the target box annotation. The predicted guided mask allows the model to focus on the overall characteristics of the target and improves the model's ability to recognize local similar features of the background. In the foreground prediction of the ship target in the figure, the probability difference between the target area and the background area is significant. The ship target mask can be extracted from the complex background through a simple binarization method. The ship target mask is combined with the category information and bounding box information in the subsequent ship target detection task prediction results to realize the semantic segmentation of the ship target and achieve more accurate recognition of the ship target.

4. The method for detecting nearshore visual ship targets based on a direct guided mask detection network according to claim 1, characterized in that: Dynamic threshold semantic segmentation is performed for each detected ship target, and the calculation of its semantic segmentation mask is shown in Formula 1, P mask is the target mask probability, P conf is the confidence of the target detection result, λ is the coefficient set to 0.8, and for each location in the bounding box, if the product of the target detection confidence and the target mask prediction probability is greater than the λ-weighted mask probability mean, then the location is retained as the semantic segmentation mask. The training loss function is shown in Equation 2 and is composed of the loss of the mask prediction task and the center confidence, regression, and classification losses of the target detection task: Loss=L mask +L det (2)。 5. The method for detecting nearshore visual ship targets based on a direct guided mask detection network according to claim 1, characterized in that: In order to effectively suppress the interference of irrelevant background features and enhance the key features of the target, the Refining module uses the predicted guide mask to constrain the features, and combines the SimAM attention mechanism to further optimize the features, so as to improve the positioning ability of the model and reduce the interference of irrelevant background features. The features with a large amount of background interference are adjusted by Conv3×3, and then multiplied with the foreground mask generated by the GuidedMaskModule for weighting to shield the interference of irrelevant background features. The features are further enhanced through the SimAM attention mechanism to enhance the key features of the ship target, and the residual structure is used to ensure the gradient backpropagation. Finally, the optimized features are output after adjustment through Conv1×1.

Images