Lightweight shielded ship detection method based on attention mechanism

Abstract

The invention belongs to the technical field of computer vision and target detection, and particularly relates to a lightweight sheltered ship detection method based on an attention mechanism, which comprises the following steps: acquiring a data set of a ship target under a to-be-processed sheltering condition; preprocessing the pictures in the data set; dividing the preprocessed data set into a training set and a test set according to a proportion; an eAodeMar model used for feature extraction, feature fusion and prediction of the position of a target and the category of the target is constructed, and the eAodeMar model comprises a position enhancement module and a lightweight multi-scale feature semantic association module; the eAodeMar model is trained on the basis of the training set, and the trained eAodeMar model is obtained; inputting the test set into the trained eAodeMar model, and achieving the detection of the marine ship target of the unmanned ship under the shielding condition. According to the eAodeMar model provided by the invention, correct positioning and rapid identification of ships in a scene can be completed, and the detection speed is greatly improved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and target detection technology, specifically relating to a lightweight occlusion ship detection method based on an attention mechanism. Background Technology

[0002] Ship targets are crucial entities and primary targets in modern naval warfare. In military terms, ship target detection is vital for precision strikes, counter-terrorism, surveillance, and tracking, serving as a crucial means of safeguarding national territorial integrity and protecting maritime rights. Real-time, accurate monitoring of maritime targets allows for the timely detection and response to potential threats, ensuring national maritime security. In civilian applications, with the increasing volume of maritime transport, the number of ships and the density of maritime traffic are constantly rising, leading to an increasingly complex maritime navigation environment. Therefore, ship target detection is essential for maritime traffic management. Occluded target detection has always been a hot topic and a challenge in computer vision research, especially since shipborne cameras are easily affected by occlusion when capturing images of maritime targets. When a ship target is occluded, the occluding object obscures the features between targets, reducing the recognition and reasoning capabilities of deep networks and lowering the model's detection performance.

[0003] The human visual system can infer object attributes from the outlines present in a scene even when local information is occluded or lost. However, effective detection of occluded objects remains challenging for deep learning-based computer vision systems. To address this issue, research on occlusion detection algorithms has made some progress. One approach involves proposing an improved mosaic data augmentation method, which crops and stitches together four different images to create a new image, enriching the training data and thus enhancing the model's generalization performance to occlusion. Similarly, Cutout and CutMix methods can alleviate the occlusion problem to some extent. However, while these methods improve the ability to identify occluded ships, the virtual occluded scene does not reflect all the features of the real scene, resulting in limited performance improvements. Another approach treats occlusion as a deformation problem, extracting invariant features of the ship, such as the co-occurrence matrix gradient orientation histogram features and constrained subspaces, to handle local occlusion. However, these feature vectors are not applicable to all ship types. Recently, to address the difficulty in feature extraction caused by feature confusion in occluded ship targets, an attention-based ship occlusion detection method—AodeMar—has been proposed. Specifically, a position enhancement module is constructed based on residual connectivity and coordinate attention mechanisms. By fusing information from the horizontal and vertical directions in the channel dimension, it models the communication between different positional information to obtain long-distance dependencies, improving the feature extraction capability for targets in occluded areas. Based on spatial pyramid pooling and a sliding window self-attention encoder, a multi-scale feature semantic association method is proposed. This method establishes communication between multi-scale target features within and across windows through window sliding, enhancing the model's feature discrimination capability at both global and local levels. This method significantly improves the detection accuracy of occluded ship targets; however, the stacking of modules increases the model's computational complexity and reduces its detection speed, resulting in shortcomings in real-time performance.

[0004] Currently, research on lightweight deep convolutional neural networks mainly focuses on two aspects: first, designing lightweight network modules, which involves improving traditional convolution operations to reduce the number of network parameters. MobileNet decomposes traditional convolution into depthwise convolution and pointwise convolution, reducing the number of channels and image resolution by introducing width and resolution multipliers, respectively. However, the extensive use of convolution increases computation. To address this, pointwise grouped convolution was proposed to reduce the number of network parameters, but feature extraction is limited to within each group, lacking inter-group interaction, resulting in insufficient feature acquisition. To enhance inter-group feature interaction, ShuffleNet introduced channel blending, enabling effective fusion and interaction of features from different groups, enhancing feature expressiveness and improving network detection performance. To improve detection speed, EfficientNets achieves a balance between accuracy and speed by expanding the model's width, depth, and resolution. Some researchers have improved the detection speed of YOLOv4-based sea surface target detection algorithms by introducing depthwise separable convolution (DSC) to reduce the number of weights. GhostNet, by using computationally less computationally intensive linear operations on top of ordinary convolutions to generate various feature maps, has achieved the goal of revealing the intrinsic features of targets at a lower cost and has been widely used. Based on the YOLOv5 architecture, some researchers have used ShuffleNetv2, which is based on the squeeze-and-excitation network (SE) attention mechanism, as the backbone network to extract features and introduced a DSC module to reconstruct the neck network, achieving high detection accuracy on the SMD dataset. However, researchers quickly discovered that the number of parameters and running speed are not simply positively correlated. In recent years, more attention has been paid to the computational cost / floating-point operations per second (FLOPs). For example, MobileNet-SSD, based on SSD, has achieved good detection results by simply combining the lightweight MobileNet backbone with the SSD detection head. MobileNetv2-SSDLite replaces the backbone with the more lightweight MobileNetv2, achieving remarkable detection speed and accuracy. While these operations reduce the number of model parameters and computational complexity and speed up detection, they also weaken the model's ability to extract and represent features, resulting in compromised detection accuracy. Secondly, model compression techniques are used to prune redundant network structures. To balance detection speed and accuracy, researchers have been exploring more effective model compression methods. Furthermore, techniques such as precision quantization, pruning, and knowledge distillation are being considered to further compress the model.In its pursuit of lightweight design, YOLO-LITE employed an extreme dimensionality reduction strategy, significantly reducing computational cost but also resulting in a severe drop in accuracy. Some researchers have proposed a pruning algorithm based on parameter weights, which mitigates the impact on model performance by using neurons with smaller weights. While these techniques have yielded significant improvements in detection speed, striking a balance between speed and accuracy remains a major challenge.

[0005] To address the aforementioned technical issues, this invention proposes a lightweight occlusion detection method based on an attention mechanism. Summary of the Invention

[0006] In the aforementioned ship target detection task based on shipborne vision, the detection accuracy is easily affected by occlusion. Existing occlusion detection algorithms suffer from technical problems such as large number of parameters, high complexity, and difficulty in deployment on embedded GPUs. Therefore, this invention provides a lightweight occlusion ship detection method based on an attention mechanism. This invention is based on the AodeMar occlusion ship detection method. By introducing lightweight Ghost convolutions, a lightweight backbone network and a lightweight feature fusion network are designed respectively, reducing the number of network parameters and computational cost to improve detection speed. Then, the lightweight model is deployed on a Jetson Xavier NX embedded GPU for structural optimization and inference acceleration, enabling the practical application of the lightweight model.

[0007] The technical means employed in this invention are as follows: A lightweight occlusion detection method based on an attention mechanism includes the following steps: Obtain the dataset of ship targets under the occlusion conditions to be processed; Preprocess the images in the dataset; The preprocessed dataset is divided into training and test sets according to a certain ratio; Construct an eAodeMar model for feature extraction, feature fusion, and prediction of the location and category of a target. The eAodeMar model includes a location enhancement module and a lightweight multi-scale feature semantic association module. The eAodeMar model is trained based on the training set to obtain the trained eAodeMar model; The test set is input into the trained eAodeMar model to achieve the detection of unmanned surface vessels at sea under occlusion conditions; The trained eAodeMar model is deployed to the tensor inference framework in Jetson Xavier NX, which performs vertical and horizontal merging of the same operations in the trained eAodeMar model to reduce the complexity of the network.

[0008] Furthermore, the eAodeMar model includes a backbone network for extracting features from the image, a neck network for fusing the extracted features, and a detection head network for predicting the location and category of the target; the backbone network, the neck network, and the detection head network are cascaded in sequence.

[0009] Furthermore, the network architecture of the backbone network includes a first CBS module, a second CBS module, a first C3 module, a third CBS module, a second C3 module, a fourth CBS module, a third C3 module, a fifth CBS module, a location enhancement module, and an SPPF module connected in sequence. The network architecture of the neck network includes a sixth CBS module, a first upsampling module, a first feature splicing module, a fourth C3 module, a seventh CBS module, a second upsampling module, a second feature splicing module, a fifth C3 module, an eighth CBS module, a third feature splicing module, a first lightweight multi-scale feature semantic association module, a ninth CBS module, a fourth feature splicing module, and a second lightweight multi-scale feature semantic association module, all connected in sequence. The network architecture of the detection head network includes a first convolutional layer, a second convolutional layer, and a third convolutional layer.

[0010] Furthermore, the second C3 module is connected to the second feature splicing module, the third C3 module is connected to the first feature splicing module, the SPPF module is connected to the sixth CBS module, the fourth C3 module is connected to the third feature splicing module, the fifth C3 module is connected to the fourth feature splicing module, the fifth C3 module is connected to the third convolutional layer, the first lightweight multi-scale feature semantic association module is connected to the second convolutional layer, and the second lightweight multi-scale feature semantic association module is connected to the first convolutional layer.

[0011] Furthermore, the location enhancement module includes a first Ghost convolution module, an RCAB module, a fifth feature concatenation module, a second Ghost convolution module, and a third Ghost convolution module. The first Ghost convolution module, the RCAB module, the fifth feature concatenation module, and the second Ghost convolution module are connected in sequence. The first Ghost convolution module is connected to the data input end, and the third Ghost convolution module is connected to the data input end on one hand and to the fifth feature concatenation module on the other hand.

[0012] Furthermore, the RCAB module consists of two consecutive 2D convolutional layers, batch normalization and SiLU activation functions, a CA module and residual connections.

[0013] Furthermore, the network architecture of the first lightweight multi-scale feature semantic association module and the second lightweight multi-scale feature semantic association module is the same. The first lightweight multi-scale feature semantic association module includes an SPP module, a fourth Ghost convolution module, an STR module, a sixth feature concatenation module, a fifth Ghost convolution module, and a sixth Ghost convolution module. The SPP module, the fourth Ghost convolution module, the STR module, the sixth feature concatenation module, and the fifth Ghost convolution module are connected in sequence. The SPP module is connected to the data input end, and the sixth Ghost convolution module is connected to the data input end on one hand and to the sixth feature concatenation module on the other hand.

[0014] Furthermore, the network architecture of the first, second, third, fourth, fifth, and sixth Ghost convolutional modules is the same, and the workflow of the first Ghost convolutional module is as follows: The input feature map is convolved to generate a feature map with half the number of channels. The feature map with half the number of channels is subjected to depthwise convolution to obtain a depthwise convolutional feature map. The feature map with half the number of channels and the feature map of the depthwise convolution are concatenated together to obtain the output of the first Ghost convolution module.

[0015] Furthermore, the tensor inference framework merges the same operations in the trained eAodeMar model both vertically and horizontally, including: The trained eAodeMar model on the workstation is converted into a model in the Open Neural Network Exchange format; Based on the model of the open neural network exchange format, a tensor inference framework is created; Based on the tensor inference framework, a computation graph engine is created using a Builder in a unified computing device architecture environment, and the graph engine is deserialized to obtain a deserialized file. Based on the deserialized file, the inference output is obtained; The location of the ship target is obtained by encoding the output reasoning result; The video image detection task is completed by using non-maximum suppression to remove redundant prediction boxes in the location of the ship target.

[0016] Compared with the prior art, the present invention has the following advantages: 1. This invention proposes a lightweight multi-scale feature semantic association module (G-SP-STR module), which not only effectively solves the problem of poor detection performance caused by the weak correlation between local features and overall features of ships in occluded scenes, thus improving detection accuracy, but also accelerates the detection speed by introducing lightweight Ghost convolution to reduce the computational load and number of parameters of the model.

[0017] 2. This invention proposes a lightweight location enhancement module (G-RCAC3 module), which utilizes a location attention mechanism to establish long-range dependencies between features and employs residual connections to fuse shallow and high-level features, enhancing the representational power of local features. This suppresses features ineffective for detection while enhancing features effective for detection. It effectively solves the problem of accurately locating target edges in occluded regions, improving the model's expressive power and detection accuracy. Furthermore, by introducing lightweight Ghost convolutions, the computational cost and parameter count of the model are reduced, accelerating the detection speed.

[0018] 3. The invention was deployed to the Jetson Xavier NX embedded platform, and its structure was optimized and inference accelerated. Through systematic performance testing and comparative experiments on the publicly available MVDD13 test set, Ling Shui harbor basin real-world video, and SMD challenging shipborne video, the effectiveness and practicality of the invention in the detection of obscured ship targets were verified.

[0019] Based on the above reasons, this invention can be widely applied in fields such as computer vision and object detection. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a block diagram of the overall structure of the eAodeMar model in this invention.

[0022] Figure 2 This is a structural diagram of the CA attention mechanism.

[0023] Figure 3 This is a structural diagram of the RCAB module of the present invention.

[0024] Figure 4 This is a comparison diagram of the Ghost convolution structure and the ordinary convolution structure in this invention.

[0025] Figure 5For experimental equipment and Jetson Xavier NX embedded GPU.

[0026] Figure 6 This invention relates to the TensorRT inter-layer merging strategy.

[0027] Figure 7 These are example images of SMD shipborne video and harbor basin video collected in an embodiment of the present invention.

[0028] Figure 8 This is a comparison chart of video detection results in an embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] like Figure 1 The Chinese meanings of each module are shown in Table 1: Table 1. Abbreviations and full names related to this invention.

[0032] This invention provides a lightweight occlusion detection method for ships based on an attention mechanism, the specific steps of which are as follows: S1. Obtain the dataset of ship targets under the occlusion conditions to be processed.

[0033] Specifically, input an image with a size of 640×640×3.

[0034] S2. Preprocess the images in the dataset.

[0035] S3. Divide the preprocessed dataset into training and test sets according to the proportions.

[0036] S4. Construct the eAodeMar model for feature extraction, feature fusion, and prediction of the target's location and category. The eAodeMar model includes a location enhancement module and a lightweight multi-scale feature semantic association module.

[0037] Specifically, such as Figure 1 As shown, the eAodeMar model includes a backbone network for feature extraction from the image, a neck network for fusing the extracted features, and a detection head network for predicting the location and category of the target; the backbone network, the neck network, and the detection head network are cascaded in sequence.

[0038] In the backbone feature extraction stage, the input image is sequentially processed through the CBS, C3, G-RCAC3, and SPPF modules. The G-RCAC3 block focuses on processing edge information in overlapping occluded regions by applying the CA method and introduces lightweight Ghost convolutions to reduce the computational cost and parameter count, thus accelerating information processing. In the neck network, top-down and bottom-up feature fusion modules are employed. The convolutional modules of the last two fusion layers are replaced with a newly designed lightweight multi-scale feature semantic association module (G-SP-STR). In G-SP-STR, SPP provides multi-scale feature semantic information, and the STR self-attention encoder learns the correlation between each feature. The introduction of lightweight Ghost convolutions further reduces the computational cost and parameter count, accelerating information processing. Three feature maps of different scales are output from the three detector heads: 20×20×54, 40×40×54, and 80×80×54.

[0039] The connection relationships within the eAodeMar model are as follows: The backbone network architecture includes a first CBS module, a second CBS module, a first C3 module, a third CBS module, a second C3 module, a fourth CBS module, a third C3 module, a fifth CBS module, a location enhancement module, and an SPPF module, which are connected in sequence.

[0040] The location enhancement module includes a first Ghost convolution module, an RCAB module, a fifth feature concatenation module, a second Ghost convolution module, and a third Ghost convolution module. The first Ghost convolution module, the RCAB module, the fifth feature concatenation module, and the second Ghost convolution module are connected in sequence. The first Ghost convolution module is connected to the data input end, and the third Ghost convolution module is connected to the data input end on one hand and to the fifth feature concatenation module on the other hand.

[0041] To quickly and accurately locate the edges of targets in occluded regions, suppress features ineffective for detection while enhancing those effective, and fuse shallow and high-level features, this invention designs a G-RCAC3 module (position enhancement module) based on lightweight Ghost convolution. Specifically, the G-RCAC3 module (see...) Figure 1 ) is a module of RCAB (see Figure 3 ) and Ghost convolution (see Figure 4 It is composed of residual connections and feature concatenation. Among them, Ghost convolution has fewer parameters and fewer floating-point operations compared to ordinary convolution, greatly reducing computational cost. The RCAB module is an improvement upon the CA module (see...). Figure 2 An enhancement of the image, consisting of two consecutive 2D convolutions, batch normalization and SiLU activation function, CA module and residual connections. The 2D convolutions are responsible for extracting local features of the image, while CA enhances effective features and suppresses ineffective features, establishing long-range dependencies between features to accurately locate the edges of targets in occluded regions. Furthermore, to enhance the representational power of local features, residual connections are used to fuse shallow and high-level features.

[0042] The RCAB module consists of two consecutive 2D convolutional layers, batch normalization and SiLU activation function, CA module and residual connection.

[0043] The network architecture of the neck network includes a sixth CBS module, a first upsampling module, a first feature splicing module, a fourth C3 module, a seventh CBS module, a second upsampling module, a second feature splicing module, a fifth C3 module, an eighth CBS module, a third feature splicing module, a first lightweight multi-scale feature semantic association module, a ninth CBS module, a fourth feature splicing module, and a second lightweight multi-scale feature semantic association module, all connected in sequence.

[0044] The network architecture of the first lightweight multi-scale feature semantic association module and the second lightweight multi-scale feature semantic association module is the same. The first lightweight multi-scale feature semantic association module includes an SPP module, a fourth Ghost convolution module, an STR module, a sixth feature concatenation module, a fifth Ghost convolution module, and a sixth Ghost convolution module. The SPP module, the fourth Ghost convolution module, the STR module, the sixth feature concatenation module, and the fifth Ghost convolution module are connected in sequence. The SPP module is connected to the data input end. The sixth Ghost convolution module is connected to the data input end on one hand and to the sixth feature concatenation module on the other hand.

[0045] The network architecture of the first, second, third, fourth, fifth, and sixth Ghost convolutional modules is the same.

[0046] To obtain multi-scale feature interactions and enhance local semantic correlation, an SPP (Signal-Based Polymerization) module is inserted as an additional module into the STR (Self-Attention) block to form an SP-STR block. This SP-STR block replaces the last two convolutional operations in the neck network to learn the semantic correlation between each feature block and assign different attention to each feature block. Because the feature maps at the network's end have low resolution, applying the Transformer to these low-resolution feature maps can reduce computational and memory costs. It is noted that the hierarchical architecture in SP-STR offers flexibility in modeling at various scales, but its computational complexity is high. Therefore, this invention designs a G-SP-STR module (Lightweight Multi-Scale Feature Semantic Association Module) based on lightweight Ghost convolutions (see...). Figure 1 It uses lightweight Ghost convolution to replace ordinary convolution, thereby reducing computational complexity by reducing the number of parameters and floating-point operations.

[0047] The specific implementation process is as follows: like Figure 4 The diagram shows the Ghost convolutional and regular convolutional structures, where... Figure 4 (a) is a regular convolution. Figure 4 (b) is a Ghost convolution. Given the input is... ,in, h , w and c These represent the height and width dimensions of the input image, as well as the number of channels.

[0048] For ordinary convolution, after... n The size of each convolutional kernel is convolution generaten If we take a feature map and ignore the bias value, the output can be represented as:

[0049] in, , For convolution operations, The input feature map is used. The number of parameters involved in this convolution operation is:

[0050] in, c and n These represent the number of channels in the input and output feature maps, respectively. The number of floating-point operations can be expressed as:

[0051] in, and These represent the height and width of the output feature map, and the number of filters, respectively. n and number of channels c The values ​​are often quite large; for example, the G-RCAC3 module. , .

[0052] like Figure 4 As shown in (b), the Ghost convolution first uses a kernel size of [missing value]. Convolution operation Generate half the number of channels generated by a regular convolution, i.e. ( n An even number of feature maps (feature maps with half the number of channels) can be represented as:

[0053] in, For the input feature map, For convolution operations, A feature map with half the number of channels. , This is the output of the i-th channel after a normal convolution. .

[0054] Then, depthwise convolutions are performed on these feature maps. Specifically, each channel of the feature map outputs... As a group, a convolutional kernel of size is used. convolution Extracting the globally important information, the corresponding first... i Each channel output is:

[0055] in, This is the output of the i-th channel after depthwise convolution. The feature map after depthwise convolution is .

[0056] Finally, the feature map with half the number of channels is concatenated with the feature map from the depthwise convolution, i.e., feature fusion. This operation reduces both information loss and computational complexity. The output of the Ghost convolution module is obtained:

[0057] in, “ "This is the splicing operation in the channel direction."

[0058] The number of parameters and the number of floating-point operations for Ghost convolution are as follows:

[0059]

[0060] in, and Each value is 1. and The values ​​are all set to 3. Taking the first CBS convolution operation (c=512) in the G-RCAC3 module as an example, the ratio of Ghost convolution to ordinary convolution in terms of parameter count and floating-point operations is shown. and They are respectively:

[0061]

[0062] After introducing Ghost convolution, the changes in the number of parameters of G-RCAC3 and G-SP-STR modules are shown in Table 2. Obviously, the number of parameters of both modules decreased significantly, with G-RCAC3 reducing the number of parameters by 21.65% and G-SP-STR reducing the number of parameters by 13.04%.

[0063] Table 2 shows the changes in computational cost and parameter count of the G-RCAC3 and G-SP-STR modules after introducing Ghost convolution.

[0064] Note: "↓" indicates a downward movement. The network architecture of the detection head network includes a first convolutional layer, a second convolutional layer, and a third convolutional layer.

[0065] The second C3 module is connected to the second feature splicing module, the third C3 module is connected to the first feature splicing module, the SPPF module is connected to the sixth CBS module, the fourth C3 module is connected to the third feature splicing module, the fifth C3 module is connected to the fourth feature splicing module, the fifth C3 module is connected to the third convolutional layer, the first lightweight multi-scale feature semantic association module is connected to the second convolutional layer, and the second lightweight multi-scale feature semantic association module is connected to the first convolutional layer.

[0066] S5. Train the eAodeMar model based on the training set to obtain the trained eAodeMar model.

[0067] S6. Input the test set into the trained eAodeMar model to achieve the detection of unmanned surface vessels (USVs) at sea under occlusion conditions.

[0068] S7. Deploy the trained eAodeMar model to the TensorRT inference framework in Jetson Xavier NX. TensorRT will merge the same operations in the trained eAodeMar model vertically and horizontally to reduce the complexity of the network.

[0069] Specifically, this invention is deployed on a Jetson Xavier NX embedded GPU (see appearance). Figure 5 (See Table 3 for configuration details). The model is optimized for acceleration to enable practical applications. First, the Jetson Xavier NX system is upgraded to Jet Pack 4.6 on Ubuntu 18.04. Then, the compatible PyTorch version and other necessary environment dependencies are installed. Finally, the trained model is imported into Jetson Xavier NX for performance testing. This invention uses the TensorRT inference framework in Jetson Xavier NX for deployment.

[0070] Table 3 Jetson Xaiver NX Hardware Parameters

[0071] The specific operation process is as follows: Low-parameter precision calculation steps: During the training phase, the convolutional network uses backpropagation to update the network weights. Due to the small learning rate and update gradient, high parameter precision (typically FP32) is required for representation and storage. However, during network inference after training, backpropagation is no longer performed, and the network output mainly includes confidence scores and coordinate offsets. The parameter precision requirements are relatively low, so the computational cost can be reduced by lowering parameter precision, such as converting to FP16 or 8-bit integers, thereby accelerating the model. On the other hand, reduced parameter precision will affect the model's detection accuracy, especially since 8-bit integers significantly reduce the network's detection accuracy. Considering both detection accuracy and speed, this invention uses FP16.

[0072] Network Reconstruction: Convolutional networks mainly consist of basic structures such as convolutional layers, bias / normalization layers, and activation layers. These can be combined in different ways to form complex network structures, such as residual structures, SPP structures, and CSP structures. When designing and building networks, TensorRT merges some network structures and similar operations both vertically and horizontally, thereby reducing network complexity.

[0073] like Figure 6 As shown, since CUDA is invoked at each input layer, merging frequently used layers can improve optimization performance. Secondly, the convolution, bias, and activation layers are vertically merged into a single CBS (Convolution-Bias-SiLU) module, reducing the computation that originally required three CUDA core calls to a single call, thus decreasing the number of CUDA startups and shortening computation time. Then, CBS layers with the same input tensors and performing the same operations are horizontally merged. Finally, a simplified network structure is obtained.

[0074] TensorRT (Tensor Runtime) is a high-performance deep learning inference optimization library developed by NVIDIA, focusing on achieving low-latency, high-throughput model deployment on NVIDIA GPUs. It significantly improves the runtime efficiency of deep learning models through model optimization, accuracy calibration, and hardware adaptation techniques, and supports models exported from frameworks such as TensorFlow, PyTorch, and ONNX. TensorRT inference acceleration framework implementation: The TensorRT inference acceleration framework implementation includes model transformation, inference engine construction, and inference implementation.

[0075] First, the trained eAodeMar model on the workstation is converted to the Open Neural Network Exchange (ONNX) format, i.e., .pt is converted to .onnx. Second, it is deployed to the Jetson Xavier NX platform, and a TensorRT inference framework is created based on the ONNX model. Then, based on the TensorRT inference framework, a computational graph engine (.engine) is created using Builder in the CUDA (Compute Unified Device Architecture) environment, and the graph engine is deserialized to obtain a deserialized file. This serialized file stores the relevant network definitions, training parameters, and intermediate activation values. Based on the deserialized file, the inference output is obtained. By encoding the output inference result, the location of the ship target is obtained. Non-maximum suppression is used to remove redundant prediction boxes in the ship target location, completing the video image detection task.

[0076] Experimental content and evaluation indicators 1. To verify the effectiveness and superiority of the proposed eAodeMar model, this invention compares it with several state-of-the-art lightweight object detection methods, such as YOLOv3-Tiny, YOLOv4-Tiny, YOLOv5-Lite, and YOLOv7-Tiny, on the MVDD13 dataset.

[0077] 2. The training and testing experiments used the MVDD13 dataset, which includes various occlusion scenarios. The training set consisted of 25,541 images, the validation set of 2,838 images, and the test set of 7,095 images. The model evaluation data used the most challenging SMD shipborne video and video data actually collected in the harbor basin by the "Zhihai 2" USV.

[0078] 3. This invention uses the average accuracy mAP (Mean Average Precision), number of parameters, computational cost FLOPs (Floating-Point Operations Per Second), and detection frame rate FPS (Frames Per Second) for different IoU thresholds (0.5, [0.5:0.05:0.95]) as quantitative evaluation indicators.

[0079] Analysis of Experimental Results 1. Lightweight modular ablation experiment The results are shown in Table 4. First, compared with the original AodeMar model, the models using the lightweight modules G-RCAC3 and G-SP-STR showed a decrease of 3.14% and 0.62% in mAP@.5, respectively, and also reduced the number of parameters and FLOPs to varying degrees, while increasing the detection speed (FPS) by 32.40% and 35.24%, respectively. Furthermore, after lightweighting the two modules, the eAodeMar lightweight model achieved the best performance in all indicators except for a 0.42% decrease in detection accuracy (mAP@.5), especially with a nearly 26 FPS (42.12%) increase in detection speed, validating the effectiveness of the lightweight model.

[0080] Table 4 compares the model performance after using the lightweight modules G-RCAC3 and G-SP-STR.

[0081] Note: "+" indicates that the module should be replaced in the corresponding position. 2. Lightweight backbone network ablation experiment To investigate the impact of lightweight backbone networks on model performance, classic lightweight backbone networks such as GhostNet, ShuffleNetv2, MobileNetv3, PP-LCNet, and EfficientNetv2 were selected, and ablation experiments were systematically conducted. For convenience, AodeMar, which replaced the backbone network, is abbreviated as "A".

[0082] The comparison results are shown in Table 5. Compared with the lightweight backbone network model, eAodeMar achieved the second-best result in detection speed, second only to A-PP-LCNet, but it lacked advantages in parameter count and FLOPs. Specifically, it was second only to A-GhostNetv2 in parameter count and had about 10% higher FLOPs, but it had a significant advantage in detection accuracy. Among the lightweight backbone network models, A-PP-LCNet had relatively fewer parameters and FLOPs, achieving a faster detection speed. A-EfficientNet achieved relatively high mAP@.5 and mAP@[.5:.95], followed closely by A-GhostNetv2. In terms of detection speed, A-PP-LCNet was the fastest, followed by A-EfficientNet. Compared to AodeMar, these lightweight backbone networks significantly reduce the number of parameters and computational cost while noticeably improving FPS (except for MobileNetv3), with A-PP-LCNet showing an increase of 44.00% and A-EfficientNet 27.05%. However, it can be observed that replacing any of the lightweight backbone networks leads to a substantial decrease in detection accuracy. The A-EfficientNet module causes the least decrease in accuracy, with mAP@.5 and mAP@[.5:.95] decreasing by 2.9655% and 0.9689%, respectively. Therefore, for the AodeMar model, there is no need to replace its backbone network with a lightweight one.

[0083] Table 5 Quantitative Comparison of Classic Lightweight Backbone Models

[0084] Note: "A" represents the AodeMar model of the replaced backbone network; the bold numbers in each column indicate the optimal value of the corresponding metric. 3. Comparison Experiment with Classic Models To verify the superiority of the proposed eAodeMar model, this invention compares it with mainstream lightweight object detection models such as YOLOv3-Tiny, YOLOv4-Tiny, YOLOv5-Lite, and YOLOv7-Tiny.

[0085] The comparison results are shown in Table 6. It can be observed that the mainstream lightweight models have relatively small parameter counts and FLOPs, especially YOLOv5-Lite, whose parameter count is approximately 20% of AodeMar / eAodeMar, and its computational FLOPs are only 5% of AodeMar / eAodeMar. However, its detection speed is only 12.28 FPS (14%) faster than eAodeMar, and it reduces mAP@[.5:.95] and mAP@.5 by 8% and 14.3%, respectively. Furthermore, it can be observed that there is no absolute direct proportionality between parameter count and FLOPs and their FPS. Although eAodeMar has relatively more parameters and FLOPs, especially larger FLOPs, its detection speed surpasses that of the classic lightweight network YOLOv7-Tiny. This verifies that eAodeMar has a significant advantage in detection accuracy, while there is considerable room for improvement in detection speed.

[0086] Table 6 Quantitative Comparison of Classic Lightweight Testing Models

[0087] 4. Performance comparison of models after TensorRT acceleration For ease of reference, the TensorRT-accelerated model will be named tAodeMar. The AodeMar and eAodeMar models used in this section were deployed directly on the Jetson Xavier NX embedded platform PyTorch framework.

[0088] Table 7 shows the detection accuracy, speed, and time consumption at each stage of the AodeMar, eAodeMar, and tAodeMar models on the MVDD13 image test set. Compared to the original AodeMar model, eAodeMar, based on the embedded platform, shows a certain reduction in time consumption in all three stages, resulting in a 4.5% improvement in overall detection speed. After acceleration with TensorRT, tAodeMar significantly reduces the inference stage time by 61.56%, doubling the overall detection speed to 37.45 FPS. Although the detection accuracy on mAP@.5 decreased by 0.22, the decrease was small at only 0.23%, validating the effectiveness of the tAodeMar model.

[0089] Table 7 Comparison of detection performance of the model before and after acceleration on the MVDD13 test set.

[0090] Note: For ease of reference, the TensorRT-accelerated eAodeMar model will be named tAodeMar. To compare the real-time performance of the models before and after acceleration, tests were conducted using real-world video footage from the "Zhihai 2" USV in the harbor basin and SMD shipborne video. The results are shown in Table 8. The AodeMar, eAodeMar, and tAodeMar models achieved detection speeds of 3.60, 5.76, and 18.31 FPS in the real-world video footage, respectively, and 6.93, 7.33, and 28.57 FPS in the SMD video stream, respectively. It can be seen that the models accelerated by TensorRT achieved significant improvements in detection speed across both types of video. The tAodeMar model's detection speed in the real-world video footage was slightly lower than the requirements for real-time video detection (video FPS is generally not less than 20 FPS), but it met the real-time requirements in the SMD video. This is largely due to the complexity of the scene; a more complex background generates more false detection boxes, thus consuming more learning time. Figure 7 As shown, it is evident that the background information in the actual video is more complex than that in the SMD video. In conclusion, the accelerated model significantly improves detection speed while maintaining a slight decrease in detection accuracy within an acceptable range, validating the effectiveness and practical application capability of the proposed model.

[0091] Table 8 compares the detection speed of the model before and after acceleration in harbor basin real-world sampling and SMD shipborne video.

[0092] To qualitatively compare the models before and after acceleration in terms of visual detection performance, four frames were selected from the actual video footage, including occlusion and local target challenge scenarios under two lighting conditions: front lighting (row 1) and backlighting (rows 2-4). The detection results are as follows: Figure 8 As shown, in the first row of the scene with occlusion caused by direct sunlight, both models accurately detected two severely occluded vessels, indicating that the lightweight model maintained its occlusion detection capability. However, due to the dim lighting caused by direct sunlight, false detections occurred. Specifically, a house with windows on the left-hand bank extending into the water was easily mistaken for a passenger ship. This is because its lightweight nature results in relatively low model complexity, leading to a decrease in its feature extraction capability, resulting in both false and missed detections. For the two targets in the second row and the local target on the right side of the third row, both models achieved accurate identification and localization with high confidence. However, for the local targets on the left side of the third and fourth rows, although they occupied a large area in the image, both models had low confidence, and the eAodeMar model even missed detection. This is because a larger area in the image does not necessarily lead to better detection performance; rather, the more complete and clear the salient features of the sailboat (i.e., sails or sail frames) are, the better the detection performance, which aligns with the characteristics of human visual recognition.

[0093] In summary, the eAodeMar model proposed in this invention can accurately locate and quickly identify ships in a scene. Although the detection effect is affected to some extent, the detection speed is greatly improved.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight occlusion detection method based on an attention mechanism, characterized in that, Includes the following steps: Obtain the dataset of ship targets under the occlusion conditions to be processed; The images in the dataset are preprocessed; The preprocessed dataset is divided into training and test sets according to a certain ratio; Construct an eAodeMar model for feature extraction, feature fusion, and prediction of the location and category of a target. The eAodeMar model includes a location enhancement module and a lightweight multi-scale feature semantic association module. The eAodeMar model is trained based on the training set to obtain the trained eAodeMar model; The test set is input into the trained eAodeMar model to achieve the detection of unmanned surface vessels at sea under occlusion conditions; The trained eAodeMar model is deployed to the tensor inference framework in Jetson Xavier NX, which performs vertical and horizontal merging of the same operations in the trained eAodeMar model to reduce the complexity of the network.

2. The lightweight occlusion ship detection method based on attention mechanism according to claim 1, characterized in that, The eAodeMar model includes a backbone network for feature extraction from images, a neck network for fusing the extracted features, and a detection head network for predicting the location and category of the target; the backbone network, the neck network, and the detection head network are cascaded in sequence.

3. The lightweight occlusion ship detection method based on attention mechanism according to claim 2, characterized in that, The backbone network architecture includes a first CBS module, a second CBS module, a first C3 module, a third CBS module, a second C3 module, a fourth CBS module, a third C3 module, a fifth CBS module, a location enhancement module, and an SPPF module, which are connected in sequence. The network architecture of the neck network includes a sixth CBS module, a first upsampling module, a first feature splicing module, a fourth C3 module, a seventh CBS module, a second upsampling module, a second feature splicing module, a fifth C3 module, an eighth CBS module, a third feature splicing module, a first lightweight multi-scale feature semantic association module, a ninth CBS module, a fourth feature splicing module, and a second lightweight multi-scale feature semantic association module, all connected in sequence. The network architecture of the detection head network includes a first convolutional layer, a second convolutional layer, and a third convolutional layer.

4. The lightweight occlusion ship detection method based on attention mechanism according to claim 3, characterized in that, The second C3 module is connected to the second feature splicing module, the third C3 module is connected to the first feature splicing module, the SPPF module is connected to the sixth CBS module, the fourth C3 module is connected to the third feature splicing module, the fifth C3 module is connected to the fourth feature splicing module, the fifth C3 module is connected to the third convolutional layer, the first lightweight multi-scale feature semantic association module is connected to the second convolutional layer, and the second lightweight multi-scale feature semantic association module is connected to the first convolutional layer.

5. The lightweight occlusion ship detection method based on attention mechanism according to claim 2, characterized in that, The location enhancement module includes a first Ghost convolution module, an RCAB module, a fifth feature concatenation module, a second Ghost convolution module, and a third Ghost convolution module. The first Ghost convolution module, the RCAB module, the fifth feature concatenation module, and the second Ghost convolution module are connected in sequence. The first Ghost convolution module is connected to the data input end, and the third Ghost convolution module is connected to the data input end on one hand and to the fifth feature concatenation module on the other hand.

6. The lightweight occlusion ship detection method based on attention mechanism according to claim 5, characterized in that, The RCAB module consists of two consecutive 2D convolutional layers, batch normalization and SiLU activation function, CA module and residual connection.

7. The lightweight occlusion ship detection method based on attention mechanism according to claim 2, characterized in that, The network architecture of the first lightweight multi-scale feature semantic association module and the second lightweight multi-scale feature semantic association module is the same. The first lightweight multi-scale feature semantic association module includes an SPP module, a fourth Ghost convolution module, an STR module, a sixth feature concatenation module, a fifth Ghost convolution module, and a sixth Ghost convolution module. The SPP module, the fourth Ghost convolution module, the STR module, the sixth feature concatenation module, and the fifth Ghost convolution module are connected in sequence. The SPP module is connected to the data input end. The sixth Ghost convolution module is connected to the data input end on one hand and to the sixth feature concatenation module on the other hand.

8. The lightweight occlusion detection method for ships based on attention mechanisms according to claim 5 or 7, characterized in that, The network architecture of the first, second, third, fourth, fifth, and sixth Ghost convolutional modules is the same. The workflow of the first Ghost convolutional module is as follows: The input feature map is convolved to generate a feature map with half the number of channels. The feature map with half the number of channels is subjected to depthwise convolution to obtain a depthwise convolutional feature map. The feature map with half the number of channels and the feature map of the depthwise convolution are concatenated together to obtain the output of the first Ghost convolution module.

9. The lightweight occlusion ship detection method based on attention mechanism according to claim 1, characterized in that, The tensor inference framework merges identical operations in the trained eAodeMar model both vertically and horizontally, including: The trained eAodeMar model on the workstation is converted into a model in the Open Neural Network Exchange format; Based on the model of the open neural network exchange format, a tensor inference framework is created; Based on the tensor inference framework, a computation graph engine is created using a Builder in a unified computing device architecture environment, and the graph engine is deserialized to obtain a deserialized file. Based on the deserialized file, the inference output is obtained; The location of the ship target is obtained by encoding the output reasoning result; The video image detection task is completed by using non-maximum suppression to remove redundant prediction boxes in the location of the ship target.

Images