Lightweight inland ship detection algorithm based on improved YOLOv8n

Abstract

The invention relates to the technical field of computer vision, in particular to an improved YOLOv8n lightweight inland ship detection algorithm. According to the technical scheme, the method comprises the steps of designing a lightweight detection head EFSDH and replacing an original decoupling detection head, introducing a neck network lightweight method slip-check, designing a global channel adaptive amplitude pruning algorithm GCAMBP for channel pruning, and designing a space-channel correlation knowledge distillation method SCCKD for knowledge distillation. According to the light-weight YOLO model for inland ship detection, the detection performance of the model is successfully improved, meanwhile, light weight is achieved, and the problems of real-time performance and accuracy in a computing resource limited environment are solved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and particularly to a lightweight inland ship detection algorithm for improving YOLOv8n. Background Art

[0002] Inland shipping plays an irreplaceable role in ensuring cargo transportation, reducing transportation costs, promoting regional economic integration, etc., and has become one of the important logistics methods in the modern economy. However, while providing convenience, frequent ship accidents also pose a great threat to shipping safety. At the same time, the problems of excessive emissions and water pollution from ships also have an impact on the environment. Ship detection technology can not only effectively improve the safety of waterways and avoid collision accidents, but also monitor ship emissions and environmental pollution. As the most important part of the shipping safety and environmental protection guarantee system, the ship detection algorithm can accurately identify and locate ship targets, timely warn and avoid potential safety risks, and at the same time provide technical support for realizing green shipping and environmental protection. Therefore, the research on inland ship detection algorithms is of great significance for improving waterway management efficiency, ensuring shipping safety, and promoting environmental protection and sustainable development.

[0003] At present, ship detection mainly relies on computer vision object detection technology. Ship object detection algorithms based on convolutional neural networks can be mainly divided into two categories: two-stage and one-stage. Two-stage detection algorithms, such as Faster R-CNN, Mask R-CNN, etc., are usually suitable for tasks with complex environments and high-precision requirements. Although these algorithms can provide higher detection accuracy, due to the large amount of computation, they are relatively slow in speed and have high requirements for computing resources. One-stage object detection algorithms such as YOLO, SSD, DETR, etc. usually have advantages in detection speed and are suitable for application scenarios with high real-time requirements. Therefore, many scholars mainly adopt this type of method in the research of ship detection. In recent years, in response to the problem of ship object detection, many scholars have gradually conducted in-depth research and proposed many innovative methods. Liu Fen et al. improved the anchor box generation algorithm of YOLOv5 through median-optimized K-means clustering and proposed the MIoU regression loss function to optimize the bounding box regression process, solving the problems of inaccurate matching and missed and false detections of infrared ship targets in complex sea-sky backgrounds. Luo Fang et al. proposed a scale-adaptive receptive field detection method SAF-YOLOX based on YOLOX, enhanced multi-scale feature fusion through a bidirectional feature pyramid, designed an adaptive feature enhancement module to reduce redundant information, and used a multi-way parallel receptive field detection head to extract feature information adapted to the target scale, solving the problems of missed and false detections in multi-scale object detection. Yang et al. proposed a feature fusion module for splitting the bidirectional feature pyramid network to reduce the interference of complex backgrounds on SAR ship images. Although the above improved algorithms have improved the detection accuracy to a certain extent, they have also increased the overall parameters and computation amount of the model. To meet their applications on devices with limited computing resources, other more lightweight detection algorithms need to be considered.

[0004] In the research on the lightweight technology of ship detection algorithms, Xie Youchen et al. proposed an optimized algorithm YOLO-WIT for small and weak ship targets, optimized and reduced the detection head, reduced the model volume while restricting the maximum offset of the anchor box, and designed a loss function for composite distance similarity measurement to improve the positioning accuracy of small and weak ship targets. Zhang Bingyan et al. replaced the C3 module in YOLOv5 with the FasterNext module, reducing the model parameters and computational cost, and adding the NAM attention mechanism and proposing an effective intersection over union loss function with a penalty term to improve the detection performance. Zhang Shen et al. took YOLOv7 as the baseline model and proposed a lightweight ship detection algorithm using MobileNetV3 as the backbone feature extraction network and introducing the WiseIoU optimized loss function. Zhao et al. based on the YOLOv5 model, designed an improved ShuffleNetv2 network integrating the attention mechanism as the backbone feature extraction, and removed the feature map with 32x downsampling in the detection head part, replacing the standard convolution in the detection head with Ghost convolution, achieving the effect of reducing the algorithm's parameter and computational amount while improving the ship detection accuracy.

[0005] The original decoupled detection head design of YOLOv8n has some deficiencies. First, the model uses separate detection heads for features of different scales and fails to fully utilize multi-scale information. This is particularly important in ship detection because ship targets often exhibit significant scale variations due to factors such as perspective, distance, and occlusion, affecting the consistent detection accuracy and robustness for targets such as small passenger ships and large general cargo ships. Second, the decoupled detection head processes the classification and regression tasks separately, and the convolution parameters of the two are not shared, further increasing the computational load. The parameter quantity of the detection head part accounts for about 25% of the total model parameter quantity.

[0006] In summary, in order to enable the algorithm to have a high accuracy and inference efficiency when running on devices with limited computing resources, therefore, we propose a lightweight inland river ship detection algorithm that improves YOLOv8n. Summary of the Invention

[0007] The purpose of the present invention is to address the problems in the background technology, such as large ship detection model parameter quantity and computational amount, low algorithm efficiency, and high hardware requirements, and propose a theme of a lightweight inland river ship detection algorithm that improves YOLOv8n.

[0008] The technical solution of the present invention: A lightweight inland river ship detection algorithm that improves YOLOv8n includes the following steps:

[0009] Design a lightweight detection head EFSDH. The detection head completes channel alignment through an independent 1×1 convolution, introduces two shared 3×3 detail enhancement convolutions DEConv to enable the same weights for feature maps of all scales, and also adds a Scale layer to dynamically adjust the scale of the feature maps. Among them, DEConv integrates central difference convolution, horizontal difference convolution, vertical difference convolution, and direction enhancement convolution operations, and uses group normalization GN to replace batch normalization BN;

[0010] Introduce a lightweight method for the neck network, slim-neck, which uses GSConv and the GSBottleneck and VoV-GSCSP modules built around it to replace the original Conv, Bottleneck, and C2f modules in the YOLOv8n neck network respectively, further optimizing the computational efficiency of feature extraction and fusion. GSConv enhances the information interaction between channels while reducing the computational amount through the grouped convolution and channel shuffle mechanisms; GSBottleneck inherits the efficient characteristics of GSConv and adds a multi-layer feature stacking structure to improve the local feature expression ability; the VoV-GSCSP module combines the characteristics of the cross-stage partial network (CSP) with a lightweight feature aggregation design to further improve the fusion efficiency of multi-scale features.

[0011] Design a global channel adaptive magnitude pruning algorithm GCAMBP for channel pruning. First, calculate the sum of the squares of the weights of each channel, then sort the sum of the squares of the channel weights in ascending order, define the GCAMBP score with dynamic normalization, select the channel with the lowest score from the global scores for pruning and remove the corresponding convolution kernel until the preset pruning rate is met, synchronously adjust the input channels of the subsequent layers of the network, and restore the model performance through fine-tuning after pruning;

[0012] Design a spatial-channel correlation knowledge distillation method SCCKD for knowledge distillation. Model the feature relationships of the model by constructing the spatial correlation matrix and channel correlation matrix of the student and teacher models. The loss function includes three parts: the loss of normal training, the inter-channel correlation loss, and the spatial correlation loss. The inter-channel correlation loss and the spatial correlation loss are adjusted by the corresponding weight factors for their loss ratios.

[0013] Optionally, in the step of designing the lightweight detection head EFSDH and replacing the original detection head, the parameter fusion mechanism formula of the DEConv is:

[0014]

[0015] In the formula, F out represents the output feature, F in represents the input feature, K i represents each convolution kernel, K cvtRepresents the sum of each convolutional kernel.

[0016] Optionally, in the introduced lightweight convolution GSConv, the time complexity formula of the GSConv calculation is:

[0017]

[0018] In the formula, W and H respectively represent the width and height of the input feature map, K1*K2 is the convolutional kernel size, C1 and C2 are the number of channels of the input and output feature maps respectively. It combines the standard convolution and depthwise separable convolution and introduces the shuffle operation to achieve efficient feature interaction. The GSBottleneck module designed by combining GSConv with the residual structure further optimizes the feature extraction process. The VoV-GSCSP module designed based on GSBottleneck through a parallel path design realizes the effective combination of deep feature extraction and shallow feature retention, providing strong support for complex feature fusion.

[0019] Optionally, the specific steps of using the global channel adaptive magnitude pruning algorithm GCAMBP for channel pruning are as follows:

[0020] Input the original model and set the pruning rate, where the original model contains convolutional layers, and the weight matrix of the convolutional layer is Represents a three-dimensional feature, C in Represents the number of input channels, C out Represents the number of output channels, and k is the convolutional kernel size;

[0021] For each convolutional layer in the model, according to the formula:

[0022]

[0023] Calculate the importance score S l,j of the j-th output channel of the l-th layer, and obtain the importance scores of each channel of each layer;

[0024] Sort the channel importance scores of the l-th layer in ascending order, and then calculate the dynamically normalized GCAMBP score for the sorted channel k:

[0025]

[0026] In the formula, S l represents the sum of the squares of the channel weights tensor of the l-th layer, S l [k] represents the sum of the squares of the channel weights corresponding to the index k. Assume that the sum of the squares of the weights is sorted in ascending order according to the given index, that is, when u ≤ v, S l [u] ≤ S l [v].

[0027] Summarize the GCAMBP scores of all layers and sort the channel scores globally;

[0028] Select the channel with the lowest score from the global scores for pruning, remove the corresponding convolutional kernel, and update the current pruning rate according to the number of pruned channels until the current pruning rate meets the set pruning rate, and synchronously adjust the input channels of the subsequent layers of the network;

[0029] After pruning, restore the model performance through fine-tuning to achieve efficient compression and inference optimization of the model.

[0030] Optionally, in the knowledge distillation using the spatial-channel correlation knowledge distillation method SCCKD, the feature maps of the teacher model and the student model are respectively and where, represents a three-dimensional feature, C and C′ represent the number of channels of the teacher and student models respectively, H and W are the height and width of the spatial dimension of the feature map, flatten the feature map F T and F S , and obtain the spatial correlation matrix S T and S S :

[0031] S T = f C (F T )·f C (F T ) T

[0032] S S = f C (F S )·f C (F S ) T

[0033] In the formula, f C (F T ) flattens the feature map into a matrix of [H×W, C], and S T and S S respectively describe the relationships between different spatial positions in the feature maps of the teacher model and the student model. Normalize the two matrices S T and S S , and the spatial correlation loss L SC is achieved by minimizing the L2 distance between the spatial correlation matrices of the student and teacher models:

[0034]

[0035] Wherein, H×W is the spatial resolution of the feature map, is the square of the L2 norm.

[0036] Optionally, flatten the feature map F T and F S , and obtain the channel correlation matrix C T and C S ;

[0037] C T = f S (F T )·f S (F T ) T

[0038] C S = f S (F S )·f S (F S ) T

[0039] Wherein, f S (F T ) flattens the feature map into a matrix of [C, H×W], C T and C S respectively describe the collaboration relationship between different channels in the feature maps of the teacher model and the student model. Normalize the two matrices C T and C S , and the channel - to - channel correlation loss L CC is achieved by minimizing the L2 distance between the channel - to - channel correlation matrices of the student and teacher models:

[0040]

[0041] Wherein, C is the number of channels, is the square of the L2 norm;

[0042] Based on the above spatial correlation loss and channel correlation loss, the total loss function of the student model in the SCCKD knowledge distillation process can be expressed as:

[0043] L SCCKD = L train + αL CC + βL SC

[0044] Wherein, L train is the loss during normal training, L CC is the channel - to - channel correlation loss, L SC is the spatial correlation loss, and α and β are weight factors.

[0045] Compared with the prior art, the present invention has the following beneficial technical effects:

[0046] Improved multi-scale detection performance: EFSDH enhances the feature consistency of multi-scale detection, which helps to enhance the robustness and detection accuracy of ship targets of different sizes and perspectives. At the same time, the added Scale layer can dynamically adjust the scale of the feature map, further enhancing the model's multi-scale perception ability, enabling it to better handle the situation of variable target scales in inland river ship detection.

[0047] Enhanced feature expression ability and improved stability: The Detail Enhancement Convolution (DEConv), as the core module of EFSDH, can capture the detail information in different directions and scales of the feature map. By integrating multiple differential convolution operations to run in parallel to capture multi-directional detail information, it further improves the feature expression ability. Moreover, using Group Normalization (GN) instead of Batch Normalization (BN) eliminates the dependence on batch size, enhances the adaptability to multi-scale features, and avoids the normalization bias caused by batch dependence and inconsistent distributions, especially improving the stability of the model during small-batch training or inference.

[0048] Reduced computational complexity and enhanced resource adaptability: Compared with the YOLOv8n detection head, EFSDH reduces the number of parameters to 51% of the original decoupled detection head, and uses the slim-neck method to lightweight the neck network to further optimize the model efficiency. On this basis, combined with distillation technology, it improves the detection accuracy of the model after pruning, achieving a good balance between performance and computational complexity. This makes the model easier to be applied on systems with limited computational resources, expands the application scenarios and scope of the model, and reduces the requirements for hardware devices.

[0049] Improved global optimization performance: GCAMPB realizes cross-layer channel pruning optimization through dynamic normalization and global sorting. While significantly reducing the computational complexity of the model, it retains the feature expression ability of important channels; the pruned model has better hardware adaptability and resource utilization rate, effectively balancing performance and efficiency, especially performing well in scenarios with limited resources. Brief Description of the Drawings

[0050] Figure 1 It is the network structure diagram of YOLO-LISD;

[0051] Figure 2 It is the comparison diagram of the YOLOv8n decoupled detection head structure and the EFSDH detection head structure;

[0052] Figure 3 It is the structure diagram of the DEConv module;

[0053] Figure 4It is the structural diagram of the GSConv module;

[0054] Figure 5 It is the structural diagrams of the GSBottleneck module and VoV-GSCSP;

[0055] Figure 6 It is the flowchart of model pruning;

[0056] Figure 7 It is the schematic diagram of SCCKD;

[0057] Figure 8 It is the diagram of common ship types;

[0058] Figure 9 It is the diagram of data augmentation;

[0059] Figure 10 It is the comparison diagram of the detection effects of YOLOv8n and YOLO-LISD. Specific implementation manners

[0060] Next, the technical solutions of the present invention will be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention.

[0061] Generally, the components of the embodiments of the present invention described and shown in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed present invention, but merely represents the selected embodiments of the present invention.

[0062] Embodiment

[0063] In this embodiment, four improvements are made to YOLOv8n. As Figure 1 shown, a lightweight inland river ship detection algorithm for improving YOLOv8n is proposed, including four steps: designing a lightweight detection head EFSDH, introducing a lightweight convolution GSConv, designing a global channel adaptive magnitude pruning algorithm GCAMBP for channel pruning, and designing a spatial-channel correlation knowledge distillation method SCCKD for knowledge distillation. The experimental results on the SeaShips image dataset after data augmentation show that YOLO-LISD not only effectively improves the detection accuracy, but also the model is light enough to facilitate edge deployment. Compared with other algorithms, the robustness and efficiency of the improved network are demonstrated. The following is a detailed description of this algorithm.

[0064] I. Direct lightweight network design

[0065] 1.1. Lightweight detection head EFSDH

[0066] There are some deficiencies in the original decoupled detection head design of YOLOv8n. First, the model uses separate detection heads for features of different scales, failing to fully utilize multi-scale information. This is particularly important in ship detection because ship targets often exhibit significant scale variations due to factors such as perspective, distance, and occlusion, affecting the consistent detection accuracy and robustness for targets such as small passenger ships and large general cargo ships. Second, the decoupled detection head processes the classification and regression tasks separately, and the convolution parameters of the two are not shared, further increasing the computational load. The number of parameters in the detection head part accounts for about 25% of the total number of model parameters. Therefore, optimizing the detection head network structure is of great significance for improving model performance and efficiency.

[0067] To overcome these deficiencies, this application designs an efficient feature-sharing detection head EFSDH (Efficient Feature Sharing Detection Head) that integrates the Detail-Enhanced Convolution (DEConv). Its structure is as follows Figure 2 shown. The left side is the structure of the YOLOv8n decoupled detection head, and the right side is the EFSDH detection head structure. Its core idea is to use the detail-enhanced convolution as a shared convolution, enabling different detection heads to share the same convolution kernel parameters. This design inspiration comes from observing that objects detected at different feature levels have certain similarities in scale, and the detail-enhanced convolution effectively captures the detail information in different directions and scales of the feature map.

[0068] First, the feature-sharing mechanism aims to reduce the overall number of parameters in the model. EFSDH first completes channel alignment through an independent 1×1 convolution, enabling unified processing of features of different scales. Secondly, two shared 3×3 detail-enhanced convolutions are introduced to achieve the sharing of the same weights for feature maps of each scale, thus maintaining a consistent feature extraction pattern. This design not only reduces the number of parameters but also improves the feature consistency of multi-scale detection, contributing to enhancing the robustness and detection accuracy for ship targets of different sizes and perspectives. The feature-sharing mechanism simplifies the training process and improves the stability and efficiency of training. In addition, to adapt to different target scales, a Scale layer is added to dynamically adjust the scale of the feature map, further enhancing the multi-scale perception ability. Compared with the YOLOv8n detection head, EFSDH reduces the number of parameters to 51% of the original decoupled detection head, making the model easier to apply on systems with limited computational resources.

[0069] Secondly, the Detail Enhancement Convolution (DEConv), as the core module of EFSDH, further improves the feature expression ability. DEConv can capture the detail information in different directions and scales of the feature map. DEConv integrates central difference convolution, horizontal difference convolution, vertical difference convolution, and direction enhancement convolution operations. By running in parallel, it captures multi-directional detail information. Since the feature distributions at different scales vary greatly, if batch normalization (BN) is used in the shared normalization layer of DEConv, normalization bias may be caused by batch dependence and inconsistent distributions, especially in the small batch training or inference stage, where it performs unstably. To solve this problem, DEConv uses group normalization (GN) instead of BN, which not only eliminates the dependence on batch size but also enhances the adaptability to multi-scale features.

[0070] The parameter fusion mechanism formula of DEConv is:

[0071]

[0072] Where, F out represents the output feature, F in represents the input feature, K i represents each convolutional kernel, and K cvt represents the sum of each convolutional kernel. Through this mechanism, the computational load and memory usage are reduced, while the feature expression ability is improved.

[0073] 1.2, Lightweight Neck Network slim-neck

[0074] Slim-neck is a lightweight method for optimizing the neck network of the object detection model. Through efficient convolutional operations such as grouped and shuffle convolution (GSConv) and structure optimization, it significantly reduces the computational load and the number of parameters, while improving the feature extraction ability. It was initially applied to the field of autonomous driving. To achieve the lightweight of the YOLOv8n neck network, this paper proposes a module replacement scheme, using GSConv and its derivatives GSBottleneck and VoV-GSCSP modules to replace the original standard convolution Conv, Bottleneck, and C2f modules respectively. These three modules work together to build an efficient feature extraction and fusion framework. The structure of GSConv is as Figure 4 shown

[0075] The GSConv module realizes efficient feature interaction between channels by integrating standard convolution and depthwise separable convolution and introducing the shuffle operation. It divides the input features into two parts. One part extracts high-level features through standard convolution, and the other part extracts fine-grained features through depthwise separable convolution. Subsequently, the two are fused and the information interaction between channels is enhanced through the shuffle module. This design reduces the computational complexity and parameters while retaining rich feature information, providing an efficient computational basis for subsequent modules. The time complexity of GSConv calculation is usually defined as:

[0076]

[0077] where W and H represent the width and height of the input feature map respectively, K1*K2 is the convolution kernel size, and C1 and C2 are the number of channels of the input and output feature maps respectively.

[0078] The GSBottleneck module designed by combining GSConv with the residual structure further optimizes the feature extraction process. It combines grouped convolution, feature rearrangement, and residual connection, improving the feature expression ability and reducing the computational cost. At the same time, the VoV-GSCSP module based on GSBottleneck and designed with parallel paths realizes the effective combination of deep feature extraction and shallow feature retention, providing strong support for complex feature fusion. At the same time, the cascade feature fusion strategy is adopted to efficiently integrate multi-scale features in the channel dimension, and finally the feature representation is further optimized through the fused convolution. This modular design enables the model to maintain excellent feature extraction and fusion capabilities while reducing the computational resource requirements. The GSBottleneck module and the VoV-GSCSP structure are as Figure 5 shown.

[0079] The effective recognition of complex geometric shapes and detailed features such as the ship's hull, deck, mast, and superstructure is also the key to improving the accuracy of ship detection and classification tasks. These features require both comprehensive analysis of the global shape and precise capture of local details. The lightweight feature processing framework constructed by GSConv, GSBottleneck, and VoV-GSCSP realizes the effective aggregation of local and global features while significantly reducing the computational complexity and the number of parameters of the model, and has strong feature extraction and fusion capabilities.

[0080] II. Model Compression

[0081] 2.1 Channel Pruning

[0082] Model pruning aims to reduce the computational complexity and the number of parameters by removing unimportant parts of the model while maintaining the model accuracy as much as possible, thus achieving model lightweighting. The literature "Layer-Adaptive Sparsity for the Magnitude-Based Pruning" proposed the Layer-Adaptive Magnitude-Based Pruning (LAMP) algorithm. By calculating the squared values of the weights of each layer and normalizing to measure the importance, and then globally sorting the weight scores, individual low-score weights are removed to achieve unstructured pruning. Since the pruning unit is an individual weight, the weight matrix will become sparse, resulting in poor adaptability to hardware.

[0083] To overcome the limitations of the above pruning algorithm, this paper proposes a Global Channel-Adaptive Magnitude-Based Pruning (GCAMBP) algorithm to optimize the model compression effect. GCAMBP is a structured pruning method that globally optimizes based on the importance scores of channels. Its core idea is to comprehensively balance the pruning strength of different layers through a dynamic normalization mechanism and global sorting to achieve channel-level model compression. Assume that the weight matrix of the current convolutional layer is represents a three-dimensional feature, where C in and C out represent the number of input channels and the number of output channels respectively, and k is the convolution kernel size. GCAMBP first calculates the sum of the squared weights of each channel:

[0084]

[0085] In the formula, S l,j represents the sum of the squared weights of the j-th output channel in the l-th layer. To more accurately reflect the relative importance of the channels within the layer, first sort the sum of the squared channel weights in ascending order, and then define the GCAMBP score for each channel k:

[0086]

[0087] In the formula, S l represents the tensor of the sum of the squared channel weights, and S l [k] represents the sum of the squared channel weights corresponding to the index k. Assume that the sum of the squared weights is sorted in ascending order according to the given index, that is, when u ≤ v, S l [u] ≤ S l [v].

[0088] The core of the dynamic normalization mechanism is that the denominator of the current channel only contains channels with higher or equal importance than it, making the score more accurately depict the importance ranking of channels. After completing the score calculation within the layer, the GCAMBP scores of all layers are aggregated, and the channel scores are sorted globally. The channel with the lowest score is selected from the global scores for pruning, and the corresponding convolutional kernel is removed until the preset pruning rate is met, and the input channels of the subsequent layers of the network are adjusted synchronously. After pruning, the model performance is restored through fine-tuning, and finally, the efficient compression and inference optimization of the model are achieved. The GCAMBP method not only realizes dynamic channel score adjustment within the layer but also effectively balances the pruning intensity between different layers through global sorting, while significantly reducing the computational complexity and the number of parameters and maintaining the stability of the model performance. The channel pruning process based on the GCAMBP score is as Figure 6 shown.

[0089] 2.2 Knowledge Distillation

[0090] As an efficient model compression technology, the core of knowledge distillation is to transfer the knowledge of a large teacher model to a lightweight student model, so that the student model can still maintain a high level of detection accuracy while significantly reducing the computational overhead. The feature maps of the teacher model are rich in spatial and channel information. If these information are not protected when transferred to the student model, the potential of the student model may be limited.

[0091] "Exploring Inter-Channel Correlation for Diversity-preserved Knowledge Distillation" proposes an inter-channel correlation knowledge distillation method (Inter-Channel Correlation for Knowledge Distillation, ICKD), which only considers the correlation of feature maps in the channel dimension and ignores the context information in the spatial dimension, which may have limitations in visual tasks that require spatial perception.

[0092] To improve the above limitations, this paper proposes a spatial-channel correlation knowledge distillation method (Spatial-Channel Correlation Knowledge Distillation, SCCKD). By calculating the spatial and channel correlation matrices, and using these matrices to propose a spatial loss function and a channel loss function. The specific process is as Figure 7 shown

[0093] In the SCCKD knowledge distillation, the feature maps of the teacher model and the student model are respectively denoted as and Among them, represents a three-dimensional feature, C and C′ respectively represent the number of channels of the teacher and student models, and H and W are the height and width of the spatial dimension of the feature map. First, flatten the feature map F along the channel dimension T and F S , and obtain the spatial correlation matrix S T and S S through matrix multiplication:

[0094] S T = f C (F T )·f C (F T ) T (4)

[0095] S S = f C (F S )·f C (F S ) T (5)

[0096] Among them, f C (F T ) flattens the feature map into a matrix of [H×W, C], and S T and S S respectively describe the relationships between different spatial positions in the feature maps of the teacher model and the student model. Normalize the two matrices S T and S S , and the spatial correlation loss L SC is achieved by minimizing the L2 distance between the spatial correlation matrices of the student and teacher models:

[0097]

[0098] Next, flatten the feature map F T and F S along the spatial dimension, and obtain the channel correlation matrix C T and C S .

[0099] C T = f S (F T )·f S (F T ) T (7)

[0100] C S = f S (F S )·f S (F S )T (8)

[0101] Among them, f S (F T ) flattens the feature map into a matrix of [C, H×W], where C T and C S describe the collaborative relationships between different channels in the feature maps of the teacher model and the student model respectively. Normalize the two matrices of C T and C S , and the inter-channel correlation loss L CC is achieved by minimizing the L2 distance between the inter-channel correlation matrices of the student and teacher models:

[0102]

[0103] Based on the above spatial correlation loss and inter-channel correlation loss, the total loss function of the student model in the SCCKD knowledge distillation process can be expressed as:

[0104] L SCCKD = L train + αL CC + βL SC (10)

[0105] In the formula, L train is the loss of normal training, L CC is the inter-channel correlation loss, L SC is the spatial correlation loss, and α and β are weight factors.

[0106] The core feature of SCCKD is that by jointly optimizing the spatial correlation loss and the inter-channel correlation loss, the student model can comprehensively learn the feature relationships of the teacher model from two dimensions. On the one hand, the spatial correlation part helps the student model capture the context information between global spatial positions and improve the perception ability of complex structures; on the other hand, the inter-channel correlation part ensures the collaboration of the student model in the channel dimension, thus retaining the diversity of features. The combination of the two enables the student model not only to efficiently reproduce the feature expressions of the teacher model but also to improve the generalization performance of the model in classification and object detection tasks.

[0107] Experiments and Analysis

[0108] 3.1 Experimental Environment

[0109] To evaluate the detection effect of the improved lightweight YOLO model on ship detection tasks, the operating system of this experimental platform is Windows 10, the processor is Intel Core i5-10400F@2.9GHz, it is equipped with 32GB of memory and an Nvidia GeForce RTX 3090Ti graphics card with a video memory capacity of 24GB. Python 3.8 is used as the programming language, and the deep learning framework is Pytorch 2.2.2+cu118. The input image size is 640×640. The key parameters for this model training are shown in Table 1.

[0110] Table 1 Hyperparameter settings

[0111]

[0112] 3.2 Experimental dataset

[0113] In this embodiment, the publicly available dataset SeaShips is used to train and evaluate the object detection model to ensure the repeatability and reliability of the experimental results. The SeaShips dataset covers a variety of scenarios and target types and is widely used in the field of ship target detection. All images are collected by cameras of the video surveillance system deployed on the coastline, covering various changes in imaging conditions, such as the hull part, proportion, illumination, perspective, background, and occlusion situation. The SeaShips dataset contains a total of 7000 samples, including six common ship types, such as Figure 8 shown, which are bulk carriers, container ships, fishing boats, general cargo ships, ore carriers, and passenger ships respectively.

[0114] To further improve the generalization ability of the model, first, the dataset is screened for quality, and some samples with inaccurate annotations or poor image quality are excluded. 1000 images are randomly selected from the remaining dataset as the validation set, and 1000 images are used as the test set. Then, among the remaining samples, various data augmentation operations such as noise addition, scale transformation, random erasing, random brightness adjustment, and horizontal flipping are randomly used to expand the dataset to 8000 images as the training set, forming a final complete dataset of 10000 images. Figure 9 The data augmentation method used in this embodiment is shown as follows.

[0115] Evaluation metrics

[0116] In this embodiment, the main research is to improve the lightweight and accuracy of the inland ship detection algorithm based on the YOLOv8n algorithm. Therefore, three items, namely precision (P), recall (R), and mean average precision (mAP), are selected as performance evaluation indicators. In the model lightweight task, the number of model parameters (Params), the number of floating-point operations (FLOPs), and the model size (Model size) should also be used as evaluation indicators. Considering the deployment of the model to terminal devices, the number of frames per second of model inference (FPS) is used as the evaluation indicator for the real-time performance of the model. Therefore, there are seven evaluation indicators in this experiment, namely P, R, mAP50:95, Params, GFLOPs, Model size, and FPS. The formulas for precision, recall, and mean average precision are as follows:

[0117]

[0118]

[0119] In the formula: where TP represents the examples where the positive samples are predicted as positive; FP represents the examples where the negative samples are predicted as positive; FN represents the examples where the positive samples are mispredicted as negative; TN represents the examples where the negative samples are predicted as negative. N represents the number of detection categories. During the calculation of FPS, to ensure accuracy and utilize the parallel computing power of the graphics card, the batch-size is set to 32, and 200 warm-up trainings are performed to make it reach the best state. Finally, 1000 tests are conducted and the average value is taken.

[0120] Ablation experiment

[0121] To verify the effectiveness of the YOLO-LISD algorithm proposed in this embodiment, in this embodiment, the original YOLOv8n model is used as the baseline, and multiple groups of ablation experiments are designed on the SeaShips dataset to evaluate the impact of each improvement on the model performance. The results of the ablation experiment are shown in Table 2.

[0122] Table 2 Ablation experiment

[0123]

[0124] Through ablation experiments, the effectiveness of the method proposed in this embodiment in improving the target detection performance of inland river ships is strongly verified. According to the experimental results in Table 2, replacing the original detection head with the EFSDH detection head reduces the number of parameters and the computational volume by about 20%, and the average detection accuracy is improved by 0.8%, and the FPS is increased by 44, making the model more suitable for the inland river ship detection task and significantly improving the accuracy of ship detection. Introducing Slim-neck further reduces the number of parameters and the computational volume of the feature fusion network, and the detection accuracy is further improved. Although this optimization has led to a slight decrease in FPS to some extent, subsequent improvement measures have effectively improved the overall detection effect. Using channel pruning guided by the GCAMBP score not only significantly reduces the number of parameters and computational overhead, but also compensates for the PFS loss caused by the previous improvements. Although pruning reduces the detection accuracy of the model to some extent, subsequent distillation improvement measures have effectively improved the overall detection effect. Compared with the original YOLOv8n model, the number of parameters is reduced from 3.01M to 0.95M, a decrease of 68.4%; the computational volume is reduced from 8.1 GFLOPs to 3.5 GFLOPs, a decrease of 56.8%; in terms of detection performance, the detection accuracy is increased from 96.8% to 97.6%, and mAP50:95 is increased from 78.9% to 80.2%. These results show that the improved method in this embodiment not only improves the model performance, but also significantly reduces the complexity and computational cost of the model, making it more suitable for practical application scenarios with limited computing resources.

[0125] Comparative experiment

[0126] Comparative experiment of different models

[0127] To better verify the advantages of the improved algorithm, we conducted a comparative experiment and compared it with the current mainstream object detection algorithms. The experiment used the YOLO object detection algorithm from the release of YOLOv4 to date as the comparison object and evaluated it on the SeaShips dataset after data augmentation.

[0128] According to the experimental results in Table 3, YOLO-LISD is superior to other YOLO series algorithms in terms of accuracy, precision, number of parameters, computational complexity, and model size, and better meets the actual requirements for the deployment of inland river ship detection algorithms on edge terminal devices.

[0129] Table 3 Comparison of different detection algorithms

[0130]

[0131]

[0132] Performance comparison of different pruning rates

[0133] The models after direct lightweight design are pruned at different ratios, and the weights of the remaining channels are optimized to restore the accuracy. The specific results are shown in Table 4. It can be seen from the table that as the pruning rate gradually increases, the number of parameters and the amount of computation of the model gradually decrease, but the model performance does not always decline. For example, when the pruning rate is 33.3%, the mean average precision of the model is improved instead; when the pruning rate reaches 40%, the accuracy and the mean average precision do not change significantly, which indicates that there may be duplicate feature learning in the convolutional kernels of the same layer in the model, resulting in waste of parameters and computing resources. Since the model maintains stable performance while significantly reducing the number of parameters and the amount of computation at a 40% pruning rate, this model is used as the student model for subsequent knowledge distillation operations. Figure 10 Figure showing the number of channels in the first 25 layers of the model before and after pruning based on the GCAMBP score at a 40% pruning rate, from which it can be intuitively seen that there is a large amount of structural redundancy in the YOLOv8n backbone network.

[0134] Table 4 Performance comparison at different pruning rates

[0135]

[0136]

[0137] Knowledge distillation effect

[0138] Taking the pruned model as the student model and YOLOv8m as the teacher model, the SCCKD method is used to perform knowledge distillation on the three output layers of the neck network. As shown in the experimental results of Table 5, after SCCKD, the P, R, mAP50, and mAP50:95 of the model are improved by 0.6%, 0.8%, 0.1%, and 0.7% respectively.

[0139] Table 5 Influence of SCCKD on model performance

[0140]

[0141] Visualization analysis

[0142] To test the detection effect of YOLO-LISD in a real environment, ships passing by on the river surface downstream of the Tongling Yangtze River Bridge are detected. The experimental results are as Figure 10 shown, where the first column and the second column are the detection results of YOLOv8n and YOLO-LISD respectively.

[0143] From the detection results, it can be seen that, firstly, when the ship blends in with the nearshore background, YOLO-LISD can correctly give the position box of the ship. Secondly, when there is occlusion or part of the hull enters the field of view, YOLO-LISD can show better performance and can identify the target more quickly and accurately. Finally, when the lighting conditions change, using YOLOv8n for detection, some ships fail to be successfully detected, resulting in missed detection, or can be detected but misclassified into other categories, while YOLO-LISD can effectively handle the lighting changes and ensure that the target can be successfully identified under different lighting conditions. In contrast, YOLO-LISD is more superior through more effective feature extraction techniques, significantly reducing the missed detection rate and effectively reducing the false detection rate, showing stronger robustness and accuracy.

[0144] In summary, YOLO-LISD is an effective improvement compared to YOLOv8n. It not only realizes the lightweight of the model but also shows significant advantages in improving detection accuracy and robustness, providing reliable technical support and solutions for ship monitoring and management in practical applications.

[0145] The feature sharing mechanism of the present invention is designed to reduce the overall number of parameters of the model. EFSDH first completes channel alignment through an independent 1×1 convolution, enabling different-scale features to be uniformly processed. Secondly, two shared 3×3 detail enhancement convolutions are introduced to make the feature maps of each scale share the same weights, thus maintaining a consistent feature extraction pattern. This design not only reduces the number of parameters but also improves the feature consistency of multi-scale detection, contributing to enhancing the robustness and detection accuracy for ship targets of different sizes and perspectives. The feature sharing mechanism simplifies the training process and improves the stability and efficiency of training. In addition, to adapt to different target scales, a Scale layer is added to dynamically adjust the scale of the feature map, further enhancing the multi-scale perception ability. Compared with the detection head of YOLOv8n, EFSDH reduces the number of parameters to 51% of the original decoupled detection head, making the model easier to apply on systems with limited computing resources.

[0146] Secondly, the Detail Enhancement Convolution (DEConv), as the core module of EFSDH, further improves the feature expression ability. DEConv can capture the detail information in different directions and scales in the feature map. DEConv integrates central difference convolution, horizontal difference convolution, vertical difference convolution, and direction enhancement convolution operations, and captures multi-direction detail information through parallel operation. Due to the large differences in the distribution of features at different scales, if batch normalization (BN) is used in the normalization layer shared in DEConv, normalization bias may be caused by batch dependence and inconsistent distribution, especially in the small batch training or inference stage, where it performs unstably. To solve this problem, DEConv uses group normalization (GN) to replace BN, which not only eliminates the dependence on batch size but also enhances the adaptability to multi-scale features.

[0147] The above specific embodiments are only several alternative embodiments of the present invention. Based on the technical solution of the present invention and the relevant revelations of the above embodiments, those skilled in the art can make various alternative improvements and combinations to the above specific embodiments.

Claims

1. A lightweight inland ship detection algorithm based on improved YOLOv8n, characterized in that: The following steps are involved: A lightweight detection head EFSDH is designed. The detection head completes channel alignment through independent 1×1 convolution, introduces two shared 3×3 detail enhancement convolutions DEConv to achieve that feature maps of different scales share the same weights, and adds a Scale layer to dynamically adjust the scale of feature maps. DEConv integrates center difference convolution, horizontal difference convolution, vertical difference convolution and direction enhancement convolution operations, and uses group normalization GN instead of batch normalization BN. The slim-neck method for lightweight neck network is introduced. GSConv and the GSBottleneck and VoV-GSCSP modules built on GSConv as the core are used to replace the original Conv, Bottleneck and C2f modules in the YOLOv8n neck network respectively, to further optimize the computational efficiency of feature extraction and fusion. A global channel adaptive amplitude pruning algorithm GCAMBP is designed for channel pruning. The square sum of each channel weight is first calculated, and then the square sum of the channel weights is sorted in ascending order. A dynamically normalized GCAMBP score is defined. The channel with the lowest score is selected from the global score for pruning and the corresponding convolution kernel is removed until the preset pruning rate is met. The input channels of the subsequent layers of the network are adjusted synchronously. After pruning, the model performance is restored through fine-tuning. The spatial-channel correlation knowledge distillation method SCCKD is designed for knowledge distillation. The spatial correlation matrix and channel correlation matrix of the student and teacher models are constructed to model the model feature relationship. The loss function includes three parts: the loss of normal training, the inter-channel correlation loss, and the spatial correlation loss. The inter-channel correlation loss and the spatial correlation loss are adjusted by the corresponding weight factors.

2. According to claim 1, the improved YOLOv8n lightweight inland ship detection algorithm is characterized in that: In the step of designing a lightweight detection head EFSDH and replacing the original detection head, the parameter fusion mechanism formula of DEConv is: In the formula, F out represents the output feature, F in represents the input features, K i Represents each convolution kernel, K cvt Represents the sum of all convolution kernels.

3. According to the improved YOLOv8n lightweight inland ship detection algorithm of claim 1, it is characterized in that: In the lightweight convolution GSConv, the time complexity formula of the GSConv calculation is: Where W and H represent the width and height of the input feature map, respectively, K1*K2 is the convolution kernel size, C1 and C2 are the number of channels of the input and output feature maps, respectively. It integrates standard convolution and depthwise separable convolution and introduces shuffle operation to achieve efficient feature interaction. The GSBottleneck module designed by combining GSConv with the residual structure further optimizes the feature extraction process. The VoV-GSCSP module is designed based on GSBottleneck and through a parallel path.

4. According to claim 1, a lightweight inland ship detection algorithm based on improved YOLOv8n is characterized in that: The channel pruning using the global channel adaptive amplitude pruning algorithm GCAMBP specifically includes the following steps: Input the original model and set the pruning rate. The original model contains a convolutional layer, and the weight matrix of the convolutional layer is Represents a three-dimensional feature, C in Indicates the number of input channels, C out represents the number of output channels, k is the size of the convolution kernel; For each convolutional layer in the model, follow the formula: Calculate the importance score S of the j-th output channel of the l-th layer l,j , get the importance score of each channel in each layer; Sort the channel importance scores of layer l in ascending order, and then calculate the dynamically normalized GCAMBP score for the sorted channel k: In the formula, S l Represents the square sum tensor of the channel weights of the lth layer, S l [k] represents the sum of squared channel weights corresponding to index k. Assume that the sum of squared weights is sorted in ascending order according to the given index, that is, when u≤v, S l [u]≤S l [v]; Aggregate the GCAMBP scores of all layers and rank the channel scores globally; Select the channel with the lowest score from the global score for pruning, remove the corresponding convolution kernel, and update the current pruning rate according to the number of pruned channels until it meets the set pruning rate, and simultaneously adjust the input channels of the subsequent layers of the network; After pruning is completed, the model performance is restored through fine-tuning to achieve efficient compression and inference optimization of the model.

5. According to the improved YOLOv8n lightweight inland ship detection algorithm of claim 1, it is characterized in that: In the knowledge distillation using the spatial-channel correlation knowledge distillation method SCCKD, the feature maps of the teacher model and the student model are and in, represents a three-dimensional feature, C and C′ represent the number of channels of the teacher and student models respectively, H and W are the height and width of the feature map space dimension; flatten the feature map F along the channel dimension T and F S , the spatial correlation matrix S is obtained by matrix multiplication operation T and S S : S T =f C (F T )·f C (F T ) T S S =f C (F S )·f C (F S ) T In the formula, f C (F T ) represents the feature map flattened along the channel dimension into a matrix of [H×W,C], S T and S S Describe the relationship between different spatial positions in the teacher model and student model feature maps respectively, and T and S S The two matrices are normalized and the spatial correlation loss L SC This is achieved by minimizing the L2 distance between the spatial correlation matrices of the student and teacher models: Where H×W is the spatial resolution of the feature map, is the square of the L2 norm.

6. According to claim 5, the improved YOLOv8n lightweight inland ship detection algorithm is characterized in that: Flatten the feature map F using the spatial dimension T and F S , the channel correlation matrix C is obtained by matrix multiplication operation T and C S ; C T =f S (F T )·f S (F T ) T C S =f S (F S )·f S (F S ) T In the formula, f S (F T ) represents flattening the feature map along the spatial dimension into a matrix of [C,H×W], C T and C S Describe the collaborative relationship between different channels in the teacher model and student model feature graphs respectively, and T and C S The two matrices are normalized, and the inter-channel correlation loss L CC This is achieved by minimizing the L2 distance between the channel-to-channel correlation matrices of the student and teacher models: Where C is the number of channels, is the square of the L2 norm; Based on the above spatial correlation loss and channel correlation loss, the total loss function of the student model in the SCCKD knowledge distillation process can be expressed as: L SCCKD =L train +αL CC +βL SC Where, L train is the loss of normal training, L CC is the inter-channel correlation loss, L SC is the spatial correlation loss, and α and β are weight factors.

Images