Target detection method for maritime unmanned aerial vehicle image and electronic equipment

Abstract

The invention relates to a target detection method of a maritime unmanned aerial vehicle image and electronic equipment. The method comprises the following steps: acquiring the maritime unmanned aerial vehicle image; a target detection network model is constructed and trained, the target detection network model is obtained based on improvement of a YOLOv5s model, a backbone network of the target detection network model comprises a C3CBAM module added with CBAM on the basis of a C3 module, and a neck network comprises a C3GS module; the C3GS module is used for replacing a Bottleneck sub-module and a convolution in the C3 module with a GSConv sub-module; and inputting the maritime unmanned aerial vehicle image into the trained target detection network model, and outputting a target detection result. Compared with the prior art, the method has the advantages that the accuracy and efficiency of offshore small target detection are improved, the parameter quantity of the model is reduced, and real-time detection is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and in particular to a target detection method and electronic equipment for marine unmanned aerial vehicle images. Background Art

[0002] The ocean covers about 71% of the earth's surface and is closely related to human survival and development. According to statistics from the World Wildlife Fund, the total economic value of the ocean is about 24 trillion US dollars each year. However, while the ocean generates economic value for humans, it also threatens human life and safety. A large number of air and shipwrecks occur around the world every year, but there is currently no perfect rescue system. Traditional search and rescue methods mainly rely on surface ships and helicopters, which not only requires a lot of manpower and material resources, but also has disadvantages such as low efficiency, small search and rescue range, and high cost. In recent years, with the development of drone technology, the manufacturing technology and performance indicators of drones have gradually matured. Due to its advantages of light weight, small structure, low cost, and high performance, drones are widely used in many fields such as agricultural production, wildfire prevention and control, and disaster relief. The combination of drones and traditional rescue systems makes up for the defect of small search and rescue range in traditional rescue systems and greatly improves rescue efficiency.

[0003] The rapid development of artificial intelligence and target detection algorithms can enable drones to play a greater role in marine search and rescue. Marine target detection, as an important branch of the target detection field, is of great significance to maritime navigation, maritime traffic, marine environmental detection and even national defense security. Combining target detection algorithms with the field of drones can enable drones to quickly locate and identify targets, reducing manpower loss and search and rescue time, and greatly improving rescue efficiency.

[0004] In the early days of the development of target detection algorithms, most of the early target detection was based on manually designed feature extractors because the hardware computing power could not support complex feature extraction and convolution operations. These target detection algorithms generally have small parameters and can be embedded in various small mobile platforms, but their biggest disadvantage is that they have poor generalization ability. For different application scenarios, users often need to design different feature extractors. Therefore, with the improvement of hardware computing power and the development of deep learning technology, especially the emergence of GPU and convolutional neural networks, target detection algorithms based on deep learning have been rapidly developed and spread. Unlike traditional feature extractors, target detection algorithms based on deep learning obtain relevant features of the target to be detected through learning, which makes the algorithm have strong generalization ability, gradually replacing traditional algorithms and becoming the mainstream research direction in the field of target detection.

[0005] According to the network architecture, deep learning-based target detection algorithms can be divided into two categories: target detection algorithms based on convolutional neural network architecture and target detection algorithms based on transformer network architecture. Target detection algorithms based on convolutional neural network architecture can be further divided into single-stage target detection algorithms and two-stage target detection algorithms. The first to appear is the two-stage target detection algorithm. The typical feature of this type of detection algorithm is that target detection is divided into two stages. First, the selective search algorithm is used to extract about 2,000 candidate frames from the input image. Then, the feature map of each candidate frame is extracted through the CNN network, and the extracted features are input into the SVM for classification. Classic two-stage target detection algorithms include: R-CNN, Fast R-CNN, Faster R-CNN, mask R-CNN, cascade R-CNN, etc. Although this algorithm has high detection accuracy, the large number of candidate frames generated during the reasoning process leads to a large amount of model calculation and a slow detection speed, which cannot meet application scenarios with high real-time requirements. In order to improve the detection speed, Redmon et al. proposed a single-stage target detection algorithm in 2015 and named it YOLO (You Only Look Once). In the next few years, YOLOv2~YOLOv10 were proposed one after another, and there were countless improved algorithms based on the YOLO series. The biggest advantages of this type of algorithm are fast reasoning speed, short training time, strong real-time performance, and small number of parameters. With the update and iteration of the version, the detection accuracy is constantly improving, and it has gradually become one of the most popular algorithms in today's target detection algorithms. Algorithms based on the transformer network architecture were initially mostly used for natural language processing due to their strong parallel computing capabilities and long sequence processing capabilities. The advent of DETR has gradually enabled people to gradually apply the transformer network architecture to the field of target detection. Among them, the classic algorithms include Deformable-DETR, Sparse-DETR, PnP-DETR, RE-DETR, etc. However, the DETR series network structure generally has the disadvantages of large model parameters, long training time, and low detection accuracy of small targets.

[0006] UAV images in marine environments generally have problems such as small targets, large interference, and complex environments. Existing models often perform poorly in complex marine scenes, are prone to false positives or ignore small targets due to interference, and lack dedicated datasets for small target detection at sea.

[0007] How to achieve fast and high-precision small target detection in marine drone images based on deep learning algorithms has become a technical problem that needs to be solved. Summary of the invention

[0008] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a target detection method and electronic device for marine UAV images, so as to solve the problems of low detection accuracy and large number of parameters of current target detection algorithms for small targets in UAV images.

[0009] The purpose of the present invention can be achieved by the following technical solutions:

[0010] According to one aspect of the present invention, a method for detecting a target in a marine UAV image is provided, the method comprising the following steps:

[0011] Step 1, obtaining marine drone images;

[0012] Step 2, construct and train a target detection network model, wherein the target detection network model is improved based on the YOLOv5s model, wherein the backbone network of the target detection network model includes a C3CBAM module in which CBAM is added on the basis of the C3 module, and the neck network includes a C3GS module; wherein the C3GS module replaces the Bottleneck submodule and convolution in the C3 module with a GSConv submodule;

[0013] Step 3: Input the marine drone image in step 1 into the trained target detection network model and output the target detection result.

[0014] Preferably, the head of the target detection network model uses a decoupled detection head, which decomposes the target detection task into two independent subtasks: classification and positioning.

[0015] Preferably, the backbone network part of the target detection network model uses an improved spatial pyramid pooling structure, namely, a SimSPPFCSPC module, and the SimSPPFCSPC module replaces convolution with a CBR unit.

[0016] More preferably, the CBR unit includes convolution, batch normalization layer and ReLU function.

[0017] More preferably, the SimSPPFCSPC module combines the simplified fast spatial pyramid pooling SimSPPF and the cross-stage local network CSP, and the output result of SimSPPF is connected to the cross-stage local network CSP to form an end-to-end network structure.

[0018] Preferably, the C3CBAM module is used to simultaneously process channel information and spatial information during feature extraction.

[0019] Preferably, the training data of the target detection network model includes images of a public data set in an ocean environment, and real ocean aerial images taken by a drone, wherein the images include a variety of images with different lighting conditions, different shooting distances and different human body postures.

[0020] Preferably, the targets include cruise ships and personnel.

[0021] Preferably, the GSConv submodule first performs conventional convolution and then performs depth convolution, and finally, after slicing the two convolution results, performs a shuffling operation to splice the corresponding channels, thereby allowing the information of the conventional convolution to penetrate into the result of the depth convolution to achieve a lightweight neck network.

[0022] According to another aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the method described above is implemented when the processor executes the program.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] 1) The present invention combines the convolutional attention mechanism module (CBAM) to improve the C3 module in the YOLOv5s backbone network, and proposes a new C3CBAM module to improve the ability to capture spatial information and position information in the image, improve the problem of small object position loss, and can also adaptively learn the features of different regions in the input image to improve the ability to extract target features; in order to solve the problem of increased model parameters, the GSConv convolution module is introduced into the neck network to improve the C3 module, and a lightweight C3GS module is proposed, which optimizes the neck network of YOLOv5s, reduces the number of parameters, and speeds up the target detection efficiency. Therefore, the accuracy and efficiency of small target detection are improved, which can meet the requirements of real-time detection.

[0025] 2) The present invention introduces a decoupled detection head in the YOLOv5s head to replace the single detection head in the YOLOv5s model, which effectively decomposes the target detection task into two independent subtasks of classification and positioning, which helps the model to better learn the category and location information of the target, thereby improving the accuracy and robustness of detecting small targets from marine UAV images.

[0026] 3) The present invention integrates the Simplified Spatial Pyramid Pooling-Fast (SimSPPF) and the CrossStage Partial Network (CSP) modules to construct the SimSPPFCSPC module. Compared with the SimSPPF module, this module can more effectively capture local detail information and enhance the ability to extract edge features of targets of different sizes, thereby improving the accuracy of small target detection.

[0027] 4) The method of the present invention reduces the number of parameters and the requirement for computing resources, making the target detection method more suitable for deployment on mobile devices, meeting real-time requirements, and having higher practical value in maritime search and rescue missions. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the overall network structure in the present invention;

[0029] Figure 2 It is a structural diagram of the SimSPPFCSPC module in the present invention;

[0030] Figure 3 It is a schematic diagram of the structure of the GSBottleneck and C3GS modules in the present invention;

[0031] Figure 4 This is the structural diagram of the improved lightweight Neck;

[0032] Figure 5 This is an example of a self-made data set sample in the present invention;

[0033] Figure 6(a) is a schematic diagram of the detection results of the EfficientDet-D1 algorithm;

[0034] Figure 6(b) is a schematic diagram of the detection results of the Faster R-CNN algorithm;

[0035] Figure 6(c) is a schematic diagram of the detection results of the CenterNet algorithm;

[0036] Figure 6(d) is a schematic diagram of the detection results of the YOLOv4 algorithm;

[0037] Figure 6(e) is a schematic diagram of the detection results of the YOLOv5s algorithm;

[0038] Figure 6(f) is a schematic diagram of the detection results of the YOLOv5m algorithm;

[0039] Figure 6(g) is a schematic diagram of the detection results of the YOLOv8s algorithm;

[0040] FIG6( h) is a schematic diagram of the detection results of the SCD-YOLO algorithm in the present invention;

[0041] Figure 7 It is a structural schematic diagram of the C3CBAM module in the present invention;

[0042] Figure 8 It is a schematic diagram of the structure of the GSConv submodule;

[0043] Fig. 9This is a schematic diagram of the structure of the decoupling detection head introduced in the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0045] This embodiment relates to a target detection method for a drone image at sea. By introducing an attention mechanism and improving the spatial pyramid pooling structure, the present invention aims to improve the model's ability to extract target features and enhance the accuracy and robustness of detection by using a decoupled detection head. In addition, in order to reduce the number of parameters of the model, the present invention also makes lightweight improvements to the neck network of the original model. Combining these improvements, the present invention proposes an algorithm specifically for small target detection in drone images in a marine environment, called the SCD-YOLO (SimSPPFCSPC C3CBAM Decoupled Head-YOLO) algorithm. Testing and evaluation of the algorithm on a self-made marine dataset show that its detection performance is significantly improved, with the average accuracy increased from 87.2% to 90.0%, meeting actual engineering needs, especially in marine search and rescue missions, improving rescue efficiency and accuracy.

[0046] Considering the demand of UAVs for computing resources and real-time detection, we chose the YOLO series as the basis for our research, and chose YOLOv5s as the benchmark model after comprehensively considering many factors such as detection accuracy and the number of model parameters.

[0047] The target detection network model corresponding to the SCD-YOLO algorithm is improved based on the YOLOv5s model, including the following improvements:

[0048] 1) Backbone network improvement: By introducing attention mechanism modules (such as CBAM, SENet or CA modules) to enhance the backbone network of the YOLOv5s model, the model's ability to extract target features in the image is improved. These attention mechanism modules enable the model to pay more attention to important information in the image, thereby improving the detection accuracy of small targets.

[0049] Improved spatial pyramid pooling structure: The spatial pyramid pooling (SPP) structure in the original YOLOv5s model is improved to enhance the model's adaptability to objects of different sizes. The improved SPP structure can better handle the multi-scale characteristics of the input image and improve detection performance.

[0050] 2) Decoupled detection head: The decoupled detection head is introduced into the YOLOv5s model to decompose the object detection task into two independent subtasks: classification and localization. This design helps the model better learn the category and location information of the target and improve the accuracy and robustness of detection.

[0051] 3) Lightweight neck network: In order to reduce the number of model parameters, the neck network of the YOLOv5s model is lightweighted and improved. By introducing the GSConv submodule and the improved C3GS module, the model structure is optimized, the number of parameters is reduced, and the model performance is maintained.

[0052] 4) Fusion strategy: Before the detection head, a fusion strategy is used to fuse infrared light, visible light, and the mask image obtained by binarization to generate a new image. This step uses channel attention and spatial attention mechanisms, as well as the RFN module to optimize the fusion of background and subject images.

[0053] 5) Decoder reconstruction: The fused image is used as the input of the decoder to reconstruct the image. The decoder network consists of multiple decoding blocks, each of which contains two 3×3 convolutional layers and is connected by short connections to achieve fine reconstruction of the image.

[0054] Through the above technical methods, the present invention can effectively improve the detection accuracy of small targets in marine drone images, while reducing the number of model parameters, making the model more suitable for deployment on mobile devices and meeting real-time requirements. These improvements make the SCD-YOLO algorithm more practical in marine search and rescue missions.

[0055] The method comprises the following steps:

[0056] Step 1. Dataset construction: To ensure the effectiveness of algorithm training and testing, a dedicated marine drone image dataset was first constructed. This dataset contains images of ships and personnel taken from different angles and under different lighting conditions, as well as images selected from public datasets such as COCO and SeaDroneSee. These images were annotated by the automatic annotation tool MakeSense to ensure the accuracy of target detection.

[0057] Step 2: Structural design of target detection network model: The target detection network model of the present invention is an SCD-YOLO network, which is improved based on the YOLOv5s model, and the following key improvements are made to it:

[0058] 1) Introducing the attention mechanism: By integrating the CBAM module into the backbone network, a new C3CBAM module is proposed, which enhances the model's ability to capture spatial and position information in the image and effectively reduces the problem of small target position loss.

[0059] C3CBAM (C3 module incorporating attention mechanisms) is a network structure that combines the C3 module and the attention mechanism, which aims to improve the performance of the target detection algorithm. The C3 module is a major building block in the YOLOv5 model. It is a feature extractor consisting of convolutional layers, which include convolution, batch normalization, and activation functions. The "C3" in the C3 module stands for "Cross Stage Partial Connection", which means that the C3 module has partial connections between different stages to promote information flow and feature sharing. The C3CBAM module adds an attention mechanism (CBAM) on the basis of the C3 module. This attention mechanism helps the model pay more attention to important feature channels, thereby improving the performance and robustness of the model. By introducing the attention mechanism, the C3CBAM module can dynamically learn the importance of different channels in the feature map to better capture the key features of the target and further improve the detection accuracy. The BAM module, that is, the convolutional block attention module, is an attention mechanism for the convolution module. Compared with SENet, which only focuses on channel attention, CBAM is an attention mechanism module that combines spatial attention and channel attention. Its structure is as follows Figure 7 As shown in the figure. The work of the CBAM module is divided into three steps. The first step is similar to the SENet module. First, the input image passes through the channel attention module to complete the recalibration of the original features in the channel dimension. Secondly, after obtaining the weighted results, the spatial attention module completes the recalibration in the spatial dimension. Finally, the output result is obtained by weighting. The CBAM module improves the expressiveness and generalization ability of the algorithm by processing channel information and spatial information at the same time. It can also adaptively learn the features of different regions in the input image to enhance the algorithm's attention to important features, thereby improving the performance of the algorithm.

[0060] The introduction of the C3CBAM module allows the model to play the role of attention in the feature extraction part, which not only improves the ability of feature fusion calculation expression, but also helps the model to detect targets more effectively in complex backgrounds. In addition, the design of the C3CBAM module takes into account the complexity of the network structure, the number of parameters and the amount of calculation, aiming to achieve fast and accurate target detection. By combining the CBAM attention mechanism with the C3 module, the C3CBAM module can enhance the model's attention to the characteristics of the target area in the image, while ignoring unimportant location information, thereby accurately identifying target feature information in complex background environments.

[0061] 2) Improved spatial pyramid pooling structure: The improved SimSPPFCSPC module is used to replace the original fast spatial pyramid pooling (Spatial Pyramid Pooling-Fast, SPPF) module, such as Figure 2 This module combines the simplified spatial pyramid pooling-fast (Simplified Spatial Pyramid Pooling-Fast, SimSPPF) and the cross-stage partial network (Cross Stage Partial Network, CSP) to more effectively capture local detail information and edge features, improving the accuracy and efficiency of small target detection.

[0062] Figure 2 In the figure, the original convolution is replaced by a CBR unit, which consists of a convolution, a batch normalization layer, and a ReLU function. The dark green CBR unit has a convolution kernel size of 1x1, which is used to capture local detail information and extract edge features. The light green CBR unit has a convolution kernel size of 3x3, which is used to learn higher-level semantic information. The output of the SimSPPF structure is connected to the CSP module to form an end-to-end network structure. This improved network structure allows the YOLO algorithm to process input images of different sizes, improves detection performance, while maintaining high efficiency and reducing computational costs. Nevertheless, the SPP structure has a high computational complexity and runs slowly. Through these improvements, we aim to improve the performance of the model when handling multi-scale object detection tasks while optimizing computational efficiency.

[0063] 3) Lightweight neck network: In order to reduce model parameters, the neck network of the YOLOv5s model is lightweighted and improved. The neck network of the YOLOv5s model is optimized by introducing the GSConv submodule, and a lightweight C3GS module is proposed to form a lightweight neck network, such as Figure 4 shown.

[0064] Lightweight C3GS modules such as Figure 3 The original Bottleneck submodule and convolution in the C3 module are replaced with the GSConv submodule, thus achieving a lightweight model.

[0065] like Figure 8, the GSConv submodule performs regular convolution first, then deep convolution, and finally performs a Shuffle operation to splice the corresponding channels after slicing the two convolution results, thereby allowing the information of regular convolution to penetrate into the result of deep convolution. The GSConv submodule achieves a better balance between model accuracy and speed than other lightweight improvement methods. In this way, the GSConv submodule is able to reduce computational complexity while maintaining expressiveness, thereby improving the efficiency of the model. This method significantly improves the effect of lightweight by introducing depthwise separable convolution (DSC) and shuffle operations, and enhances the nonlinear expression ability of the model by adding DSC and Shuffle layers. Adding GSConv submodules to other stages of the model is a good way to reduce the number of parameters and the amount of calculation, but the network of the model will be deepened, increasing the resistance of data flow, which can be alleviated by introducing an attention mechanism. With this improvement, GSConv will better handle connected feature maps, thereby reducing redundant information, without compression, and the attention module works better.

[0066] 4) Detection head optimization: This paper introduces a decoupled detection head in the YOLOv5s model to decompose the target detection task into two independent subtasks: classification and positioning. The structure of the decoupled detection head is as follows: Fig. 9 This design allows the model to better learn the category and location information of the target and learn target features more effectively during training.

[0067] In the iteration process of the YOLO series of algorithms, most of the improved algorithms focus on improving the backbone network and multi-scale feature fusion network, or using some plug-and-play modules to enhance the performance of the model. However, from YOLOv3 to YOLOv5, the initial detection head module of the algorithm has been continued, which only uses one detection head to simultaneously predict the category and location information of the object. This single detection design may cause coupling between the category and location information of the object, thereby limiting the performance of the model.

[0068] This paper introduces a decoupled detection head to replace the single detection head in the YOLOv5 algorithm, which can effectively decompose the target detection task into two independent subtasks: classification and localization. Its structure is shown in the figure below. This decoupled design helps the model better learn the category and location information of the object, thereby improving the accuracy and robustness of detection. Specifically, the classification subtask focuses on predicting the category label of the object, while the localization subtask focuses on predicting the location information of the object, such as the coordinates and size of the bounding box. The benefit of this dismissal is that the classification and localization subtasks can be optimized separately without interfering with each other. At the same time, the algorithm can learn the characteristics of the target more efficiently during the training process, thereby improving the stability and efficiency of the training process. In addition, since the classification and localization tasks are optimized separately during the training process, various loss functions can also be used for different task characteristics to further improve the detection ability of the model.

[0069] Step 3: Train the SCD-YOLO network and make the following configurations: In the experiment, the present invention uses NVIDIA GeForceRTX 2080GPU and Intel Core i7-10700 CPU, running Windows 10 operating system. Pytorch 1.10.0 and Python 3.8 are used for model training. The hyperparameter settings of the model include image size of 640×640, momentum of 0.9, batch size of 16, training cycle of 300, and learning rate of 0.005.

[0070] Step 4: Input the real-time collected marine UAV image into the target detection network model trained in step 3, and output the marine small target detection result.

[0071] This embodiment also involves the application of a method for target detection in marine drone images. This embodiment has conducted an in-depth study of multiple public data sets in marine environments, such as: MS2ship, SMD, SeaDroneSee, etc., but most of the image data in the data sets are very different from the actual scenes of maritime rescue. In order to ensure the practicality and authenticity of the algorithm without losing robustness and generalization ability, and at the same time meet the diversity of training data, this embodiment captures some images from real ocean aerial videos taken by drones, and then selects some images from public data sets such as Tiny Person and COCO to form a new data set of about 2,000 images. Sample data set images are as follows: Figure 5 The main targets in the image include ships and people, which include multiple images with different lighting conditions, different shooting distances and different human postures, and meet factors such as small target size and complex environment.

[0072] After the complete data set is prepared, this embodiment performs a statistical analysis on the self-made data set. The number of labels for sea personnel in the data set is about 1,800, and the number of labels for ships is about 2,200. The images in the data set have a high proportion of small targets, which meets the actual needs of sea target detection, and most of the targets are distributed near the center of the vertical axis of the image. It can be seen that this data set can approximately simulate the real maritime rescue environment and meet the experimental requirements.

[0073] In order to further study the actual effect of each improved module on the overall network model, we designed and completed the ablation experiment and conducted a detailed analysis of the experimental results. Table 1 shows the ablation experiment results of each improved method.

[0074] Table 1

[0075]

[0076] The first group of data is the experimental results of the original model (YOLOv5s), and the 2nd to 5th groups are the experimental results after the original model is improved in turn using the C3CBAM module, SimSPPFCSPC module, C3GS module and Decoupled Head. It is not difficult to see that the four improved modules have improved the average detection accuracy (mAP50) of the model, among which the C3CBAM module and C3GS module have also improved the detection frame rate (FPS) and reduced the number of parameters (Params). Therefore, these four improved modules have played a contributing role in the overall improvement of the model performance. Comparing the second and third groups of experiments with the sixth group of experiments, it can be found that the mAP50 values ​​of the sixth group of experiments are higher than those of the second and third groups. This shows that the C3CBAM module and the SimSPPFCSPC module have a superimposed effect on the improvement of the mAP50 performance index of the model, and the C3CBAM module can still reduce the number of parameters (Params) and improve the detection frame rate (FPS). The comparison results between group 7 and group 4 and 5 are similar to those between group 6 and group 2 and 3. The C3GS module can still reduce the number of parameters and improve the detection frame rate. However, the average detection accuracy of the superposition of the C3GS module and the Decoupled Head module is not significantly improved compared with the use of the Decoupled Head module alone. Comparing the experimental results of the original model and SCD-YOLO, mAP50 increased from 87.2% to 90.0%. Although the detection frame rate decreased slightly, it can still meet the real-time requirements. Although the number of parameters of the improved model has increased significantly compared with the original model, it has decreased by 8.2% compared with the experimental results of the eighth group without lightweight, and the number of parameters can still meet the actual needs. In summary, the improved strategy combining the C3CBAM module, SimSPPFCSPC module, C3GS module and Decoupled Head can improve the detection performance of the original model and provide a better solution for maritime rescue and target detection.

[0077] In order to further verify the performance and generalization ability of the improved model, we conducted comparative experiments on the SCD-YOLO algorithm and other mainstream target detection algorithms on a self-made dataset. The comparison results are shown in Table 2.

[0078] Table 2

[0079]

[0080]

[0081] As can be seen from Table 2, the mAP50 value of the SCD-YOLO algorithm is the highest, reaching 90.0%, which is 19.7%, 12.4%, 5.7%, 2.8%, 1.9%, 1.5%, and 1% higher than the EfficientDet-D1 algorithm, Faster R-CNN algorithm, CenterNet algorithm, YOLOv5s algorithm, YOLOv4 algorithm, YOLOv5m algorithm, and YOLOv8s algorithm, respectively. It can be seen from the table that our improved algorithm has significant advantages in detection accuracy and maintains a high level of detection frame rate, which can meet the real-time requirements. By comparing with mainstream algorithms, we have verified the superior performance of the improved SCD-YOLO algorithm in marine target detection tasks, and it still performs well in real-time.

[0082] The experimental results of the average detection accuracy of each category of the improved model and other seven mainstream models on the self-made dataset are shown in Table 3.

[0083] Table 3

[0084]

[0085] As can be seen from Table 3, the SCD-YOLO algorithm verifies the detection of various categories of targets in the maritime target detection task by comparing with the mainstream algorithms. The average detection accuracy of SCD-YOLO algorithm for humans is the highest, reaching 83.4%; the average detection accuracy for ships is 96.5%, which is comparable to YOLOv8s; the comprehensive average detection accuracy of humans and ships is 90%, which is better than other mainstream algorithms, verifying the superior performance of SCD-YOLO algorithm for detecting various categories of targets.

[0086] Comprehensive analysis Figure 6(a) to Figure 6(h)From the detection results of the eight target detection algorithms (sorted by the algorithms in Table 2), it can be found that the detection effect of the EfficientDet-D1 algorithm in Figure 6(a) is the worst, with a large number of missed detections of objects such as people, boats, and lifebuoys, and the confidence is generally low, which also indirectly confirms the authenticity of the lowest mAP50 value of the EfficientDet-D1 algorithm in Table 2. "Boat 0.86" in Figure 6(a) indicates that the object is detected as a boat, and the corresponding confidence is 0.86. The detection results of the CenterNet algorithm in Figure 6(c) and the YOLOv4 algorithm in Figure 6(d) also have missed detections. Since the backbone network in the Faster R-CNN algorithm, whether it is VGGNet or ResNet, the feature map extracted is only a single layer, and the resolution is usually small, it performs poorly in the detection of small objects and multi-scale objects. This leads to a large number of false detections and repeated detections in the detection results of the Faster R-CNN algorithm in Figure 6(b). The detection results of the YOLOv5s algorithm in Figure 6(e) and the YOLOv5m algorithm in Figure 6(f) are roughly the same, and the confidence, false detection rate and missed detection rate are better than the first four algorithms, which also shows the rationality of selecting the YOLOv5 algorithm as the baseline model in this paper. A detailed comparison of the detection results of the two algorithms shows that the confidence in the detection results of the YOLOv5m algorithm is generally higher than that of the YOLOv5s algorithm. The main reason is that the YOLOv5m algorithm has more residual structures and convolution structures, which makes the network's feature extraction and feature fusion capabilities stronger, thereby obtaining a higher confidence, but it will lead to an increase in the number of model parameters and a decrease in the detection frame rate. In Table 2, we can see that the detection accuracy of the YOLOv5m algorithm is higher than that of the YOLOv5s algorithm, but the detection frame rate is lower than that of the YOLOv5s algorithm, and the number of parameters is also much higher than that of the YOLOv5s algorithm. The detection results of the YOLOv8s algorithm are relatively good, and there is no false detection or missed detection, but in terms of small target detection, the confidence is low and cannot meet the actual needs. Comparing Figure 6(g) and Figure 6(h), the confidence of the small target in the detection result of the SCD-YOLO algorithm in Figure 6(h) is significantly better than that of the YOLOv8s algorithm in Figure 6(g). From Table 3, we can also get the same conclusion. The mAP50 value of the human category in the detection result of the SCD-YOLO algorithm is 83.4, which is better than 82.1 of the YOLOv8s algorithm. By analyzing the results of the eight detection algorithms as a whole, we can find that the first six algorithms have false detections, missed detections and repeated detections in the left ship area, especially the first four algorithms. Although the YOLOv8s algorithm has no false detections, missed detections and repeated detections in this area, its confidence is low and cannot meet the actual needs, which shows that the existing target detection algorithms are insufficient in small target detection.The overall confidence performance of the SCD-YOLO algorithm proposed in this paper is excellent, especially in small target detection, which is much higher than other algorithms. It can meet the requirements of real-time and high-precision target detection in images taken by drones in maritime rescue.

[0087] This paper first collected images with the ocean as the background from public datasets such as MS COCO, Seadromes, and drone video sequences, and produced training data that meets the requirements of this experiment. The images in the dataset cover objects of different angles, lighting conditions, and target sizes, which can meet the diversity requirements of training data. Based on the self-made dataset, this paper takes YOLOv5s as the benchmark model, improves its backbone network, feature fusion network, and detection module respectively, and proposes the SCD-YOLO detection algorithm. The improvement strategies of this algorithm are to use CBAM to improve the C3 module in the backbone network, use the improved SimSPPFCSPC to replace SPPF, improve the lightweight Neck network based on GSConv, and use the decoupled detection head to replace the single detection head in the original model, which can greatly improve the accuracy of marine target detection tasks. The average detection accuracy is increased from 87.2% to 90.0%, and it can meet the real-time requirements.

[0088] Through comparative experiments, the present invention finds that the introduced C3CBAM module, SimSPPFCSPC module, C3GS module and decoupling detection head all improve the mAP50 value of the model. In particular, the C3CBAM module and C3GS module reduce the number of parameters while also improving the detection frame rate. The test results on the self-made ocean dataset show that the detection performance of the improved SCD-YOLO algorithm is significantly improved, and the average accuracy is increased from 87.2% to 90.0%.

[0089] The present invention analyzes the detection results of different target detection algorithms and finds that the SCD-YOLO network has significant advantages in small target detection. Compared with other algorithms, the SCD-YOLO network has higher detection accuracy for small targets, which is particularly important in maritime rescue missions. In addition, while maintaining high detection accuracy, the SCD-YOLO network also has a faster detection speed, meeting the real-time requirements.

[0090] The SCD-YOLO network proposed in this paper effectively improves the detection accuracy of small targets in marine drone images by improving the YOLOv5s model. By constructing a special dataset, introducing an attention mechanism, improving the network structure, and optimizing the detection head, the SCD-YOLO network has achieved excellent performance on a self-made marine dataset, providing strong technical support for marine search and rescue missions. Future research will further improve detection efficiency and accuracy to adapt to a wider range of application scenarios.

[0091] The electronic device of the present invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0092] Multiple components in the device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0093] The processing unit performs the various methods and processes described above. For example, in some embodiments, the method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of the method described above can be performed. Alternatively, in other embodiments, the CPU can be configured to perform the method in any other appropriate manner (e.g., by means of firmware).

[0094] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0095] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0096] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0097] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A target detection method for marine drone images, characterized in that: The method comprises the following steps: Step 1, obtaining marine drone images; Step 2, construct and train a target detection network model, wherein the target detection network model is improved based on the YOLOv5s model, wherein the backbone network of the target detection network model includes a C3CBAM module in which CBAM is added on the basis of the C3 module, and the neck network includes a C3GS module; wherein the C3GS module replaces the Bottleneck submodule and convolution in the C3 module with a GSConv submodule; Step 3: Input the marine drone image in step 1 into the trained target detection network model and output the target detection result.

2. The target detection method for marine drone images according to claim 1 is characterized in that: The head of the object detection network model uses a decoupled detection head, which decomposes the object detection task into two independent subtasks: classification and positioning.

3. The target detection method for marine drone images according to claim 1 is characterized in that: The backbone network part of the target detection network model uses an improved spatial pyramid pooling structure, namely, the SimSPPFCSPC module, and the SimSPPFCSPC module replaces convolution with a CBR unit.

4. The target detection method for marine drone images according to claim 3 is characterized in that: The CBR unit includes convolution, batch normalization layer and ReLU function.

5. The target detection method for marine drone images according to claim 3 is characterized in that: The SimSPPFCSPC module combines the simplified fast spatial pyramid pooling SimSPPF and the cross-stage local network CSP, and the output result of SimSPPF is connected to the cross-stage local network CSP to form an end-to-end network structure.

6. The target detection method for marine drone images according to claim 1 is characterized in that: The C3CBAM module is used to process channel information and spatial information simultaneously during feature extraction.

7. The target detection method for marine drone images according to claim 1 is characterized in that: The training data of the target detection network model includes images of a public data set in an ocean environment, and real ocean aerial images taken by a drone, wherein the images include a variety of images with different lighting conditions, different shooting distances, and different human body postures.

8. The target detection method for marine drone images according to claim 1 is characterized in that: The targets include cruise ships and people.

9. The target detection method for marine drone images according to claim 1, characterized in that: The GSConv submodule first performs conventional convolution and then depth convolution, and finally, after slicing the two convolution results, performs a shuffle operation to splice the corresponding channels, thereby allowing the information of the conventional convolution to penetrate into the result of the depth convolution to achieve a lightweight neck network.

10. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 9 is implemented.

Images