Aerial photography marine target detection method based on improved YOLOv8n

Abstract

The invention discloses an aerial photography marine target detection method based on improved YOLOv8n. The method comprises the following steps: acquiring a SeaDrones See V2 marine aerial image data set, preprocessing the SeaDrones See V2 marine aerial image data set, and randomly dividing the SeaDrones See V2 marine aerial image data set into a training set, a verification set and a test set in proportion; a progressive feature pyramid network is introduced on the basis of a YOLOv8n network structure, and a four-scale detection head structure is constructed to enhance the multi-scale feature fusion capability and the semantic expression capability of a small-scale target, so that an optimized model is obtained, and training is completed; the method comprises the following steps: training an unmanned aerial vehicle, converting the weight of a trained model into an ONNX format, deploying the model in an embedded reasoning environment, preprocessing a maritime image collected by the unmanned aerial vehicle, inputting the preprocessed maritime image into the model, outputting a target category, position information and confidence, and transmitting a detection result to an upper computer in a remote communication mode for visual display. The method improves the precision and stability of sea target detection under complex sea conditions, and is suitable for application scenes of maritime search and rescue, cruise monitoring and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and unmanned aerial vehicle intelligent perception technology, in particular to a kind of aerial sea target detection method based on deep learning, especially a kind of aerial sea small scale target detection method based on improved YOLOv8n Network model, suitable for intelligent perception and identification of sea target under complex sea conditions. BACKGROUND

[0002] However, the sea aerial scene has the characteristics of complex background, severe light change, strong sea surface reflection and large target size difference, and the target is mostly in the form of small scale, such as fallen personnel, small boats and buoys, which brings great challenges to the target detection algorithm. The traditional target detection method based on artificial features has poor robustness under complex sea conditions, and it is difficult to meet the actual application requirements.

[0003] In recent years, the target detection method based on deep learning has made significant progress, among which the single-stage target detection algorithm represented by YOLO series has achieved a good balance between detection accuracy and inference speed, and is suitable for real-time application scenarios. However, the existing YOLO network still has problems such as insufficient perception ability for small scale targets, insufficient multi-scale feature fusion and inconsistent cross-level semantics when facing sea aerial images, which is prone to miss detection and false detection.

[0004] Therefore, there is an urgent need for an aerial sea target detection method that can enhance the multi-scale feature fusion capability, improve the detection accuracy of small scale sea targets, and also consider the real-time performance, in order to meet the actual application requirements under complex sea conditions. SUMMARY

[0005] The purpose of the present application is to provide an aerial sea target detection method based on improved YOLOv8n, which improves the network structure, introduces a progressive feature pyramid network and combines a self-adaptive spatial feature fusion mechanism to enhance the semantic consistency and spatial expression ability of multi-scale features, and at the same time, constructs a four-scale detection head to improve the detection performance of small scale sea targets, so as to realize high-precision and high-stability aerial sea target detection under complex sea conditions.

[0006] The technical solution of the present application to achieve the above purpose includes the following steps:

[0007] Step 1: Obtain SeaDronesSee V2 public sea aerial image dataset, clean the data, unify the label format and size normalization, and construct sea target detection dataset;

[0008] Step 2: Preprocess the marine target detection dataset and divide it into training, validation, and test sets according to a preset ratio to obtain processed training data;

[0009] Step 3: Improve the network structure of the YOLOv8n model to obtain an optimized YOLOv8n target detection model, which specifically includes:

[0010] Step 3.1: Build a backbone network to perform multi-layer feature extraction on the input aerial marine images to obtain feature maps of different scales;

[0011] Step 3.2: Introduce an AFPN in the neck network of the YOLOv8n model to enhance the cross-scale feature interaction capability through multi-scale feature step-by-step fusion and semantic alignment;

[0012] Step 3.3: Build a four-scale detection head structure to input the multi-scale feature maps output by the AFPN into the corresponding detection head to improve the detection capability of small-scale marine targets;

[0013] Step 3.4: Initialize the parameters of the optimized YOLOv8n model, input the processed training set into the optimized YOLOv8n model for training, and use the validation set to evaluate and adjust the model parameters to obtain a trained optimized YOLOv8n model;

[0014] Step 4: Convert the PT format weight file of the trained optimized YOLOv8n model into an ONNX format weight file compatible with ONNXRuntime inference to obtain an ONNX target detection model;

[0015] Step 4.1: Configure image acquisition dependent libraries, model inference environments, and ONNX Runtime running environments on the operation device performing aerial marine target detection, and deploy the ONNX target detection model to the operation device;

[0016] Step 5: Preprocess the aerial marine images collected by the UAV camera in real-time to meet the input requirements of the ONNX model, input the preprocessed aerial marine images into the ONNX target detection model, output detection boxes containing target categories, position information, and confidence, and perform confidence filtering on the detection boxes;

[0017] Step 6: Superimpose the filtered detection boxes on the original aerial marine images and transmit the detection results containing target categories and position information to the host computer for visual display to obtain the aerial marine target detection results.

[0018] Further, the specific process of step 1 is as follows:

[0019] A public SeaDronesSee V2 maritime aerial image dataset is obtained, and the aerial images in the dataset are cleaned, including removing images with blur, severe occlusion, strong reflection, abnormal exposure, and unidentifiable target regions. The retained effective aerial images are uniformly labeled and normalized in size to construct a standardized dataset for maritime target detection.

[0020] Further, the specific process of step 2 is as follows:

[0021] The aerial maritime target detection dataset is preprocessed, including image size normalization, pixel value standardization, and data format conversion. The original training set in the dataset is randomly divided according to a predetermined proportion using a Python program while keeping the original validation set unchanged, resulting in a training set and a test set. The training set is used for model parameter learning, the validation set is used for performance evaluation during model training, and the test set is used for final verification of model detection performance, thereby obtaining training data for maritime target detection model training and evaluation.

[0022] Further, the number of images in the training set, validation set, and test set obtained in step 2 is in the ratio of 7:2:2.

[0023] Further, the preprocessing operation in step 2 also includes data augmentation processing of the aerial maritime images, which includes one or more of random cropping, horizontal flipping, rotation, and scaling.

[0024] Further, the specific process of step 3.2 is as follows:

[0025] An AFPN (Asymmetric Feature Pyramid Network) is introduced into the neck network of the YOLOv8n model to align the spatial sizes of different scale feature maps output by the backbone network. Through step-by-step upsampling and downsampling, the adjacent scale features are fused, allowing the semantic information in high-level features to gradually pass to low-level features, thereby enhancing the semantic consistency between multi-scale features.

[0026] Further, the progressive feature pyramid network AFPN in step 3.2 introduces an adaptive spatial feature fusion mechanism ASFF, which performs pixel-level adaptive weighting on the fused multi-scale features at the same spatial position, dynamically allocates fusion weights for different scale features, realizes fine fusion and stable interaction of cross-scale features, retains the spatial detail information of low-level features, introduces the semantic information of high-level features, thereby improving the feature expression ability of small-scale maritime targets under complex sea conditions, and effectively alleviating the feature interference between overlapping targets.

[0027] Further, in step 3.3, a four-scale detection head structure is constructed at the output end of the progressive feature pyramid network AFPN, and multi-scale fused features of different spatial resolutions are input into corresponding detection heads for target prediction; each detection head adopts a decoupled prediction structure and is used for target class prediction and target bounding box regression, respectively, a high-resolution feature detection head is introduced to enhance the perception ability of small-scale maritime targets, thereby forming a four-layer target detection system covering multi-scale targets, and improving the precision and stability of aerial maritime target detection under complex sea conditions.

[0028] Further, the specific process of step 4 is as follows: after completing the optimization YOLOv8n model training, the trained model weight is converted from the original model format supported by the deep learning training framework to the general neural network exchange format ONNX; and the converted ONNX model is deployed to the target operation device, so that the target operation device can complete model loading and inference operation without relying on the original training framework, thereby meeting the real-time inference requirements of aerial maritime target detection.

[0029] Further, the specific process of step 5 is as follows: the aerial maritime image collected by the unmanned aerial vehicle camera in real time is preprocessed to meet the input requirements of the ONNX target detection model; the preprocessed aerial maritime image is input into the ONNX target detection model for inference operation, and the detection result containing target class, position information and confidence is output; and the detection result is subjected to confidence threshold screening to retain the effective detection frame meeting the preset condition, thereby obtaining the aerial maritime target detection result for subsequent display and transmission.

[0030] Further, the specific process of step 6 is as follows: the effective detection frame screened in step 5 is superimposed and displayed in the original aerial maritime image, and the aerial maritime target detection result containing target class and position information is transmitted to the upper computer through the Web-based video stream server for visual display, thereby realizing remote display and monitoring of the aerial maritime target detection result.

[0031] The present application has the following beneficial technical effects:

[0032] The present application realizes the step-by-step fusion and semantic alignment of multi-scale features by improving the YOLOv8n network structure, introducing a progressive feature pyramid network in the neck network and combining an adaptive spatial feature fusion mechanism, and enhances the stable interaction ability between cross-scale features. At the same time, by constructing a four-scale detection head structure, the perception and recognition ability of the model for small-scale maritime targets is improved, thereby effectively reducing the missed detection and false detection phenomenon in complex sea conditions and multi-target scenes. In addition, the present application converts and deploys the trained target detection model in an embedded inference environment, and combines a remote visual display method to realize efficient inference and real-time display of aerial maritime target detection results, has good engineering applicability and practical application value, and is suitable for application scenes such as maritime search and rescue and cruise monitoring. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a flowchart of the present application;

[0034] Figure 2 is a structure diagram of the optimized YOLOv8n model of the present application;

[0035] Figure 3 is a visual aerial maritime target detection result of the present application. DETAILED DESCRIPTION

[0036] The present application will be further described in detail below in combination with specific embodiments. It should be understood that the embodiments are only used to illustrate the technical solutions of the present application, and not to limit the present application. Those skilled in the art can make various forms of deformation or replacement to the present application without departing from the technical idea and protection scope of the claims, and all should fall within the protection scope of the present application.

[0037] Reference Figure 1 As shown in the figure, an aerial maritime target detection method based on improved YOLOv8n includes the following steps:

[0038] Step 1: Construct an aerial maritime target detection dataset, the specific process is as follows:

[0039] Step 1.1: Obtain the public SeaDronesSee V2 maritime aerial image dataset, which is a unmanned aerial vehicle aerial image dataset for maritime search and rescue scenarios, containing aerial image data under various complex sea conditions, and the targets in the image include maritime target categories such as fallen personnel, ships, motorboats, buoys and life jackets;

[0040] Step 1.2: Screen and clean the aerial images in the SeaDronesSee V2 dataset, remove images with severe blur, strong reflection, overexposure, unidentifiable target area or severe obstruction, and retain valid aerial images with clear target features and complete annotations; unify the annotation format of the retained images, and normalize the image size, thereby constructing a standardized dataset for aerial maritime target detection.

[0041] Step 2: Preprocess the aerial maritime target detection dataset and divide it into training, validation, and test sets, the specific process is as follows:

[0042] Step 2.1: Preprocess the images in the aerial maritime target detection dataset, including size normalization, pixel value standardization, and data YOLO format conversion of aerial maritime images to meet the input requirements of subsequent target detection model training and inference;

[0043] Step 2.2: Randomly divide the original training set in the dataset according to the preset ratio of 7:2:2 using a Python program while keeping the original validation set unchanged, obtaining the training set and test set; the training set is used for model parameter learning, the validation set is used for performance evaluation during model training, and the test set is used for final verification of model detection performance;

[0044] Step 2.3: Further, perform data augmentation on the aerial maritime images in the training set, including one or more of random cropping, horizontal flipping, rotation, and scaling to enhance the model's adaptability to target size changes and view angle changes.

[0045] Step 3: Improve the network structure of the YOLOv8n model to obtain an optimized YOLOv8n target detection model as shown in Figure 2 , the specific process is as follows:

[0046] Step 3.1: Construct a backbone network for multi-layer feature extraction of input aerial maritime images, extract feature representations of different spatial resolutions through convolution operations and downsampling operations, and obtain multi-scale feature maps containing rich spatial detail information and high-level semantic information, providing basic feature representations for subsequent feature fusion and target detection.

[0047] Step 3.2: Introduce the progressive feature pyramid network AFPN in the neck network of the YOLOv8n model to perform spatial size alignment processing on different scale feature maps from the backbone network, and fuse adjacent scale features through step-by-step upsampling and downsampling to realize information transmission and semantic alignment of cross-scale features; in the feature fusion process, introduce the adaptive spatial feature fusion mechanism ASFF to perform pixel-level adaptive weighting on multi-scale features at the same spatial position, thereby realizing fine fusion and stable interaction of multi-scale features.

[0048] In the present embodiment, the fusion process of multi-scale features can be represented in the following manner: for four level features mapped to the same scale, the fusion features at the spatial position may be represented as:

[0049]

[0050] wherein, , , and represent the feature vectors from different levels and mapped to the layer after scale alignment; , , and represent the fusion weights of the corresponding scale features at the spatial position , which are adaptively learned by the network during training and satisfy the sum of each fusion weight being 1.

[0051] In the above manner, while preserving the local spatial details in high-resolution features, global semantic information in low-resolution features is introduced, thereby enhancing the feature expression ability of small-scale maritime targets and improving the consistency of cross-scale features.

[0052] Step 3.3: Build a four-scale detection head structure at the output end of the AFPN, and input multi-scale fusion features of different spatial resolutions into the corresponding detection head for target prediction; each detection head adopts a decoupled prediction structure for target class prediction and target bounding box regression, and a high-resolution feature detection head is introduced to enhance the perception and detection ability of small-scale maritime targets, thereby forming a detection system covering multi-scale targets and improving the overall performance of aerial maritime target detection in complex sea conditions.

[0053] Step 3.4: Initialize the parameters of the optimized YOLOv8n model, input the processed training set into the optimized YOLOv8n model for training, and evaluate and adjust the model parameters using the validation set to obtain the trained optimized YOLOv8n target detection model.

[0054] Step 4: Convert the weight file of the trained and optimized YOLOv8n model from the original model format supported by the deep learning training framework to the general neural network exchange format ONNX, and deploy the converted ONNX model to the target operation device;

[0055] Step 4.1: Configure the image acquisition dependent library, model inference environment and ONNX Runtime running environment on the operation device performing aerial marine target detection, so that the target operation device can complete model loading and inference operation;

[0056] Step 5: Preprocess the aerial marine images collected by the unmanned aerial vehicle camera in real time to meet the input requirements of the ONNX model, input the preprocessed aerial marine images into the ONNX target detection model for inference, output the detection results containing target categories, position information and confidence, and perform confidence screening on the detection results;

[0057] Step 6: Superimpose the screened detection results on the original aerial marine image, and transmit the aerial marine target detection results to the upper computer for visual display through the Web-based video stream server, thereby obtaining the aerial marine target detection results as shown in Figure 3

[0058] Preferably, the upper computer is used to remotely control the unmanned aerial vehicle platform, so that the RGB camera carried by the unmanned aerial vehicle can collect real-time aerial video data of the marine environment, and transmit the collected aerial image data to the target operation device for processing. Preferably, the operation device performing aerial marine target detection is a Raspberry Pi mainboard, which is used to complete the loading and inference operation of the target detection model.​

Claims

1. A method for aerial maritime target detection based on an improved YOLOv8n, characterized in that, Includes the following steps: Step 1: Obtain the publicly available marine aerial image dataset from SeaDronesSee V2, clean the data, standardize the annotation format, and normalize the size to construct a marine target detection dataset; Step 2: Preprocess the marine target detection dataset and divide it into training set, validation set and test set according to a preset ratio to obtain the processed training data; Step 3: Improve the network structure of the YOLOv8n model to obtain an optimized YOLOv8n object detection model, specifically including: Step 3.1: Construct a backbone network to perform multi-layer feature extraction on the input aerial marine images and obtain feature maps at different scales; Step 3.2: Introduce the Progressive Feature Pyramid Network (AFPN) into the neck network of the YOLOv8n model to enhance cross-scale feature interaction capabilities through the hierarchical fusion and semantic alignment of multi-scale features. Step 3.3: Construct a four-scale detection head structure and input the multi-scale feature maps output by AFPN into the corresponding detection heads to improve the detection capability of small-scale maritime targets; Step 3.4: Initialize the optimized YOLOv8n model parameters, input the processed training set into the optimized YOLOv8n model for training, and use the validation set to evaluate and adjust the model parameters to obtain the trained optimized YOLOv8n model; Step 4: Convert the PT format weight file of the trained and optimized YOLOv8n model into an ONNX format weight file compatible with ONNXRuntime inference to obtain the ONNX object detection model; Step 4.1: Configure the image acquisition dependency library, model inference environment, and ONNX Runtime runtime environment on the computing device for performing aerial marine target detection, and deploy the ONNX target detection model to the computing device; Step 5: Preprocess the aerial sea images captured in real time by the UAV camera to meet the input requirements of the ONNX model. Input the preprocessed aerial sea images into the ONNX target detection model and output detection boxes containing target category, location information and confidence level. Then, filter the detection boxes by confidence level. Step 6: Overlay the filtered detection boxes onto the original aerial marine image, and transmit the detection results, which include target category and location information, to the host computer for visualization via remote communication to obtain the aerial marine target detection results.

2. The aerial maritime target detection method based on improved YOLOv8n according to claim 1, characterized in that, The specific process of step 1 is as follows: obtain the publicly available SeaDronesSee V2 marine aerial image dataset; filter the image data in the dataset, remove aerial images that are blurry, severely occluded, strongly reflective, abnormally exposed, or have unidentifiable target areas, and retain clear and valid aerial images with complete target features; perform annotation format standardization and category verification on the retained aerial images, thereby constructing a standardized image dataset for aerial marine target detection.

3. The aerial maritime target detection method based on improved YOLOv8n according to claim 1, characterized in that, The specific process of step 2 is as follows: preprocessing the aerial marine target detection dataset, including image size normalization, pixel value standardization, and data format conversion; while keeping the original validation set unchanged, using a Python program to randomly divide the original training set in the dataset according to a preset ratio to obtain a training set and a test set; wherein, the training set is used for model parameter learning, the validation set is used for performance evaluation during model training, and the test set is used for final verification of model detection performance, thereby obtaining training data for training and evaluation of the aerial marine target detection model.

4. The aerial maritime target detection method based on improved YOLOv8n according to claim 3, characterized in that, The ratio of the number of images in the training set, validation set, and test set obtained in step 2 is a preset ratio.

5. The aerial maritime target detection method based on improved YOLOv8n according to claim 3, characterized in that, The preprocessing operation in step 2 also includes data augmentation processing of the aerial marine images, which includes one or more of random cropping, horizontal flipping, rotation, and scaling.

6. The aerial maritime target detection method based on improved YOLOv8n according to claim 1, characterized in that, The specific process of step 3.2 is as follows: A progressive feature pyramid network (AFPN) is introduced into the neck network of the YOLOv8n model to perform spatial size alignment on feature maps of different scales from the backbone network, and feature fusion is performed between adjacent scales through progressive upsampling and downsampling. In the feature fusion process, AFPN dynamically adjusts the multi-scale features by introducing an adaptive spatial weighting method to achieve semantic alignment and stable interaction of cross-level features, thereby forming a multi-scale fused feature representation that takes into account both spatial details and high-level semantic information.

7. The aerial maritime target detection method based on improved YOLOv8n according to claim 6, characterized in that, The adaptive spatial weighting method is the adaptive spatial feature fusion mechanism ASFF. After each feature fusion stage of AFPN, ASFF performs pixel-level weighting on features of different scales at the same spatial location. By dynamically assigning fusion weights to features of each scale, it achieves fine-grained fusion of cross-scale features, thereby enhancing the expressive power of key spatial features and alleviating feature conflicts between overlapping targets.

8. The aerial maritime target detection method based on improved YOLOv8n according to claim 1, characterized in that, The specific process of step 3.3 is as follows: a four-scale detection head structure is constructed at the output end of the progressive feature pyramid network AFPN, and multi-scale fused features with different spatial resolutions are input into the corresponding detection heads for target prediction. Each detection head adopts a decoupled prediction structure, which is used for target category prediction and bounding box regression respectively. By introducing a high-resolution feature detection head, the perception capability of small-scale maritime targets is enhanced, thus forming a four-layer target detection system covering multi-scale targets, improving the detection accuracy and stability of aerial maritime targets under complex sea conditions.

9. The aerial maritime target detection method based on improved YOLOv8n according to claim 1, characterized in that, The specific process of step 4 is as follows: after completing the training of the optimized YOLOv8n model, the trained model weights are converted from the original model format supported by the deep learning training framework to the general neural network exchange format ONNX. The converted ONNX model is then deployed to the target computing device, enabling the target computing device to complete model loading and inference operations without relying on the original training framework.

10. The aerial maritime target detection method based on improved YOLOv8n according to claim 1, characterized in that, The remote communication in step 6 is achieved through a web-based video streaming server to transmit the aerial marine target detection results to a host computer for visualization.

Images