YOLOv8-based unmanned aerial vehicle aerial target detection method in low-illumination scene

Abstract

The invention discloses a YOLOv8-based unmanned aerial vehicle aerial target detection method in a low-illumination scene, and belongs to the technical field of target detection. The method comprises the following steps: firstly, acquiring aerial images of an unmanned aerial vehicle, forming a data set by a plurality of images, and normalizing the image size; then, improving a YOLOv8 network, replacing the last C2f module of a YOLOv8 backbone network with a feature enhancement module, embedding a shallow feature fusion network in a neck network, and embedding an adaptive feature fusion module in a head network to obtain a target detection model; and finally, training the target detection model by using the data set, and applying the trained target detection module to the detection of the aerial photography target of the unmanned aerial vehicle in the low-illumination scene. The feature enhancement module can better capture useful information in the feature map and suppress noise interference in a low-illumination scene; and the shallow feature fusion network fuses the shallow features extracted by the backbone network, more detailed image information is captured, and the small target detection precision of the model in a low-illumination environment is significantly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a method for detecting targets in drone aerial photography in low-illumination scenes based on YOLOv8. Background Art

[0002] With the rapid development of drone technology, drones are widely used in traffic patrol, environmental monitoring, maritime search and rescue and other fields. When performing tasks, drones often need to deal with low-light environments such as night, haze or dawn and dusk. Drone aerial images in low-light environments usually have problems such as insufficient brightness, high noise levels, and blurred details, which seriously affect the accuracy and efficiency of target detection.

[0003] Although deep learning-based target detection algorithms have been widely used in target detection of drone aerial images, there are still some problems. For example, some algorithms improve image quality through image enhancement technology, but it is still difficult to significantly improve detection accuracy and it is difficult to meet real-time requirements. Although some target detection algorithms, such as YOLO and R-CNN series, have good detection performance under good lighting conditions, their detection accuracy and speed are significantly reduced in low-light environments. Therefore, drone aerial target detection in low-light environments places higher requirements on detection algorithms.

[0004] In addition, with the increasing demand for real-time detection in drone systems, the algorithm is not only required to have high precision, but also high efficiency and high robustness to cope with complex and changing application scenarios. The YOLOv8 network has become the preferred algorithm for drone target detection because it can significantly improve the detection speed while maintaining high detection accuracy. In order to make the YOLOv8 network still have good detection performance in low-light environments, the present invention improves the YOLOv8 network and proposes a drone aerial target detection method in low-light scenarios based on YOLOv8. Summary of the invention

[0005] In view of the shortcomings of the prior art, the technical problem that the present invention intends to solve is to provide a method for detecting drone aerial photography targets in low-illumination scenes based on YOLOv8.

[0006] The present invention solves the technical problem by adopting the following technical solution:

[0007] A method for detecting drone aerial photography targets in low-light scenes based on YOLOv8, characterized in that the method comprises the following steps:

[0008] Step 1: Obtain drone aerial images, several images form a data set, and normalize the image size;

[0009] Step 2: Improve the YOLOv8 network by replacing the last C2f module of the YOLOv8 backbone network with a feature enhancement module, embed a shallow feature fusion network in the neck network, and embed an adaptive feature fusion module in the head network to obtain the target detection model.

[0010] The shallow feature fusion network includes two feature fusion branches, the first feature fusion branch is located at the end of the bottom-up branch of the original YOLOv8 neck network, including upsampling and splicing operations connected from bottom to top, the output feature map of the last C2f module of the bottom-up branch of the original YOLOv8 neck network is spliced ​​with the output feature map of the first C2f module of the backbone network after the upsampling operation, so as to obtain the output feature map of the first feature fusion branch; the second feature fusion branch is located at the beginning of the top-down branch of the original YOLOv8 neck network, including a C2f module, a CBS module, a splicing operation and a C2f module connected in sequence from top to bottom, the output feature map of the first feature fusion branch passes through a C2f module and a CBS module in sequence, and then is spliced ​​with the output feature map of the last C2f module of the bottom-up branch of the original YOLOv8 neck network, and then passes through a C2f module to obtain the output feature map of the second feature fusion branch;

[0011] The adaptive feature fusion module performs weighted fusion on the output feature maps x1 and x2 of the two C2f modules of the second feature fusion branch of the shallow feature fusion network and the output feature maps x3 and x4 of the two C2f modules of the top-down branch of the original YOLOv8 neck network to obtain four fused feature maps y1 to y4. The four fused feature maps are expressed as:

[0012]

[0013] In the formula, α i , β i , ε i and η i They are all weight matrices, i=1,2,3,4;

[0014] Step 3: Use the data set to train the target detection model, and use the trained target detection module to detect drone aerial photography targets in low-light scenes.

[0015] Furthermore, the representations of the weight matrices are similar, α i It is expressed as:

[0016]

[0017] Where λ is a learnable parameter.

[0018] Furthermore, the feature enhancement module includes a CBS module, a FEB block and a Bottleneck block; the input feature map f0 passes through a CBS module to obtain a feature map f1; the feature map f1 is split into two identical feature maps f2 and f2 ’ , the feature map f2 passes through multiple parallel feature enhancement branches composed of FEB blocks and Bottleneck blocks connected in sequence, and then is combined with the feature map f2 ’ The feature map f3 is concatenated and passed through a CBS module to obtain the output feature map of the feature enhancement module. Among them, the FEB block uses a multi-head attention mechanism to extract features.

[0019] Furthermore, in the FEB block, the input feature map is multiplied with three updateable weight matrices respectively to obtain the query vector, key vector and value vector; the key vector is sequentially subjected to linear transformation and SiLU activation function, and then matrix multiplied with the value vector after linear transformation to obtain the key-value fusion vector; the query vector is sequentially subjected to linear transformation and SiLU activation function, and then matrix multiplied with the key-value fusion vector, and then normalized to obtain the output feature map of the FEB block.

[0020] Furthermore, the image size normalization process includes:

[0021] The scaling ratio is calculated by formula (1):

[0022]

[0023] In the formula, H orig , W orig is the original height and width of the image, H new and W new is the normalized height and width of the image;

[0024] Calculate the size of the image after scaling according to formula (2-3):

[0025] H scaled =γ×H orig (2)

[0026] W scaled =γ×W orig (3)

[0027] In the formula, H scaled , W scaled The height and width of the image after scaling;

[0028] Calculate the filling size according to formula (4-5) and fill the scaled image;

[0029]

[0030] Where padding h 、padding w Fill the image with the required height and width.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. The backbone network of YOLOv8 extracts hierarchical features from the input image through the CBS module and the C2F module, transitioning from low-level edge and texture information to high-level semantic features, and gradually abstracting the content of the input image. The CBS module plays a role in hierarchical extraction, and the C2f module enhances the features extracted by the previous CBS module, so that the latter CBS module can extract local and global information more comprehensively, significantly improving the feature extraction capability in low-light scenes. The feature enhancement block (FEB) of the C2fFEB module improves the global receptive field and context feature learning capabilities by introducing an attention mechanism, better capturing useful information in the feature map, and suppressing noise interference in low-light scenes. The FEB block uses the associative law of matrices as the theoretical basis to convert the high-dimensional matrix multiplication operation of the multi-head attention mechanism into multiple low-dimensional matrix multiplications, and replaces the complex Softmax activation function with a lightweight SiLU activation function, effectively reducing the computational complexity while ensuring detection accuracy.

[0033] 2. In order to solve the problem of blurred details of small targets in drone aerial photography in low-light scenes, a shallow feature fusion network and an additional detection head are added to the YOLOv8 network. The shallow feature fusion network is embedded in the neck network of YOLOv8 and is used to fuse the shallow features extracted by the backbone network to capture more detailed image information, which significantly improves the small target detection accuracy of the model in low-light environments.

[0034] 3. The adaptive feature fusion module (ASFF) is introduced into the head network of YOLOv8. The four feature maps extracted by the neck network are adaptively weighted and fused through the weight matrix. This can better solve the problems of inconsistent feature scales and blurred details caused by different flight altitudes and angles. It can make the predicted box output by the detection head have a higher degree of overlap with the real box, further improving the detection performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is the structural diagram of the target detection model;

[0036] Figure 2 This is the structural diagram of the C2fFEB module;

[0037] Figure 3 It is the structural diagram of FEB block;

[0038] Figure 4This is a comparison chart of the test results of different models on the DroneVehicle dataset;

[0039] Figure 5 This is a comparison chart of the test results of different models on the LLVIP dataset. DETAILED DESCRIPTION

[0040] Specific embodiments are given below in conjunction with the accompanying drawings. The specific embodiments are only used to introduce the technical solutions of the present invention in detail and are not intended to limit the protection scope of the present application.

[0041] The present invention provides a method for detecting drone aerial photography targets in low-light scenes based on YOLOv8 (hereinafter referred to as the method, see Figures 1 to 5 ), including the following steps:

[0042] Step 1: Obtain drone aerial images. Several drone aerial images form a data set and normalize the image size.

[0043] First, the scaling ratio is calculated by formula (1):

[0044]

[0045] In the formula, H orig , W orig is the original height and width of the image, H new and W new is the normalized height and width of the image;

[0046] Then, calculate the image size after scaling according to formula (2-3):

[0047] H scaled =γ×H orig (2)

[0048] W scaled =γ×W orig (3)

[0049] In the formula, H scaled , W scaled The height and width of the image after scaling;

[0050] Finally, the padding size is calculated according to formula (4-5) and the scaled image is padded to make the image size consistent with the normalized size;

[0051]

[0052] Where padding h 、padding w Fill the image with the required height and width.

[0053] Step 2: Improve the YOLOv8 network, such as Figure 1 As shown in the figure, the last C2f module of the YOLOv8 backbone network is replaced with a feature enhancement module (C2fFEB), a shallow feature fusion network is embedded in the neck network, and an adaptive feature fusion module (ASFF) is embedded in the head network to obtain a target detection model;

[0054] The C2fFEB module is obtained by embedding the feature enhancement block (FEB) on the basis of the C2f module, in order to better capture the useful information in the feature map and suppress the noise interference in low-light scenes. Figure 2 As shown in FIG. 1 , the C2fFEB module includes a CBS module, a FEB block and a Bottleneck block; the input feature map f0 passes through a CBS module to obtain a feature map f1; the feature map f1 is split into two identical feature maps f2 and f2 through a split operation (Split). ’ , feature maps f2 and f2 ’ The number of channels of feature map f1 is half of that of feature map f1; feature map f2 is respectively subjected to multiple parallel feature enhancement branches consisting of FEB blocks and Bottleneck blocks connected in sequence, and then combined with feature map f2 ’ The feature map f3 is obtained by splicing; the feature map f3 passes through a CBS module to obtain the output feature map of the C2fFEB module.

[0055] The FEB block uses a multi-head attention mechanism to capture global information, such as Figure 3 As shown in the figure, the input feature map is multiplied with three updateable weight matrices to obtain the query vector (Q), key vector (K) and value vector (V); the key vector is sequentially subjected to linear transformation (Linear) and SiLU activation function, and then matrix multiplied (MatMul) with the value vector after linear transformation to obtain the key-value fusion vector; the query vector is sequentially subjected to linear transformation and SiLU activation function, and then matrix multiplied with the key-value fusion vector, and then normalized (Scale) to obtain the output feature map of the FEB block.

[0056] The query vector, key vector, and value vector are converted into vectors of dimensions n×d, d×n, and n×d respectively through linear transformation, where n>>d; if the query vector is matrix-multiplied with the key vector first, the calculation dimension will become n×n, which will lead to a sharp increase in the amount of calculation. Therefore, the FEB block preferentially performs matrix multiplication of the key vector and the value vector, and then performs matrix multiplication with the query vector, and the calculation dimension becomes d×d, thereby avoiding the large computational complexity caused by large-scale matrix multiplication and reducing the computational overhead. The SiLU activation function further improves the computational efficiency and significantly shortens the detection time. Therefore, the FEB block effectively reduces the computational cost of the model while ensuring the ability to extract global information.

[0057] The shallow feature fusion network is used to extract the shallow feature map extracted by the YOLOv8 backbone network (i.e., the output feature map of the first C2f module), such as Figure 1 As shown in the figure, the network includes two feature fusion branches. The first feature fusion branch is located at the end of the bottom-up branch of the original YOLOv8 neck network, including upsampling and splicing operations connected from bottom to top. The output feature map of the last C2f module of the bottom-up branch of the original YOLOv8 neck network is spliced ​​with the output feature map of the first C2f module of the backbone network after the upsampling operation to obtain the feature map g, which is the output feature map of the first feature fusion branch; the second feature fusion branch is located at the beginning of the top-down branch of the original YOLOv8 neck network, including the C2f module, the CBS module, the splicing operation and the C2f module connected in sequence from top to bottom. The output feature map of the first feature fusion branch is the input feature map of the second feature fusion branch. After the feature map g passes through a C2f module and a CBS module in sequence, it is spliced ​​with the output feature map of the last C2f module of the bottom-up branch of the original YOLOv8 neck network, and then passes through a C2f module to obtain the feature map x2, which is the output feature map of the second feature fusion branch.

[0058] The output feature maps of the improved YOLOv8 neck network include four, which are the output feature maps x1 and x2 of the two C2f modules of the second feature fusion branch of the shallow feature fusion network and the output feature maps x3 and x4 of the two C2f modules of the top-down branch of the original YOLOv8 neck network. The feature maps x1, x2, x3 and x4 are input into the ASFF module, which performs weighted fusion on the four feature maps to obtain four fused feature maps y1 to y4. The four fused feature maps are expressed as:

[0059]

[0060] In the formula, α i , β i , ε i and η i They are all weight matrices, i=1,2,3,4;

[0061] Each weight matrix is ​​expressed as:

[0062]

[0063] Where λ is a learnable parameter;

[0064] The fused feature maps y1~y4 are passed through a detection head respectively to classify the target object and generate the location information of the target object, including the center point coordinates, height and width of the prediction box. The center point coordinates (x, y) of the prediction box are calculated by formula (8).

[0065]

[0066] Where σ is the sigmoid function, (x anchor ,y anchor ) is the coordinate of the center point of the anchor box, Δx and Δy are the offsets of the prediction box in the x and y directions;

[0067] The height and width of the prediction box are calculated by formula (9):

[0068]

[0069] In the formula, p h and p w are the height and width of the anchor box, Δh and Δw are the height and width offsets of the predicted box.

[0070] The four detection heads output prediction frames of different scales, large, medium and small. The outputs of the four detection heads are integrated together and processed by NMS to obtain the best prediction frame. Since multiple prediction frames of different sizes may be obtained at the same position, the intersection and union ratio of the prediction frame with the highest confidence and the remaining prediction frames is calculated according to formula (10). The larger the intersection and union ratio, the higher the overlap, that is, the redundant frame, which needs to be suppressed to obtain the best prediction frame;

[0071]

[0072] Where A is the prediction box with the highest confidence, B is the remaining prediction box, A∩B represents the overlapping area of ​​the two boxes, and A∪B represents the total coverage area of ​​the two boxes.

[0073] Step 3: Use the data set to train the target detection model, and use the trained target detection module to detect drone aerial photography targets in low-light scenes.

[0074] Example

[0075] In this embodiment, the target detection model of the present invention is tested on two data sets, DroneVehicle and LLVIP, and the existing models YOLOv5, YOLOv6, YOLOv8 and YOLOv10 are used for comparison. The mAP50 and mAP50:95 indicators are selected to evaluate the model performance. mAP50 means that the average average precision of all categories is calculated when the IoU is 0.50, and mAP50:95 means that the average precision of each category is calculated under multiple thresholds of IoU from 0.50 to 0.95 (with a step size of 0.05), and the average of all average precisions is taken.

[0076] Figure 4 , 5 They are the test results of different models on the DroneVehicle and LLVIP data sets. As can be seen from the figure, compared with the existing model, the target detection model of the present invention has the maximum values ​​of mAP50 and mAP50:95 on the two data sets, indicating that the model has good detection accuracy. The reason is: first, in order to suppress noise interference in low-light scenes, a feature enhancement block (FEB) is embedded in the last C2f module of the YOLOv8 backbone network, which not only takes advantage of the multi-head attention mechanism's ability to capture global information, but also avoids the large amount of computational cost consumed by the multi-head attention mechanism. At the same time, the SiLU function is used instead of the Softmax function to save computing time, ensure accuracy and improve efficiency. Second, in order to make full use of the rich details and edge information contained in the shallow features, a shallow feature fusion network is embedded in the YOLOv8 neck network. Third, in order to more accurately identify and locate the target, the ASFF module is added to the YOLOv8 head network, and the four feature maps output by the neck network are adaptively weighted and fused, so that the predicted frame obtained by the detection head has a greater overlap with the real frame. In summary, the target detection model of the present invention has higher accuracy and consumes less time in drone aerial photography target detection in low-light scenes.

[0077] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A method for detecting drone aerial targets in low-light scenes based on YOLOv8, characterized in that: The method comprises the following steps: Step 1: Obtain drone aerial images, several images form a data set, and normalize the image size; Step 2: Improve the YOLOv8 network by replacing the last C2f module of the YOLOv8 backbone network with a feature enhancement module, embed a shallow feature fusion network in the neck network, and embed an adaptive feature fusion module in the head network to obtain the target detection model. The shallow feature fusion network includes two feature fusion branches, the first feature fusion branch is located at the end of the bottom-up branch of the original YOLOv8 neck network, including upsampling and splicing operations connected from bottom to top, the output feature map of the last C2f module of the bottom-up branch of the original YOLOv8 neck network is spliced ​​with the output feature map of the first C2f module of the backbone network after the upsampling operation, so as to obtain the output feature map of the first feature fusion branch; the second feature fusion branch is located at the beginning of the top-down branch of the original YOLOv8 neck network, including a C2f module, a CBS module, a splicing operation and a C2f module connected in sequence from top to bottom, the output feature map of the first feature fusion branch passes through a C2f module and a CBS module in sequence, and then is spliced ​​with the output feature map of the last C2f module of the bottom-up branch of the original YOLOv8 neck network, and then passes through a C2f module to obtain the output feature map of the second feature fusion branch; The adaptive feature fusion module performs weighted fusion on the output feature maps x1 and x2 of the two C2f modules of the second feature fusion branch of the shallow feature fusion network and the output feature maps x3 and x4 of the two C2f modules of the top-down branch of the original YOLOv8 neck network to obtain four fused feature maps y1 to y4. The four fused feature maps are expressed as: In the formula, α i , β i , ε i and η i They are all weight matrices, i=1,2,3,4; Step 3: Use the data set to train the target detection model, and use the trained target detection module to detect drone aerial photography targets in low-light scenes.

2. According to the YOLOv8-based drone aerial target detection method in low-light scenes according to claim 1, it is characterized in that: The representation of each weight matrix is ​​similar, α i It is expressed as: Where λ is a learnable parameter.

3. The method for detecting drone aerial targets in low-light scenes based on YOLOv8 according to claim 1 or 2, characterized in that: The feature enhancement module includes a CBS module, a FEB block and a Bottleneck block; the input feature map f0 passes through a CBS module to obtain a feature map f1; the feature map f1 is split into two identical feature maps f2 and f2 ’ , the feature map f2 is respectively passed through multiple parallel feature enhancement branches consisting of FEB blocks and Bottleneck blocks connected in sequence, and then combined with the feature map f2 ’ The feature map f3 is concatenated and obtained; the feature map f3 passes through a CBS module to obtain the output feature map of the feature enhancement module; among them, the FEB block uses a multi-head attention mechanism to extract features.

4. The method for detecting drone aerial targets in low-light scenes based on YOLOv8 according to claim 3 is characterized in that: In the FEB block, the input feature map is multiplied with three updateable weight matrices respectively to obtain the query vector, key vector and value vector; the key vector is sequentially subjected to linear transformation and SiLU activation function, and then matrix multiplied with the value vector after linear transformation to obtain the key-value fusion vector; the query vector is sequentially subjected to linear transformation and SiLU activation function, and then matrix multiplied with the key-value fusion vector, and then normalized to obtain the output feature map of the FEB block.

5. The method for detecting drone aerial targets in low-light scenes based on YOLOv8 according to claim 1, characterized in that: Image size normalization includes: The scaling ratio is calculated by formula (1): In the formula, H orig , W orig H is the original height and width of the image. new and W new is the normalized height and width of the image; Calculate the size of the image after scaling according to formula (2-3): H scaled =γ×H orig (2) IN scaled =γ×W orig (3) In the formula, H scaled , W scaled The height and width of the image after scaling; Calculate the filling size according to formula (4-5) and fill the scaled image; Where padding h 、padding w Fill the image with the required height and width.

Images