Weak illumination target detection method based on improved YOLOv8

Abstract

The invention relates to the field of computer vision target detection, in particular to a weak illumination target detection method based on improved YOLOv8, which comprises the following steps: acquiring a weak illumination image data set, converting the weak illumination image data set into a data set suitable for a YOLO format, and dividing the data set into a training set, a verification set and a test set; an improved YOLOv8 weak illumination image target detection model is constructed; wherein a space and channel recombination convolution and weighted bidirectional feature pyramid network module is introduced into the neck network; an efficient channel attention module is introduced into the backbone network and the neck network; and finally, introducing a loss function based on an auxiliary frame to optimize an original loss function. Training the improved YOLOv8 target detection model by using the training set to obtain a model pre-training weight; inputting a to-be-detected image into the optimal weight model for detection; according to the method, the target detection task in a weak light environment is better aimed at, the detection accuracy is improved, the model is lighter, and deployment on mobile equipment is facilitated.

Description

Technical Field

[0001] The present invention relates to the field of computer vision object detection, and specifically to a method for detecting faint light objects based on improved YOLOv8. Background Technique

[0002] Object detection, as one of the popular research fields in computer vision, is widely used in fields such as video surveillance and drone reconnaissance, and has broad application prospects. However, most of the existing object detection methods often face problems such as insufficient illumination, low contrast, and noise interference in faint light. These factors will make the feature extraction and fusion capabilities of object detection algorithms weak, affect the detection performance, and cause excessive computing power loss, making it difficult to be deployed on resource-constrained devices.

[0003] With the continuous progress of deep learning technology, object detection algorithms based on deep learning have become a research hotspot, mainly divided into two categories: two-stage object detection algorithms represented by RCNN, Fast RCNN, and Faster RCNN; single-stage object detection algorithms represented by SSD and YOLO. In addition, deep learning methods can learn feature representations from a large amount of data and obtain higher-level abstract information through deep network structures, so as to better adapt to the faint light environment.

[0004] Therefore, a method for detecting faint light objects based on improved YOLOv8 is proposed. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for detecting faint light objects based on improved YOLOv8 to solve the problems mentioned in the above background technique.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] A method for detecting faint light objects based on improved YOLOv8, the method comprising the following steps:

[0008] Step 1, obtain the ExDark dataset;

[0009] Step 2, convert the obtained ExDark dataset to a dataset suitable for the YOLO format;

[0010] Step 3, divide the ExDark dataset converted to the YOLO format into a training set, a validation set, and a test set;

[0011] Step 4, construct an improved YOLOv8 faint light object detection network, which improves the feature extraction layer, feature fusion layer, and loss function of the YOLOv8 network;

[0012] Step 5: Use the improved weak light object detection network of YOLOv8 as the detection model, and use the training set and validation set to train and validate the detection model to obtain the optimal detection model;

[0013] Step 6: Use the optimal detection model during training, take the images in the weak light environment as the input, and test the object detection network of the improved YOLOv8;

[0014] Preferably, in the above step 1, the ExDark dataset is a publicly available dataset downloaded from the official website.

[0015] Preferably, in the above step 2, since the original dataset is not applicable to YOLO, it is converted into a txt file in the YOLO sequence format.

[0016] Preferably, in the above step 3, the ExDark dataset converted into the YOLO format is divided into a training set, a validation set, and a test set.

[0017] Preferably, the above step 4 includes:

[0018] The improved weak light object detection model of YOLOv8 includes: introducing a spatial and channel recombination convolution module (SCConv) in the neck network to replace the Bottleneck module in C2f, and introducing a weighted bidirectional feature pyramid network module (BiFPN module) to replace the Concat module; introducing an efficient channel attention module (ECA-Net module) in the backbone network and the neck network, adding the ECA-Net module to the SPPF layer in the backbone network and adding the ECA-Net module to each BiFPN module in the neck network; finally, introducing a loss function based on auxiliary bounding boxes to optimize the original loss function.

[0019] Introduce SCConv to improve the neck network: The C2f module in the original network has a large computational burden, resulting in excessive computational load and slow detection speed. In order to enable the model to improve the detection accuracy while reducing the demand for computing resources, the lightweight convolution module spatial and channel recombination convolution (SCConv) is used to replace the Bottleneck module in the original C2f module.

[0020] Introducing BiFPN to Improve the Neck Network: YOLOv8 continues to use the bidirectional fusion pyramid structure of PAN-FPN, which fuses multi-scale features together through horizontal connections and a pyramidal hierarchical structure. However, due to the different resolutions of the input characteristics, the contribution of the PAN-FPN structure to the fused output characteristics is often uneven, and feature information at different scales is often treated equally, resulting in the underutilization of characteristics between different scales. The present invention uses BiFPN to improve the neck network. By introducing weights and performing different fusions on different input features, the network model focuses more on learning key feature information, thereby enhancing the feature fusion ability of the network.

[0021] Introducing the ECA-Net Module to Improve the Backbone Network and Neck Network: ECA-Net first processes the input feature map through global average pooling, compressing the two-dimensional feature matrix into a single value. Then, while keeping the dimension unchanged, the module generates channel weights through a one-dimensional convolution of size k to capture the interdependence between channels. Finally, the generated channel weights are then element-wise multiplied by the original input feature map to complete the recalibration of features in the channel space, that is, the feature map is weighted and adjusted by ECA-Net. While reducing the parameters, it improves the model's attention to the feature information of images under weak illumination.

[0022] Introducing Inner-IoU to Optimize the Loss Function of the Original Model: Inner-IoU focuses on the core region of the target to be detected under weak illumination, making the evaluation results more accurate. In addition, a ratio factor is added to Inner-IoU, which can control the proportion size of the auxiliary bounding box. By using bounding boxes of different scales for different datasets and detectors, the limitation of the weak generalization ability of existing methods can be overcome. For the ExDark dataset, the ratio is set to 1.3.

[0023] Preferably, in step 5, the weak-light target detection network of YOLOv8 is improved and trained, and the validation set data is used to detect whether the data is overfitted to complete the training of the model and obtain the optimal weak-light target detection model of the improved YOLOv8.

[0024] Preferably, in step 6, the trained model weight file is input into the test set of the ExDark dataset, and the model will mark and frame the detected targets on the output pictures, and finally obtain the actual recognition results in a weak environment.

[0025] On the other hand, a weak-light target detection device based on the improved YOLOv8 is provided for implementing the method described in any one of the above, and the device includes:

[0026] Data acquisition module: used to acquire a weak light image dataset, perform data format conversion and divide it into a training set, a validation set, and a test set;

[0027] Model improvement module: used to build an improved YOLOv8 weak light object detection model;

[0028] Among them, the improved YOLOv8 weak light object detection model includes: introducing a spatial and channel reorganization convolution - SCConv in the neck network to replace the Bottleneck module in C2f, and introducing a weighted bidirectional feature pyramid network module - BiFPN module to replace the Concat module; introducing an efficient channel - ECA - Net attention module in the backbone network and the neck network, adding an ECA - Net module to the SPPF layer in the backbone network and adding an ECA - Net module to each BiFPN module in the neck network; finally, introducing a loss function based on auxiliary bounding boxes - Inner - IoU to optimize the original loss function;

[0029] Training and detection module: used to input the weak light image dataset into the improved YOLOv8 weak light object detection model for training and obtain the detection results.

[0030] On the other hand, an electronic device is provided, and the electronic device includes:

[0031] A memory for storing a computer program that can run on a processor; a processor for executing the steps of the weak light object detection method as described above when running the computer program.

[0032] On the other hand, a computer - readable storage medium is provided, and a computer program is stored on the storage medium, and the computer program can execute the steps of the weak light object detection method as described above when executed by at least one processor.

[0033] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0034] (1) Using the C2f_SC module to replace the original C2f module in the feature fusion network effectively reduces the redundant features in the spatial dimension and the channel dimension, and adding an ECA - Net module behind each C2f_SC module enables the model to learn more efficient feature map information and improves the detection accuracy.

[0035] (2) Replacing the Concat module with the BiFPN module in the feature fusion network, thereby introducing weights to perform different fusions on different input features, making the network model focus more on learning key feature information, and thus enhancing the learning ability of the network.

[0036] (3) An ECA-Net attention module is introduced after the SPPF layer in the feature extraction network to enhance the model's feature recognition ability under weak illumination.

[0037] (4) The loss function of the original network is optimized using Inner-IoU to enhance the accuracy and convergence ability of the model detection.

[0038] This invention conducts detection in weak illumination scenarios. Compared with the traditional YOLOv8 model, the accuracy rate is increased by 5.7%, the recall rate is increased by 5.7%, and mAP_0.5 is increased by 5.3%, effectively improving the performance in the detection task under weak illumination. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0040] Figure 1 is a flowchart provided by an embodiment of the present invention;

[0041] Figure 2 is a structural diagram of an improved YOLOv8 weak illumination target detection model provided by an embodiment of the present invention;

[0042] Figure 3 is a structural diagram of SCConv provided by an embodiment of the present invention;

[0043] Figure 4 is a structural diagram of BiFPN provided by an embodiment of the present invention;

[0044] Figure 5 is a structural diagram of ECA-Net provided by an embodiment of the present invention;

[0045] Figure 6 is a structural diagram of Inner-Iou provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] Please refer to Figures 1-6 , the present invention provides the following technical solutions:

[0048] Embodiment 1:

[0049] Weak Light Object Detection Method Based on Improved YOLOv8, such as Figure 1 shown, includes the following steps:

[0050] Step 1: Obtain the ExDark dataset;

[0051] Step 2: Convert the obtained ExDark dataset to a dataset suitable for the YOLO format;

[0052] Step 3: Divide the ExDark dataset converted to the YOLO format into a training set, a validation set, and a test set;

[0053] Step 4: Construct an improved weak light object detection network for YOLOv8, which improves the feature extraction layer, the feature fusion layer, and the loss function of the YOLOv8 network.

[0054] Step 5: Use the improved weak light object detection network for YOLOv8 as the detection model, and use the training set and the validation set to train and validate the detection model to obtain the optimal detection model;

[0055] Step 6: Use the optimal detection model during training, take the image in the weak light environment as the input, and test the object detection network of the improved YOLOv8.

[0056] Preferably, step 1 is specifically as follows:

[0057] The Exclusively-Dark dataset downloaded publicly from the official website is used in the present invention. The categories of this dataset include Bicycle, Boat, Bottle, Bus, Car, Cat, Chair, Cup, Dog, Motorbike, People, Table, and a total of 12 common objects are labeled.

[0058] Preferably, step 2 is specifically as follows:

[0059] The ExDark dataset format is not suitable for YOLO and needs to be converted. It is converted to a txt file in the YOLO sequence through a python script.

[0060] Preferably, step 3 is specifically as follows:

[0061] Use a python script to randomly divide the converted ExDark dataset suitable for the YOLO format into a training set, a validation set, and a test set.

[0062] Preferably, step 4 is specifically as follows:

[0063] The improved weak light object detection model for YOLOv8, such as Figure 2As shown in the figure, specifically including: introducing spatial and channel reorganization convolution - SCConv in the neck network to replace the Bottleneck module in C2f, and introducing the weighted bidirectional feature pyramid network module - BiFPN module to replace the Concat module; introducing the efficient channel - ECA - Net attention module in the backbone network and the neck network, adding the ECA - Net module to the SPPF layer in the backbone network and adding the ECA - Net module to each BiFPN module in the neck network; finally, introducing the loss function based on auxiliary bounding boxes - Inner - IoU to optimize the original loss function;

[0064] First, introduce the SCConv module, as Figure 3 shown in the figure, to improve the neck network: the C2f module in the original network has a large computational burden, resulting in excessive operation load and slow detection speed. To enable the model to reduce the demand for computing resources while improving the detection accuracy, the lightweight convolution module spatial and channel reorganization convolution (SCConv) is used to replace the Bottleneck module in the original C2f module. The SCConv module consists of two core components: the spatial reorganization unit (SRU) and the channel reorganization unit (CRU). The SRU uses the separation and reorganization mechanism to reduce spatial redundancy, thereby obtaining the spatial detail feature X^ω, thus reducing the computational burden. For the input feature X, first perform group normalization (GN) to evaluate the information volume of each feature. The formula is as follows:

[0065]

[0066] where μ is the mean, σ is the standard deviation, ε is a small positive number used to avoid numerical instability in the calculation, γ and β are trainable parameters in the model, and the channel weight W after normalization γ , the formula is as follows:

[0067]

[0068] Then, re - weight the weight W γ through the sigmoid function, map its value to the interval (0, 1), and set the gating threshold to 0.5. Set the weights exceeding this threshold to 0 to obtain the key feature W1, and set the weights not exceeding it to 1 to obtain the non - key feature W2. The formula for W is as follows:

[0069] W = Gate(Sigmoid(W γ (GN(X))))

[0070] Finally, adopt the cross - addition operation to enhance the feature discrimination and reduce the redundant features in the spatial dimension, thereby obtaining the spatial detail feature X w . The calculation process is as follows:

[0071]

[0072] To continue to weaken X w 's redundancy, the CRU uses operations of splitting, transformation, and fusion. In the splitting stage, X w is divided into X up and X low . In the transformation stage, X up adopts grouped convolution (GWC) and pointwise convolution (PWC) to obtain feature Y1, and X low adopts pointwise convolution (PWC) operation to obtain feature Y2. In the fusion stage, the model can integrate the features Y1 and Y2 output in the transformation stage to obtain the channel detail feature Y. By continuously applying SRU and CRU, the computational burden of the model can be effectively reduced, and the information redundancy of the feature map can be greatly weakened.

[0073] Then the BiFPN module is introduced, as Figure 4 shown, to improve the neck network: YOLOv8 continues to use the bidirectional fusion pyramid structure of PAN-FPN to fuse multi-scale features together through horizontal connections and a pyramid-shaped hierarchical structure. However, due to the different resolutions of the input characteristics, the contribution of the PAN-FPN structure to the fused output characteristics is often uneven, and the feature information of different scales is often treated equally, resulting in the characteristics between different scales not being fully utilized. The present invention uses BiFPN to improve the neck network. By introducing weights, different fusions are performed on different input features, making the network model focus more on learning key feature information, thereby improving the feature fusion ability of the network.

[0074] Then the ECA-Net module is introduced, as Figure 5 shown, to improve the backbone network and the neck network: ECA-Net first processes the input feature map through global average pooling, compressing the two-dimensional feature matrix into a single value. Then, while keeping the dimension unchanged, this module generates channel weights through one-dimensional convolution of size k to capture the interdependence between channels. Finally, the generated channel weights are then multiplied element-wise by the original input feature map to complete the recalibration of the features in the channel space, that is, the feature map is weighted and adjusted by ECA-Net. In addition, ECA-Net uses the k-nearest neighbor mechanism to promote local interaction, significantly reducing the computational amount and complexity required for full-channel interaction. By generating weights for each feature channel through one-dimensional convolution of size k, the correlation between feature channels is revealed. This design not only optimizes the feature extraction process but also improves the sensitivity of the model to key information while reducing the computational burden of the model. The calculation formula for the kernel size is as follows:

[0075]

[0076] Among them, C is the number of channels of the input features, and γ and b are hyperparameters. In the formula, |t| odd represents the closest odd number to t. Moreover, in the experiments of this article, γ and b are set to 2 and 1 respectively. While reducing the parameters, it improves the model's attention to the image information features under weak light.

[0077] Finally, Inner-IoU is introduced. As Figure 6 shown, the loss function of the original model is optimized: Inner-IoU makes the evaluation result more accurate by focusing on the core area of the target to be detected under weak light. In addition, a scale factor (ratio) is added to Inner-IoU, which can control the scale of the auxiliary bounding box. By using bounding boxes of different scales for different datasets and detectors, the limitation of the weak generalization ability of the existing methods can be overcome. When the ratio is greater than 1, there is a larger auxiliary bounding box, which improves the model when the IoU loss is small. When the ratio is less than 1, the size of the auxiliary bounding box is smaller, but its gradient is larger than the gradient of the IoU loss, which helps the convergence of samples with high IoU loss. For the ExDark dataset, the ratio is set to 1.3. Therefore, the present invention uses Inner-CIoU Loss instead of CIoU Loss. The definition formula of Inner-CIoU is as follows:

[0078] L Inner-CloU = L CIoU + IoU - IoU inner

[0079] In the formula: L CIoU and IoU are the loss function and intersection over union in the original model. IoU inner is the intersection over union.

[0080] Preferably, step 5 is specifically:

[0081] Input the ExDark training set in step 3 into the configured environment, and train on the training set of the ExDark dataset until the loss function curve converges. Obtain the optimal training weight file of the model.

[0082] Preferably, step 6 is specifically:

[0083] Use the weak light target detection model described above to detect weak light images: judge the extracted features, accurately identify and locate the targets under weak light, and obtain the actual detection result map of the improved YOLOv8 model in the weak light environment.

[0084] Embodiment 2:

[0085] This embodiment also provides a low-light target detection device based on improved YOLOv8, which specifically includes the following:

[0086] Data acquisition module: used to implement obtaining the low-light image dataset in steps 1-3, performing data format conversion, and dividing it into a training set, a validation set, and a test set;

[0087] Model improvement module: used to implement constructing an improved YOLOv8 low-light target detection model in step 4;

[0088] Among them, the improved YOLOv8 low-light target detection model includes: introducing a spatial and channel recombination convolution - SCConv in the neck network to replace the Bottleneck module in C2f, and introducing a weighted bidirectional feature pyramid network module - BiFPN to replace the Concat module; introducing an efficient channel - ECA-Net attention module in the backbone network and the neck network, adding an ECA-Net module to the SPPF layer in the backbone network and adding an ECA-Net module to each BiFPN module in the neck network; finally, introducing a loss function based on auxiliary bounding boxes - Inner-IoU to optimize the original loss function;

[0089] Training and detection module: used to input the low-light image dataset into the improved YOLOv8 low-light target detection model for training and obtain the detection results.

[0090] Embodiment 3:

[0091] This embodiment also provides an electronic device, which includes: a memory for storing a computer program that can run on a processor; a processor for executing a low-light target detection method based on improved YOLOv8 described in Embodiment 1 when running the computer program.

[0092] Embodiment 4:

[0093] This embodiment also provides a computer-readable storage medium, on which a computer program is stored, and the computer program implements the steps of the low-light target detection method based on improved YOLOv8 when executed by at least one processor.

[0094] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for detecting faint light targets based on improved YOLOv8, characterized in that: The method includes: Step 1, obtain the ExDark dataset; Step 2, convert the obtained ExDark dataset to a dataset suitable for the YOLO format; Step 3, divide the ExDark dataset converted to the YOLO format into a training set, a validation set, and a test set; Step 4, construct an improved YOLOv8 low-light object detection network, which improves the feature extraction layer, the feature fusion layer, and the loss function of the YOLOv8 network; Step 5, use the improved YOLOv8 low-light object detection network as the detection model, and use the training set and the validation set to train and validate the detection model to obtain the optimal detection model; Step 6, use the optimal detection model during training, take the image in the low-light environment as the input, and test the object detection network of the improved YOLOv8.

2. The method for detecting faint light targets based on improved YOLOv8 according to claim 1, wherein In the step 2, the ExDark dataset file is converted into a txt file in the YOLO sequence.

3. A method for detecting faint light targets based on improved YOLOv8 as claimed in claim 1, characterized in that, The YOLOv8 low-light object detection network in the step 4 includes a backbone network, a neck network, and a Head network; The backbone network is used to extract the image features in the low-light environment; The neck network is used to perform feature fusion on the extracted image features; The Head network is used to predict the bounding box and the category according to the image after feature fusion, and obtain the weight file required for the prediction process.

4. The method for detecting faint light targets based on improved YOLOv8 according to claim 3, wherein, The improvement of the backbone network includes: adding an efficient channel attention module to the next layer of the SPPF layer in the backbone network and the next layer of each C2f_SC in the neck network; The efficient channel attention module includes: first, the input feature map is processed by global average pooling to compress the two-dimensional feature matrix into a single value. Then, without changing the dimension, the efficient channel attention module generates channel weights through a one-dimensional convolution of size k to capture the interdependence between channels. Finally, the generated channel weights are then element-wise multiplied by the original input feature map to complete the recalibration of the features in the channel space, that is, the feature map is weighted and adjusted by ECA-Net; while reducing the parameters, it improves the model's attention to the image information features under low light.

5. The method for detecting dim light targets based on improved YOLOv8 according to claim 3, characterized in that, The improvement of the neck network includes: introducing a spatial and channel recombination convolution module and a weighted bidirectional feature pyramid network module respectively; For the spatial and channel recombination convolution module, set as the C2f_SC module, the Bottleneck module in the original C2f is replaced by the C2f_SC module, which significantly reduces the redundancy of features in the spatial and channel dimensions, reduces the model parameter quantity while reducing the computational consumption, and thus improves the detection accuracy of the model under low light; For the weighted bidirectional feature pyramid network module, set as the BiFPN module, the concat module in the original neck network is replaced by the BiFPN module to introduce weights, perform different fusions on different input features, make the network model focus more on learning key feature information, and thus improve the feature fusion ability of the network.

6. The method for detecting weak light targets based on improved YOLOv8 according to claim 1, characterized in that, The improvement of the loss function includes: Design a new auxiliary target prediction box loss function and obtain the prediction loss value L according to the formula Inner-CloU : L Inner-CloU = L CIoU + IoU - IoU inner Among them, L CIoU and IoU are the loss function and intersection over union in the original model.

7. The method for detecting weak light targets based on improved YOLOv8 according to claim 1, characterized in that, In step 5, with the goal of improving the low-light object detection network of YOLOv8 as the object detection model, the training set in the ExDark dataset is used to train the improved YOLOv8 network, and the validation set data is used to detect whether the model is overfitting. After completing the training of the model, the optimal low-light object detection model of the improved YOLOv8 is obtained.

8. A method for detecting faint light targets based on improved YOLOv8 according to claim 1, characterized in that, In step 6, when the trained model weight file is input into the test set of the ExDark dataset, the model will label and frame the detected objects on the output image, and finally obtain the actual recognition results in a low-light environment.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements the steps in a low-light object detection method based on the improved YOLOv8 as described in any one of claims 1-6.

10. A computer device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the program, it implements the steps in a low-light object detection method based on the improved YOLOv8 as described in any one of claims 1-6.

Images