A lightweight low-illumination target detection method

Abstract

A lightweight low-illumination target detection method, comprising: obtaining public sample data Exdark dataset and dividing into training set, verification set and test set; replacing the 4th, 6th and 8th layers in the Backbone feature extraction network of YOLOv8 with Adown downsampling; replacing the FPN feature pyramid of Neck with BiFPN; replacing the CIoU of the bounding box regression loss function with Powerful-IoU (PIoU); optimizing the YOLOv8 target detection algorithm, constructing a lightweight low-illumination target detection model; training the optimized low-illumination target detection model based on the training set; using the trained low-illumination target detection model and the image to be detected to detect the image under low illumination, and outputting the target detection result. Finally, the model size is reduced and the detection accuracy of the model is enhanced. The present application studies the target detection in the low-illumination scene by improving the YOLOv8 algorithm, and detects the multi-scale, small target and other test samples under low illumination; the target detection accuracy under insufficient illumination can be improved; the reliability and stability of the system are also improved.

Description

Technical Field

[0001] This invention relates to the field of target detection technology in low-light scenes, and specifically to a lightweight low-light target detection method. Background Technology

[0002] Object detection is a crucial prerequisite for advanced vision research and a core issue in computer vision research. Currently, object detection tasks under normal lighting conditions have experienced rapid development thanks to technological advancements.

[0003] Low-light target detection methods have broad application prospects in fields such as nighttime surveillance, intelligent transportation, military security, autonomous driving, and disaster relief. They can improve the perception capabilities of equipment and its ability to cope with complex environments, providing strong support for safety and management in related fields. However, under conditions such as at night, insufficient indoor lighting, or inclement weather, targets in low-light environments often appear blurry and lack detail, and are also subject to noise interference, making it difficult for target detection systems to accurately detect targets.

[0004] To address the issues of low overall brightness, low contrast, and unexpected noise in low-light images, researchers have proposed numerous low-light image enhancement techniques, such as image enhancement based on dark channel priors, enhancement methods based on histogram equalization, and enhancement techniques based on multi-frame image fusion. While these techniques can effectively improve the brightness and contrast of low-light images and reduce noise, the visual effect of the enhanced images deteriorates, resulting in image quality degradation. This not only reduces the performance of computer vision and image analysis algorithms but also diminishes the visual aesthetics of the images, making them unsuitable for use as input in other computer vision tasks such as object classification and detection. Summary of the Invention

[0005] To address the aforementioned technical problems, this technical solution provides a lightweight low-light target detection method. By improving the YOLOv8 structure, the model performance is enhanced, improving the target detection accuracy and speed in low-light scenes, while reducing the model size to achieve real-time target detection in low-light scenes; thus effectively solving the above problems.

[0006] This invention is achieved through the following technical solution: A lightweight low-light target detection method, comprising the following steps: Step 1: Obtain the publicly available sample data Exdark dataset, convert its labels to YOLO training format, and divide it into training, validation, and test sets; Step 2: Optimize the YOLOv8 object detection algorithm to build a lightweight low-light object detection model; the specific operation method is as follows: Step 2.1: Replace layers 4, 6, and 8 of the YOLOv8 Backbone feature extraction network with Adown downsampling; the improved YOLOv8 backbone network includes the following sequentially connected modules: first convolutional module, second convolutional module, first C2f module, first Adown downsampling module, second C2f module, second Adown downsampling module, third C2f module, third Adown downsampling module, fourth C2f module, and SPPF spatial pyramid pooling structure. Step 2.2: Replace Neck's FPN feature pyramid with BiFPN, and make skip connections between the 10th and 20th layers of the model, as well as between the 11th and 23rd layers; Step 2.3: Replace the bounding box regression loss function in YOLOv8 from CIoU to Powerful-IoU (PIoU); Step 3: Train the optimized low-light target detection model based on the training set divided in Step 1 to obtain the optimal low-light target detection model; Step 4: Use the trained low-light target detection model and the image to be detected to detect the image under low light conditions and output the target detection results.

[0007] Furthermore, the sample data Exdark dataset mentioned in step 1 is the Exdark dataset proposed by Yuen Peng Loh et al. in 2018, which is currently the largest set of object-level annotations for low-light images under natural light.

[0008] Furthermore, the Exdark dataset contains 7,363 low-light images under 10 different conditions, ranging from very low-light environments to dusk, as well as target bounding boxes for 12 target classes at the image class level and with local annotations.

[0009] Furthermore, in step 3, when training the optimized low-light object detection model based on the training set divided in step 1, the input image size is set to 640*640, the batch size is 16, the number of training iterations is 300, the initial learning rate is 0.01, the learning rate momentum is 0.937, the weight decay coefficient is 0.0005, the optimizer is SGD, the random seed is fixed during the experiment, and no pre-trained weights are used throughout the process; after training, the model will save the best weight file best.pt.

[0010] Furthermore, in step 4, the method of using the trained low-light target detection model and the image to be detected to detect images under low light conditions involves using best.pt as the weight file for detecting low-light images and the image to be detected as the data source for detection. Beneficial effects

[0011] The lightweight low-light target detection method proposed in this invention has the following advantages compared with existing technologies: This invention replaces layers 4, 6, and 8 of the YOLOv8 Backbone feature extraction network with Adown downsampling. Adown downsampling uses a generalized efficient layer aggregation network (GELAN). GELAN integrates the design concepts of CSPNet and ELAN, creating a flexible network architecture that can be adjusted according to different application requirements and computing resources. It comprehensively considers lightweight, inference speed, and accuracy, enabling the model to detect faster, with fewer parameters and higher accuracy.

[0012] This invention replaces the Feature Pyramid Network (FPN) in the Neck of YOLOv8 with BiFPN. BiFPN is a feature pyramid network structure for object detection, designed to improve the model's detection performance for objects at different scales. BiFPN integrates bidirectional cross-scale connections and fast normalized fusion. BiFPN introduces a dynamic weight mechanism that adaptively adjusts the weights of features based on the quality of each feature map, enabling the network to more effectively aggregate multi-scale features.

[0013] This invention replaces the bounding box regression loss function in YOLOv8 with CIoU (PIoU). Powerful-IoU (PIoU) delves into the reasons for anchor box enlargement and is a powerful IoU loss function that combines an adaptive penalty factor for the target size with a gradient adjustment function based on anchor box quality. The Powerful-IoU (PIoU) loss guides the anchor boxes to regress along an effective path, resulting in faster convergence than existing Powerful-IoU (PIoU) based loss functions, thus accelerating model inference.

[0014] The improved YOLOv8 model of this invention exhibits superior performance in target detection, enhancing detection accuracy and speed while reducing model size. This results in superior target detection performance, ultimately improving detection accuracy and reducing model size. Furthermore, by researching low-light target detection methods, the accuracy of target detection under insufficient lighting conditions can be improved, enhancing system reliability and stability. This also promotes technological innovation and development in related fields, driving the entire scientific and technological sector forward.

[0015] This invention studies target detection in low-light scenarios using an improved YOLOv8 algorithm, detecting test samples containing multi-scale and small targets under low light conditions. Addressing the difficulty of target recognition in low-light scenarios, an effective lightweight target detection model is proposed, achieving accurate target recognition in such conditions. Attached Figure Description

[0016] Figure 1 This is a flowchart of the low-light target detection method in this invention.

[0017] Figure 2 This is a diagram of the improved YOLOv8 network structure in this invention.

[0018] Figure 3 This is a diagram of the Adown downsampling network structure in this invention.

[0019] Figure 4 This is a diagram of the BiFPN feature pyramid network structure in this invention.

[0020] Figure 5 This is a schematic diagram of the results of low-light target detection using the original YOLOv8.

[0021] Figure 6 A schematic diagram illustrating the results of low-light target detection using the improved YOLOv8. Detailed Implementation

[0022] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. The described embodiments are merely some embodiments of the present invention, and not all embodiments. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the design concept of the present invention should fall within the protection scope of the present invention. Example

[0023] A lightweight low-light target detection method, such as Figure 1 As shown, the method includes the following steps: Step 1: Obtain the publicly available sample data Exdark dataset, convert its labels to YOLO training format, and divide it into training, validation, and test sets; In this embodiment, the sample data Exdark dataset is the one proposed by Yuen Peng Loh et al. in 2018, which is currently the largest collection of object-level annotations for low-light images under natural light. The Exdark dataset contains 7363 low-light images under 10 different conditions, ranging from very low-light environments to dusk, as well as bounding boxes of 12 object classes at the image class level and in local annotations.

[0024] Step 2: Optimize the YOLOv8 object detection algorithm to build a lightweight low-light object detection model; the specific operation method is as follows: Step 2.1: Replace layers 4, 6, and 8 of the YOLOv8 Backbone feature extraction network with Adown downsampling; the improved YOLOv8 backbone network includes the following sequentially connected modules: first convolutional module, second convolutional module, first C2f module, first Adown downsampling module, second C2f module, second Adown downsampling module, third C2f module, third Adown downsampling module, fourth C2f module, and SPPF spatial pyramid pooling structure.

[0025] Replace layers 4, 6, and 8 of the YOLOv8 Backbone feature extraction network with Adown downsampling. The Adown downsampling network structure diagram is shown below. Figure 3 As shown, Adown downsampling uses a generalized efficient layer aggregation network (GELAN) that takes into account lightweight, inference speed, and accuracy.

[0026] GELAN is a generalized, efficient layer aggregation network designed by combining two neural network architectures: Gradient Path Planning (CSPNet) and ELAN. GELAN comprehensively considers lightweight design, inference speed, and accuracy. It integrates the design principles of CSPNet and ELAN, creating a flexible network architecture that can be tailored to different application requirements and computational resources. CSPNet reduces redundant computation by segmenting and merging feature maps, while ELAN uses layer aggregation to enhance feature representation. Unlike traditional deep network architectures that rely on specific types of computational units, GELAN allows the use of various types of computational blocks within its framework, such as traditional convolutional layers, depthwise separable convolutions, or other novel computational units.

[0027] GELAN improves convolutional operations, enabling the network to better preserve and utilize gradient information during training. Gradient information is crucial for optimization in deep learning because it indicates how the loss function changes with model parameters. In this way, GELAN achieves higher parameter efficiency than traditional convolutional neural networks, maintaining high detection accuracy even with a smaller network size. It addresses the information loss problem inherent in traditional convolutional neural networks when handling object detection tasks and improves the efficiency of model parameter utilization. GELAN optimizes the network structure through a gradient path planning technique, allowing the network to more effectively optimize weights while preserving the integrity of input information, thereby improving detection accuracy.

[0028] By replacing layers 4, 6, and 8 of the YOLOv8 Backbone feature extraction network with Adown downsampling, the model not only improved performance but also reduced the number of parameters and increased speed.

[0029] Step 2.2: Replace Neck's FPN feature pyramid with BiFPN, and make skip connections between the 10th and 20th layers of the model, as well as between the 11th and 23rd layers.

[0030] The BiFPN feature pyramid network structure diagram is as follows: Figure 4 As shown, BiFPN is a feature fusion network for target detection. It improves the accuracy and efficiency of target detection through multi-level feature fusion and dynamic feature weight allocation. Specifically, BiFPN employs a bidirectional feature pyramid network structure, effectively addressing the information bottleneck and feature distortion problems inherent in feature pyramid networks through adaptive feature fusion and feature selection. Furthermore, BiFPN utilizes cross-level feature connections and multi-scale feature fusion to further enhance target detection performance.

[0031] BiFPN primarily addresses the issues of insufficient information transfer and incomplete feature fusion in feature pyramid networks. It improves the performance of feature pyramid networks by introducing a bidirectional information flow mechanism and cross-layer connections, enabling the network to better capture target features at different scales.

[0032] Key features of BiFPN include: simultaneous upsampling and downsampling operations, allowing information to flow freely between layers of different resolutions for better fusion and utilization of multi-scale features; the use of lateral and vertical connection mechanisms, enabling features from different layers to directly influence each other, helping to reduce information loss and improve information transfer efficiency; and the introduction of a dynamic weight mechanism, adaptively adjusting feature weights based on the quality of each feature map, allowing the network to more effectively aggregate multi-scale features.

[0033] In summary, BiFPN combines mechanisms such as bidirectional information flow, cross-layer connections, and dynamic weight adjustment to effectively improve the performance of feature pyramid networks in object detection tasks. Replacing the Neck FPN feature pyramid in YOLOv8 with BiFPN brings better accuracy and robustness to the model while reducing computational cost.

[0034] Step 2.3: Replace the bounding box regression loss function of YOLOv8 from CIoU to Powerful-IoU (PIoU).

[0035] Bounding box regression (BBR) is one of the core tasks in object detection, and the loss function Q of BBR has a significant impact on its performance. However, existing IoU-based loss functions are affected by unreasonable penalty factors, causing anchor boxes to swell during the regression process, significantly slowing down the convergence speed. Powerful-IoU (PIoU) delves into the reasons for anchor box enlargement and is a powerful IoU (Powerful-IoU(PIoU)) loss function that combines an adaptive penalty factor for the target size and a gradient adjustment function based on anchor box quality. The Powerful-IoU(PIoU) loss guides the anchor boxes to regress along an effective path, thus converging faster than existing Powerful-IoU(PIoU) loss functions and accelerating the model's inference speed. Furthermore, by studying the focusing mechanism, a non-monotonic attention layer is introduced and combined with PIoU to obtain a new loss function, PloU v2. The PIoU v2 loss enhances the focusing ability on medium-quality anchor boxes.

[0036] Step 3: Train the optimized low-light target detection model based on the training set divided in Step 1 to obtain the optimal low-light target detection model.

[0037] The optimized low-light object detection network was trained to obtain the optimal low-light object detection model. The input image size was set to 640*640, the batch size was 16, the number of training iterations was 300, the initial learning rate was 0.01, the learning rate momentum was 0.937, the weight decay coefficient was 0.0005, the optimizer used was SGD, the random seed was fixed during the experiment, and no pre-trained weights were used throughout the process.

[0038] The optimized low-light object detection model was trained using the training set partitioned in step 1 to obtain the optimal low-light object detection model. During training, the input image size was set to 640*640, the batch size was 16, the number of training iterations was 300, the initial learning rate was 0.01, the learning rate momentum was 0.937, the weight decay coefficient was 0.0005, the optimizer used was SGD, the random seed was fixed during the experiment, and no pre-trained weights were used throughout the process.

[0039] After training is complete, the model will save the best weight file, best.pt, based on the best training results.

[0040] During training, the superiority of a network can be determined through various model evaluation metrics. These metrics include: Precision (precision / accuracy), Recall (recall / accuracy), and Average Precision (AF). A PR curve can be plotted with recall on the horizontal axis and precision on the vertical axis. The area under the PR curve is defined as AP, and mAP (average AP for each class) is the average AP across all classes. The formulas for calculating each evaluation metric are as follows: ; ; ; ; Where IP represents the number of positive classes predicted as positive; TN represents the number of positive classes predicted as negative; FP represents the number of negative classes predicted as positive; FN represents the number of negative classes predicted as negative; r(k) represents the k-th largest recall; max(p(r(k))) is the maximum precision value at point r.

[0041] Step 4: Use the trained low-light target detection model and the image to be detected to detect the image under low light conditions. Use best.pt as the weight file when detecting low-light images and the image to be detected as the data source for detection; output the target detection results.

[0042] To verify the effectiveness of the improved modules in this invention in improving the model, eight ablation experiments were conducted under the same parameter conditions, focusing on the three innovative aspects of this invention. The experimental results are shown in Table 1: 0.719 11.2 28.6 94.3 21.5 √ 0.739 11.2 28.6 112.3 21.5 √ 0.721 10.0 26.4 103.0 19.4 √ 0.720 7.4 25.0 86.6 14.3 √ √ 0.733 10.0 26.4 113.8 19.4 √ √ 0.725 7.4 25.0 98.1 14.3 √ √ 0.733 6.3 23 98.1 12.2 √ √ √ 0.737 6.3 23 101.8 12.2 Table 1 Ablation Experiment As shown in the table above, replacing the loss function with PIOU improved mAP50 by 2% and FPS by 18; adding Adown downsampling improved mAP50 by 0.2%, reduced the number of parameters by 1.2M, reduced FLOPS by 2.2B, improved FPS by 8.7, and reduced size by 2.1M; replacing Neak with BiFPN instead of FPN improved mAP50 by 0.1%, reduced the number of parameters by 3.8M, reduced FLOPS by 3.6B, and reduced size by 7.2M; combining the three innovations improved mAP50 by 1.8%, reduced the number of parameters by 4.9M, reduced FLOPS by 5.6B, improved FPS by 7.5, and reduced size by 9.3M. The data results show that each improvement point of this invention helps to improve the model, and the combination of the three is even better.

[0043] Table 2 shows a comparison of the detection results of this invention and YOLOv8s: YOLOv8s 0.719 11.2 28.6 94.3 21.5 Ours 0.737 6.3 23 101.8 12.2 Table 2 Comparative Experiments As can be seen from the table above, after making improvements to YOLOv8s, this invention achieves a 1.8% higher mAP50, a 43.8% lower Param, a 19.6% higher FLOPS, a 7.5 higher FPS, and a 9.3M smaller size than YOLOv8s. In other words, it has significantly improved performance in terms of detection accuracy, detection speed, and model size compared to YOLOv8.

[0044] Table 3 shows a comparison of the mAP@0.5% detection results for the twelve categories before and after the network structure improvement: All 0.719 0.737 Bicycle 0.808 0.837 Boat 0.742 0.721 Bottle 0.578 0.560 Bus 0.919 0.938 Car 0.777 0.808 Cat 0.694 0.757 Chair 0.642 0.636 Cup 0.642 0.684 Dog 0.751 0.774 Motorbike 0.776 0.802 People 0.736 0.751 Table 0.565 0.581 Table 3 Comparison of Categories As shown in the table above, when detecting these twelve categories of objects under low light conditions, the accuracy (AP) of nine categories has been improved. Specifically, Bicycle improved by 2.9%, Bus by 1.9%, Car by 3.1%, Cat by 6.3%, Cup by 4.2%, Dog by 2.3%, Motorbike by 2.6%, People by 1.5%, and Table by 1.6%. This invention effectively improves the detection accuracy of target objects under low light conditions.

[0045] The data in the three tables from the comparative and ablation experiments above show that the model has improved accuracy, speed, and AP for each category, while reducing the number of model parameters. This embodiment effectively improves model accuracy and reduces model size.

[0046] Figure 5 and Figure 6 The results of this embodiment and the original YOLOv8 on some low-light image detections in the Exdark dataset are presented. Figure 5 river Figure 6 The results of this embodiment and the original YOLOv8 on some low-light image detections in the Exdark dataset are presented. Figure 5 This is the target detection performance of the original YOLOv8 algorithm. Figure 6 This embodiment demonstrates the target detection performance. A comparison of the two shows that the original YOLOv8 may fail to detect occluded targets or small targets, or may mistake them for other objects. Furthermore, its detection accuracy is low for all targets. This embodiment effectively reduces the probability of missed and false detections and improves detection accuracy. Moreover, this embodiment significantly improves detection speed and reduces model size.

[0047] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered by the present invention.

Claims

1. A lightweight low-light target detection method, characterized in that: The method includes the following steps: Step 1: Obtain the publicly available sample data Exdark dataset, convert its labels to YOLO training format, and divide it into training, validation, and test sets; Step 2: Optimize the YOLOv8 object detection algorithm to build a lightweight low-light object detection model; the specific operation method is as follows: Step 2.1: Replace layers 4, 6, and 8 of the YOLOv8 Backbone feature extraction network with Adown downsampling; the improved YOLOv8 backbone network includes the following sequentially connected modules: first convolutional module, second convolutional module, first C2f module, first Adown downsampling module, second C2f module, second Adown downsampling module, third C2f module, third Adown downsampling module, fourth C2f module, and SPPF spatial pyramid pooling structure. Step 2.2: Replace Neck's FPN feature pyramid with BiFPN, and make skip connections between the 10th and 20th layers of the model, as well as between the 11th and 23rd layers; Step 2.3: Replace the bounding box regression loss function in YOLOv8 from CIoU to Powerful-IoU (PIoU); Step 3: Train the optimized low-light target detection model based on the training set divided in Step 1 to obtain the optimal low-light target detection model; Step 4: Use the trained low-light target detection model and the image to be detected to detect the image under low light conditions and output the target detection results.

2. The lightweight low-light target detection method according to claim 1, characterized in that: The Exdark dataset contains 7,363 low-light images under 10 different conditions, ranging from very low-light environments to dusk, as well as bounding boxes of 12 target classes at the image class level and with local annotations.

3. The lightweight low-light target detection method according to claim 1, characterized in that: In step 3, when training the optimized low-light target detection model based on the training set divided in step 1, the input image size is set to 640*640, the batch size is 16, the number of training iterations is 300, the initial learning rate is 0.01, the learning rate momentum is 0.937, the weight decay coefficient is 0.0005, the optimizer is SGD, the random seed is fixed during the experiment, and the pre-trained weights are not used in the entire process. After training is complete, the model will save the best weight file, best.pt, based on the best training results.

4. The lightweight low-light target detection method according to claim 3, characterized in that: The step 4 describes using the trained low-light target detection model and the image to be detected to detect images under low light conditions. This involves using best.pt as the weight file for detecting low-light images and the image to be detected as the data source for detection.

Images