Lightweight small target detection method based on improved YOLOv8

Abstract

The invention discloses a lightweight small target detection method based on improved YOLOv8. The method comprises the following specific steps: constructing a small target data set; building an Improved-YOLO small target detection model on the basis of a YOLOv8 framework; training an Improved-YOLO small target detection model by using the data set, and performing model quantization operation in the training process to obtain a lightweight detection model; reasoning acceleration of the lightweight detection model is carried out, and the model is embedded into an edge computing terminal for deployment; a camera is used for collecting video data, and the collected data are input into an edge computing terminal for processing; and outputting the processed video data, and displaying the processed video data on a client in real time. According to the method, efficient training and quantification of the model are realized, the model is successfully deployed in the edge computing terminal more innovatively, and the problems of poor real-time performance, limited computing resources and the like of a traditional target detection method in an edge environment are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image detection, and in particular relates to a lightweight small target detection method based on improved YOLOv8. Background Art

[0002] In recent years, with the rapid development of deep learning, one-stage object detection algorithms, such as YOLO and SSD, have become the mainstay of the field of object detection. The YOLOv8 algorithm has significantly improved the real-time performance and prediction accuracy of object detection through multiple optimization iterations. However, current advanced algorithms mainly focus on processing larger objects, because these large-scale objects carry richer information and the relevant data sets are more extensive, making detection more convenient. Despite significant progress in large objects, small object detection is still an important problem that needs to be solved. In small object detection tasks, lightweight models can run more efficiently, ensuring high real-time and accuracy. It helps to optimize system performance, especially for small object detection tasks in resource-limited environments, thereby improving the efficiency of the overall system.

[0003] The definition of small targets involves objects with pixel values ​​less than 32×32 or the square root of the ratio of the length and width pixel product to the length and width pixel product of the entire image is less than 3%. Such targets occupy a limited proportion of the image, have low resolution, and are often accompanied by complex background information. The importance of small targets lies in the fact that they may contain key information, especially in the fields of security monitoring and intelligent transportation. The fast and accurate detection of small targets is crucial to the overall performance of the system. As edge computing service technology has become a hot topic in the application field, traditional video detection or monitoring systems have some shortcomings, such as challenges in small target detection, resulting in poor real-time performance and high cost. Summary of the invention

[0004] In order to make up for the shortcomings of the prior art, the present invention aims to provide a lightweight small target detection method based on improved YOLOv8 to detect small targets in images in real time and improve their detection accuracy. To achieve this purpose, the specific technical solutions of this application are as follows: The lightweight small target detection method based on improved YOLOv8 has the following steps: S1. Construct a small target dataset; S2. Build an Improved-YOLO small target detection model based on the YOLOv8 framework; S3. Use the data set in step S1 to train the Improved-YOLO small target detection model, perform model quantization during the training process, and obtain a lightweight detection model; S4. Accelerate the reasoning of lightweight detection models and embed them into edge computing terminal deployment; S5. Use the camera to collect video data and input the collected data into the edge computing terminal for processing; S6. Output the processed video data and display it in real time on the client.

[0005] Furthermore, in step S1, the small target refers to a target smaller than 32×32 pixels. In actual application scenarios, when the square root of the ratio of the product of the length and width pixels of the annotation box of an object in the image to the product of the length and width pixels of the entire image is less than 3%, the object is defined as a small target.

[0006] Furthermore, the Improved-YOLO small target detection model in step S2 includes four parts: Input, Backbone, Neck, and Head, wherein the Input part is used to input the small target image to be detected, the Backbone part is used to extract the backbone features of the small target image to be detected, and the Head part is used to predict and classify the small target image to be detected. During the construction process, the original C2f module in the Backbone part is changed to the C2f_ContextGuided module, only part of the feature enhancement network is retained in the Neck part, only a small target detection head is retained in the Head part, and the standard convolution therein is replaced with a partial convolution.

[0007] Furthermore, the C2f_ContextGuided (cross-stage context-guided feature fusion) module extracts surrounding and local features from the target area respectively through DConv (dilated convolution) and Conv (standard convolution) in the ContextGuided (context-guided feature fusion) module, and the two are merged after BN (batch normalization) and PReLU (ParametricRectified linear unit) activation to form a preliminary joint feature, and then the above joint feature is dynamically adjusted and weighted.

[0008] Furthermore, during the construction of the Improved-YOLO small target detection model, the standard convolution for feature fusion in the Head part is changed to partial convolution. Partial convolution is to select part of the channels of ordinary convolution for convolution, which is used to balance the model detection speed and accuracy; and NWD Loss (normalized Wasserstein distance loss) and IoU Loss (intersection over union loss) are used as the loss function of the bounding box coordinates. At the same time, Focal Loss is introduced to optimize the sample imbalance problem in the bounding box regression task, and alleviate the problem of missed detection of small targets in dense and scene conditions, namely: , , Among them, N A and N B Represents two normal distributions, represented by the location A=(cx A ,cy A , w A ,h A ) and B = (cx B ,cy B , w B ,h B ) and scale parameter definition; cx A 、cy A N A The center coordinates, w A and h A N A The width and height of cx B 、cy B N B The center coordinates, w B and h B N B The width and height of N A and N B The square of the Wasserstein distance between two distributions measures their similarity by calculating the Euclidean distance between the location and scale parameters; C is a parameter related to the data set, used to normalize the Wasserstein distance; T represents the transpose symbol of the matrix, NMD (N A , N B ) indicates N A and N B The normalized wasserstein distance between two distributions is processed by an exponential function to obtain a similarity between 0 and 1, which is used to measure N A and N B The similarity.

[0009] Furthermore, in step S3, during the model quantization process, the original data set and the trained model need to be input into the middle layer together to be converted into the middle layer model, and the middle layer model will be continuously quantized and calibrated until it meets the accuracy requirements.

[0010] Furthermore, in step S4, the lightweight detection model is converted into ONNX format and then input into the TensorRT framework for inference acceleration processing, specifically including tensor fusion and low-precision / integer quantization of the network layer, wherein the tensor fusion of the network layer includes: ① horizontal merging, that is, the Convolution layer, Bias layer and ReLU layer with the same parameters in the model are combined into a CBR layer, ② vertical merging, that is, the layers with the same model network structure but different parameters are superimposed and combined into a CBR layer; in the process of inference acceleration, FP16 and INT8 precision tensors are used to replace FP32 tensors.

[0011] Furthermore, in step S4, in the model deployment stage, the lightweight Improved-YOLO small target detection model that has been trained and quantized is first deeply optimized to adapt it to the edge computing environment with limited resources; the quantized model is finally converted into bmodel on the x86 side, and then deployed on the edge computing terminal, and the model calculation amount and memory usage are reduced through algorithm-level pruning, quantization and tensor fusion technology.

[0012] Compared with the prior art, the present invention has the following advantages: the method of the present invention not only realizes the efficient training and quantization of the model, but also innovatively deploys it successfully on the edge computing terminal, solving the problems of poor real-time performance and limited computing resources of traditional target detection methods in edge environments; it also greatly broadens the application scope of small target detection technology in the Internet of Things, mobile devices and various low-power scenarios, and provides strong support for the real-time and accurate detection of small targets in the fields of security monitoring, intelligent transportation, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a flow chart of the method of the present invention; Figure 2 This is a flow chart of the present invention at the model deployment stage. DETAILED DESCRIPTION

[0014] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the present invention is further described below in conjunction with the accompanying drawings.

[0015] like Figure 1 As shown in FIG. 1 , a lightweight small target detection method based on improved YOLOv8 includes the following steps: (1) Construct a small target dataset.

[0016] In the field of object detection, a small target is defined as a target that is smaller than 32×32 pixels. In actual application scenarios, when the square root of the ratio of the length and width pixel product of the object's annotation box in the image to the length and width pixel product of the entire image is less than 3%, the object is defined as a small target.

[0017] In order to expand the number of samples contained in the data set, data augmentation technology is used to expand the data set. The specific operations include: random rotation, horizontal translation, vertical translation, scaling, and horizontal inversion. The expanded data set is then randomly divided into a training set and a test set in a ratio of 7:3.

[0018] (2) Build the Improved-YOLO small target detection model based on the YOLOv8 framework.

[0019] The Improved-YOLO small target detection model consists of four parts: Input, Backbone, Neck, and Head. The Input part is used to input the small target image to be detected; the Backbone part is used to extract the backbone features of the small target image to be detected; the Head part is used to predict and classify the small target image to be detected. During the construction process, the original C2f module in the Backbone part is changed to the C2f_ContextGuided (cross-stage context-guided feature fusion) module, only part of the feature enhancement network is retained in the Neck part, only a small target detection head is retained in the Head part, and the standard convolution is replaced with partial convolution.

[0020] Backbone uses CSPDarknet (cross-stage partial network) to extract feature information from infrared images by layered convolution to generate the first, second and third feature layers. CSPDarknet in YOLOv8s is used as the backbone network for feature extraction, where CSPDarknet consists of 5 CBS modules (for performing convolution, normalization and activation operations), 4 C2f (convolution to feature) modules and 1 SPPF (fast spatial pyramid pooling) module. The CBS module retains the feature map and its information by performing 1×1 convolution, batch normalization (BN) and SiLU (Sigmoid linear unit) activation functions in sequence. The 4 C2f modules are responsible for deep feature extraction, converting the input infrared image into features of different levels and abstraction levels through two convolutional layers. The SPPF module can rescale the input image of any size to a fixed size and generate a fixed-length feature vector to reduce the amount of calculation. At the same time, it extracts and encodes infrared image features at multiple scales to ensure that the most significant key information in the image is extracted.

[0021] After CBS processing of the infrared image, the first CBS result is obtained, and then the first C2f module is used to extract features to generate primary shallow features; the specific process is: CBS processing is performed again on the first CBS result, and features are extracted through the first C2f module to obtain shallow features, and feature mapping and channel expansion are preliminarily realized, and the final number of channels is 64. The shallow features are further processed by the second C2f module to obtain the first feature, and stored in the first feature storage module; the operation flow of the second C2f module is: CBS processing is performed on the shallow features, and then features are extracted through this module, and the first feature with a channel number of 128 is output. Next, the features in the first feature storage module are deeply extracted through the third C2f module to obtain the second feature, and it is saved in the second feature storage module. This process generates a second feature with a channel number of 256 by performing CBS processing on the stored first feature and then extracting it through the third C2f module. The second feature is further extracted through the fourth C2f module to obtain the third feature. Specifically, the fourth C2f module first performs CBS processing on the second feature, and then further extracts it through the SPPF module to generate a second deep feature with a channel number of 512 and a third feature, and the number of channels of the two is consistent. Subsequently, the third feature is upsampled, fused with the second feature, and the fourth fused feature is generated and stored. The fourth fused feature is then upsampled and fused with the first feature to generate the first fused feature and save it. After the stored first fused feature is processed by CBS, it is fused with the fourth fused feature for the second time to generate the second fused feature and store it. After that, the second fused feature is processed by CBS and fused with the third feature to generate the third fused feature.

[0022] It can be understood that the neck network of YOLOv8s consists of upsampling, downsampling and 4 C2f modules. This application improves its neck network by replacing the 4 C2f modules with 4 fusion modules (the first fusion module, the second fusion module, the third fusion module and the fourth fusion module) to form a new neck network structure. CSPDarknet is used as the backbone network. The first, second and third features output are multi-scale fused in the new neck network, combining shallow textures, edge details and deep semantic information (such as shape and size) to improve the feature representation and generalization capabilities of the model. The four fusion modules use the same CSP structure, which avoids the frequent slicing and splicing of feature tensors in the C2f module during feature processing, which is suitable for low-computing power devices. At the same time, more feature information is retained to achieve more accurate and efficient feature fusion, and ultimately enhance the feature quality of the model. This improvement also continues the lightweight design and reduces the number of parameters and computational complexity of the model.

[0023] The C2f_ContextGuided (cross-stage context-guided feature fusion) module extracts surrounding and local features from the target area through the DConv (dilated convolution) and Conv (standard convolution) modules in the ContextGuided (context-guided feature fusion) modules respectively. The two are combined after BN (batch normalization) and PReLU (Parametric Rectified Linear Unit) activation to form a preliminary joint feature, which is then dynamically adjusted and weighted.

[0024] During the construction of the Improved-YOLO small target detection model, the standard convolution for feature fusion in the Head part is changed to partial convolution. Partial convolution is to select part of the channels of ordinary convolution for convolution, which is used to balance the model detection speed and accuracy. NWD Loss (normalized Wasserstein distance loss) and IoU Loss (intersection over union loss) are used as the loss function of the bounding box coordinates. At the same time, Focal Loss is introduced to optimize the sample imbalance problem in the bounding box regression task, and alleviate the problem of missed detection in small target detection in dense and scenes, that is: , , Among them, N A and N B Represents two normal distributions, represented by the location A=(cx A ,cy A , w A ,h A ) and B = (cx B ,cy B , w B ,h B ) and scale parameter definition; cx A 、cy A N A The center coordinates, w A and h A N A The width and height of cx B 、cy B N B The center coordinates, w B and h B N B The width and height of N A and N BThe square of the Wasserstein distance between two distributions measures their similarity by calculating the Euclidean distance between the location and scale parameters; C is a parameter related to the data set, used to normalize the Wasserstein distance; T represents the transpose symbol of the matrix, NMD (N A , N B ) indicates N A and N B The normalized wasserstein distance between two distributions is processed by an exponential function to obtain a similarity between 0 and 1, which is used to measure N A and N B The similarity.

[0025] (3) Use the data set in step 1 to train the Improved-YOLO small target detection model. During the training process, perform model quantization to obtain a lightweight detection model. Figure 2 As shown in the figure, during the model quantization process, the original data set and the trained model need to be input into the middle layer together to be converted into the middle layer model. The middle layer model will be continuously quantized and calibrated until it meets the accuracy requirements.

[0026] (4) Accelerate the reasoning of lightweight detection models and embed them into edge computing terminal deployment.

[0027] The lightweight detection model is converted into ONNX format and then input into the TensorRT framework for inference acceleration processing, including tensor fusion and low-precision / integer quantization of the network layer. The tensor fusion of the network layer includes: ① horizontal merging, that is, the Convolution layer, Bias layer and ReLU layer with the same parameters in the model are combined into a CBR layer; ② vertical merging, that is, the layers with the same model network structure but different parameters are stacked and combined into a CBR layer; in the process of inference acceleration, FP16 and INT8 precision tensors are used to replace FP32 tensors.

[0028] In the model deployment phase, the lightweight Improved-YOLO small target detection model that has been trained and quantized is first deeply optimized to adapt it to the resource-limited edge computing environment. The quantized model is finally converted into bmodel on the x86 side and then deployed on the edge computing terminal. Through algorithm-level pruning, quantization, and tensor fusion technology, the model calculation amount and memory usage are reduced while maintaining high detection accuracy.

[0029] Specifically, during the model conversion process, the quantized model file is converted into a specific format suitable for edge devices (such as bmodel). This format is not only easy to load and execute in embedded systems, but also can effectively utilize hardware acceleration functions such as neural network processors (NPUs), GPUs or TPUs, thereby significantly improving the reasoning speed.

[0030] In view of the characteristics of edge computing terminals, the present invention designs a complete end-to-end deployment solution. First, ensure the compatibility of lightweight models under different architectures and operating systems; secondly, by optimizing the data stream processing process, reduce the delay of video stream transmission, and combine efficient frame buffering and preprocessing technology to ensure seamless connection between input data and model reasoning. At the same time, in order to improve the energy efficiency of edge computing terminals and extend battery life, the edge deployment solution in the embodiment of the present invention also includes an intelligent power management strategy, which dynamically adjusts the allocation of computing resources according to the actual load to ensure that energy consumption is reduced as much as possible while meeting real-time detection requirements. In addition, when deploying the lightweight Improved-YOLO small target detection model on the edge computing terminal, the stability and reliability of the system are also taken into account. Through the anomaly detection mechanism and fault recovery strategy, when errors occur in the detection process or system resources are tight, countermeasures can be taken in time to ensure the continuous and stable operation of the overall system.

[0031] (5) Use the camera to collect video data and input the collected data into the edge computing terminal for processing.

[0032] (6) The processed video data is output and displayed in real time on the client.

[0033] In summary, the present invention provides a lightweight small target detection method based on improved YOLOv8, which not only realizes the efficient training and quantization of the model, but also innovatively deploys it successfully on the edge computing terminal, solving the problems of poor real-time performance and limited computing resources in traditional target detection methods in edge environments. It greatly broadens the application scope of small target detection technology in the Internet of Things, mobile devices and various low-power scenarios, and provides strong support for the real-time and accurate detection of small targets in the fields of security monitoring, intelligent transportation, etc.

[0034] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight small target detection method based on improved YOLOv8, characterized in that: The steps of the method are as follows: S1. Construct a small target dataset; S2. Build an Improved-YOLO small target detection model based on the YOLOv8 framework; S3. Use the data set in step S1 to train the Improved-YOLO small target detection model, perform model quantization during the training process, and obtain a lightweight detection model; S4. Accelerate the reasoning of lightweight detection models and embed them into edge computing terminal deployment; S5. Use the camera to collect video data and input the collected data into the edge computing terminal for processing; S6. Output the processed video data and display it in real time on the client.

2. A lightweight small target detection method based on improved YOLOv8 according to claim 1, characterized in that: In step S1, a small target refers to a target that is smaller than 32×32 pixels. In actual application scenarios, when the square root of the ratio of the length and width pixel product of the object's annotation box size in the image to the length and width pixel product of the entire image is less than 3%, the object is defined as a small target.

3. A lightweight small target detection method based on improved YOLOv8 according to claim 1, characterized in that: The Improved-YOLO small target detection model in step S2 includes four parts: Input, Backbone, Neck, and Head. The Input part is used to input the small target image to be detected, the Backbone part is used to extract the backbone features of the small target image to be detected, and the Head part is used to predict and classify the small target image to be detected. During the construction process, the original C2f module in the Backbone part is changed to the C2f_ContextGuided module, only part of the feature enhancement network is retained in the Neck part, only a small target detection head is retained in the Head part, and the standard convolution therein is replaced with a partial convolution.

4. A lightweight small target detection method based on improved YOLOv8 according to claim 3, characterized in that: The C2f_ContextGuided module extracts surrounding and local features from the target area through DConv and Conv in ContextGuided respectively, and the two are combined after BN and PReLU activation to form a preliminary joint feature, which is then dynamically adjusted and weighted.

5. A lightweight small target detection method based on improved YOLOv8 according to claim 4, characterized in that: During the construction of the Improved-YOLO small target detection model, the standard convolution for feature fusion in the Head part is changed to partial convolution. Partial convolution is to select part of the channels of ordinary convolution for convolution, which is used to balance the model detection speed and accuracy; and NWD Loss and IoU Loss are used as the loss function of the bounding box coordinates. At the same time, Focal Loss is introduced to optimize the sample imbalance problem in the bounding box regression task, and alleviate the problem of missed detection of small targets in dense and scenes, that is: ， ， Among them, N A and N B Represents two normal distributions, represented by the location A=(cx A ,cy A , w A ,h A ) and B = (cx B ,cy B , w B ,h B ) and scale parameter definition; cx A 、cy A N A The center coordinates, w A and h A N A The width and height of cx B 、cy B N B The center coordinates, w B and h B N B The width and height of N A and N B The square of the Wasserstein distance between two distributions measures their similarity by calculating the Euclidean distance between the location and scale parameters; C is a parameter related to the data set, used to normalize the Wasserstein distance; T represents the transpose symbol of the matrix, NMD (N A , N B ) indicates N A and N B The normalized wasserstein distance between two distributions is processed by an exponential function to obtain a similarity between 0 and 1, which is used to measure N A and N B The similarity.

6. A lightweight small target detection method based on improved YOLOv8 according to claim 4, characterized in that: In step S3, during the model quantization process, the original data set and the trained model need to be input into the middle layer to be converted into the middle layer model. The middle layer model will be continuously quantized and calibrated until it meets the accuracy requirements.

7. A lightweight small target detection method based on improved YOLOv8 according to claim 1, characterized in that: In the step S4, the lightweight detection model is converted into ONNX format and then input into the TensorRT framework for inference acceleration processing, specifically including tensor fusion and low-precision / integer quantization of the network layer, wherein the tensor fusion of the network layer includes: ① horizontal merging, that is, combining the Convolution layer, Bias layer and ReLU layer with the same parameters in the model into a CBR layer, ② vertical merging, that is, stacking and combining the layers with the same model network structure but different parameters into a CBR layer; in the process of inference acceleration, the FP32 tensor is replaced by the tensor with FP16 and INT8 precision.

8. A lightweight small target detection method based on improved YOLOv8 according to claim 7, characterized in that: In step S4, in the model deployment stage, the lightweight Improved-YOLO small target detection model that has been trained and quantized is first deeply optimized to adapt it to the edge computing environment with limited resources; the quantized model is finally converted into bmodel on the x86 side, and then deployed on the edge computing terminal, and the model calculation amount and memory usage are reduced through pruning, quantization and tensor fusion technology at the algorithm level.

Images