Target detection method based on YOLOv8 improved model

Abstract

The invention discloses a target detection method based on a YOLOv8 improved model, and the method comprises the steps: constructing and training a target detection network model based on YOLOv8, and the target detection network model comprises a backbone network, a neck network and a task alignment detection head; inputting a to-be-detected image into the trained target detection network model, and obtaining three-scale feature maps through a backbone network and a neck network; and finally, respectively inputting the feature maps of the three scales into respective task alignment detection heads to obtain a final detection result. According to the method, the task is adopted to align the detection head, the neck network structure is improved, the fusion capability of different scale features is remarkably enhanced, the target features in a small target and a complex scene can be better captured, and the detection precision is improved.

Description

Technical Field

[0001] The present application belongs to the field of target detection technology, and in particular, relates to a target detection method based on an improved YOLOv8 model. Background Art

[0002] With the development of automobile assisted driving systems, automobile safety assisted driving systems based on machine vision are becoming more and more important in modern transportation. These systems can capture the vehicle's surrounding environment in real time, identify potential dangers through image processing technology, and improve driving safety. However, in complex traffic scenes, target detection faces challenges such as complex background, occlusion, and blur. At the same time, existing deep learning models usually have a large number of parameters, resulting in long detection time, making it difficult to achieve efficient real-time detection on embedded devices.

[0003] At present, the mainstream target detection algorithms are divided into two categories: single-stage detection algorithms represented by YOLO (You Only Look Once) and SSD (Single Shot Multi-Box Detector-CNN) series and two-stage detection algorithms represented by R-CNN (Region-CNN) series. Although the two-stage target detection algorithm can achieve high detection accuracy, its real-time performance is poor and cannot meet actual needs. However, many parameters of the SSD algorithm in the stage need to be set manually, and the debugging process is very dependent on the experience of researchers. The image resolution has a great impact on the network output results, and there are certain limitations in practical applications. The regression-based detection method YOLO does not need to generate regions, but directly regresses the bounding box of targets with high confidence, and classifies the targets in the box. YOLOv8 is one of the latest versions in the YOLO series, which has significantly improved accuracy and detection speed. However, as a single-stage model, YOLOv8 has low parameter volume and computational complexity, so there is some loss in accuracy. Summary of the invention

[0004] The purpose of this application is to provide a target detection method based on the improved YOLOv8 model to overcome the problem of low detection accuracy of the prior art under complex road conditions, and to improve the detection accuracy without significantly increasing the amount of calculation.

[0005] In order to achieve the above purpose, the technical solution of this application is as follows: A target detection method based on an improved YOLOv8 model, comprising: Build and train a YOLOv8-based target detection network model, which includes a backbone network, a neck network, and a task alignment detection head; The image to be detected is input into the trained object detection network model, and feature maps of three scales are obtained through the backbone network and the neck network; The feature maps of the three scales are input into their respective task alignment detection heads to obtain the final detection results; Among them, the task alignment detection head performs the following operations: Perform one and two convolution operations on the input feature map respectively, and then concatenate them to obtain the interactive features; Convolution operations are performed on the interaction features to generate masks and offsets; The interactive features are passed through a task decomposition module to obtain the positioning branch features; The interactive features are passed through another task decomposition module to obtain classification branch features; Perform convolution operations on the interactive features to obtain dynamic weights; Perform dynamic convolution operation on the mask and offset with the positioning branch features to obtain dynamic features; The classification branch features are fused with the dynamic weights to obtain fused features; Perform localization convolution and classification convolution operations on dynamic features and fusion features respectively to obtain localization output and classification output; The positioning output and classification output are scaled and spliced ​​through the scale processing module to obtain the detection result.

[0006] Furthermore, the task decomposition module performs the following operations: Perform global average pooling on the interaction features to obtain the average features; The average features are converted into attention weights through convolution operations; Fuse the attention weight with the base weight of the convolution kernel to obtain the fused weight; The fusion weights are convolved with the interaction features to obtain the output features of the task decomposition module.

[0007] Furthermore, the neck network performs the following operations: The feature maps of three scales extracted by the backbone network are used as the first scale features, second scale features and third scale features input to the neck network; The first scale feature, the second scale feature and the third scale feature are input into the created multi-scale context aggregation module for the first fusion to obtain enhanced features; The enhanced features are concatenated with the third-scale features after the convolution operation, and then passed through the feature conversion module to obtain the converted third-scale features; The enhanced features are upsampled and concatenated with the first-scale features, and then passed through a feature conversion module to obtain the converted first-scale features; The enhanced features, the converted first-scale features, and the converted third-scale features are input into the created multi-scale context aggregation module for a second fusion; After the second fusion, the features are convolved to obtain the second-scale feature map output by the neck network; The features after the second fusion are convolved and then concatenated with the converted third-scale features to obtain the third-scale feature map output by the neck network; The features after the second fusion are upsampled and concatenated with the first-scale features after conversion to obtain the first-scale feature map output by the neck network.

[0008] Furthermore, the multi-scale context aggregation module performs the following operations: Down-sample the input first-scale feature map; perform convolution operation on the input second-scale feature map; up-sample and convolution operation on the input third-scale feature map; then concatenate to obtain the first feature; The first feature performs four depth convolutions with different convolution kernels respectively, and the four features after the depth convolution are added together with the first feature to obtain the second feature; After convolution, the second feature is added to the first feature to obtain the output feature of the multi-scale context aggregation module.

[0009] This application proposes a target detection method based on the improved yolov8 model, which introduces a target detection method based on computer vision and deep learning for embedded devices. It can improve the detection accuracy of current vehicle-mounted assisted driving equipment for pedestrians, vehicles and cyclists in complex road conditions, and this single-stage detection algorithm is more suitable for embedded devices with limited computing power and limited resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a flow chart of the target detection method based on the improved YOLOv8 model in this application.

[0011] Figure 2 This is a schematic diagram of the structure of the task alignment detection head in an embodiment of the present application.

[0012] Figure 3 This is a schematic diagram of the neck network structure in an embodiment of the present application.

[0013] Figure 4 This is a schematic diagram of the structure of a multi-scale context aggregation module in an embodiment of the present application. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0015] An embodiment of the present application, such as Figure 1 As shown, a target detection method based on the improved YOLOv8 model is provided, including: Step S1: construct and train a target detection network model based on YOLOv8, wherein the target detection network model includes a backbone network, a neck network, and a task alignment detection head.

[0016] The existing basic YOLOv8 model is as follows Figure 2 As shown, it includes a backbone network (backbone), a neck network (neck) and a detection head (head). This embodiment improves the detection head. The detection head of the existing basic YOLOv8 model usually uses independent classification and positioning branches, which leads to a lack of interaction between the two tasks. The improved detection head is called the Task-aligned Detection Head (TADH). The TADH detection head learns task interaction features from multiple convolutional layers through a feature extractor to obtain joint features. The positioning branch uses DCNV2 and interactive features to generate offsets and masks, and the classification branch uses interactive features for dynamic feature selection.

[0017] The improved task alignment detection head of this embodiment will be described in detail in step S3.

[0018] Step S2: Input the image to be detected into the trained target detection network model, and obtain feature maps of three scales through the backbone network and the neck network.

[0019] The backbone network and neck network of this embodiment are consistent with the backbone network and neck network in the existing basic YOLOv8 model, and will not be repeated here.

[0020] For example, the outputs of the 4th, 6th, and 9th layers in the YOLOv8 backbone network represent feature maps of 80*80, 40*40, and 20*20, respectively, which are feature maps of three scales output by the backbone network. These three feature maps are input to the neck network, and the neck network outputs the corresponding feature maps of three scales, which are input to the subsequent detection head.

[0021] Step S3: Input the feature maps of the three scales into their respective task alignment detection heads to obtain the final detection results.

[0022] Specifically, this application sets three task alignment detection heads for the feature maps of three scales output by the neck network, and each task alignment detection head performs the following operations: Step 301: Perform one convolution and two convolution operations on the input feature map respectively, and then concatenate them to obtain interactive features.

[0023] For example, the input feature maps are feature maps from different network layers of the neck network , , , these feature maps represent target information at different scales: in For each input feature map , undergo one and two 3×3 convolution operations respectively, and are standardized using Group Normalization (GN).

[0024] The specific steps are as follows: Each input feature map After a 3×3 convolution operation and normalization using GN, the feature : At the same time, the feature map Will do it again for the second time Convolution and GN normalization to obtain features : Then, the feature map after two convolutions and the feature map after one convolution are concatenated to obtain the interactive feature : .

[0025] Step 302: Perform a convolution operation on the interaction features to generate a mask and an offset.

[0026] This step inputs the interactive features into a convolutional layer to generate the mask and offset required for dynamic convolution: in represents the generated mask, Indicates the offset.

[0027] Step 303: The interactive features are passed through a task decomposition module to obtain positioning branch features.

[0028] This step uses a task decomposition module to generate positioning branch features , the formula is as follows: in, Represents the operation of the task decomposition module.

[0029] Step 304: The interactive features are passed through another task decomposition module to obtain classification branch features.

[0030] This step uses a task decomposition module to generate positioning branch features , the formula is as follows: in, Represents the operation of the task decomposition module.

[0031] In order to enhance the collaborative interaction between classification and positioning tasks, this application introduces the idea of ​​TAP (Task-Aligned Predictor) in the task decomposition module, so that the module can dynamically adapt to the needs of the task, thereby improving the detection accuracy. The task decomposition module performs the following operations:

[0032] Step F1: Perform global average pooling on the interaction features to obtain average features.

[0033] For example, for the interaction feature Perform global average pooling to obtain average features : GAP stands for global average pooling, which is used to extract the global information of the feature map.

[0034] It should be noted that there are two task decomposition modules, which perform the same operation respectively and obtain the positioning branch features and classification branch features respectively through different network parameters after learning and training.

[0035] Step F2: Convert the average features into attention weights through convolution operation.

[0036] First go through Convolution Conv1 converts the global feature map The number of channels is reduced to 1 / 8 of the original: Then, through a Convolution Conv2 will Converted to attention weight : .

[0037] Step F3: Fuse the attention weight with the basic weight of the convolution kernel to obtain the fused weight.

[0038] In order to achieve dynamic adjustment of features, this embodiment fuses the generated attention weights with the basic weights of the convolution kernel. Given the basic convolution kernel weights , the fusion process is as follows: .

[0039] Step F4: Perform convolution operation on the fusion weight and the interaction feature to obtain the output feature of the task decomposition module.

[0040] This step will fuse the weights Interaction feature map Perform convolution operations to obtain task-specific output feature maps After processing by Group Normalization (GN) and activation function, the final feature output is obtained:

[0041] Among them, the SiLU activation function is used.

[0042] Step 305: Perform a convolution operation on the interactive features to obtain dynamic weights.

[0043] Specifically, the interactive features are convolved to obtain dynamic weights : , Among them, ReLU activation function and Sigmoid activation function are used.

[0044] Step 306: Perform a dynamic convolution operation on the mask and the offset with the positioning branch feature to obtain a dynamic feature.

[0045] This step is to locate the features of the branch , combined with the generated mask and offset , processed by the dynamic convolution module to obtain dynamic features : , .

[0046] Step 307: Fuse the classification branch features with the dynamic weights to obtain fused features.

[0047] This step is for classification branch features , combined with the generated weight features , element-by-element multiplication realizes the fusion of features and obtains the fusion feature : .

[0048] Step 308: Perform positioning convolution and classification convolution operations on the dynamic features and the fusion features respectively to obtain positioning output and classification output.

[0049] In this step, the dynamic features and fusion features are respectively subjected to positioning convolution and classification convolution operations to obtain positioning output and classification output: Positioning convolution: dynamic features Perform positioning convolution to obtain positioning output : Classification convolution: Fusion features Perform classification convolution to obtain classification output : .

[0050] This embodiment introduces the TAP (Task-Aligned Predictor) module in the Task-aligned One-stage Object Detection into the task decomposition module to achieve interaction between classification and positioning tasks, and solve the problem of independent processing of classification and positioning tasks in the YOLOv8 detection head and lack of collaboration. The positioning branch uses dynamic convolution and interactive features to generate offsets and masks to improve the positioning accuracy of the target frame; the classification branch further improves the accuracy of classification by dynamically selecting features.

[0051] Step 309: scaling the positioning output and the classification output through the scale processing module, and splicing them to obtain the detection result.

[0052] Finally, the scale processing module performs scaling processing and splices the scaled positioning output with the classification output to form the final detection result.

[0053] It should be noted that in YOLOv8, the model uses three detection heads for multi-scale detection to ensure good detection results for objects of different sizes. The first detection head (large feature map) is responsible for detecting small objects. The second detection head (medium feature map) is responsible for detecting medium-sized objects. The third detection head (small feature map) is responsible for detecting large objects, and it receives feature map inputs with lower resolution. Each detection head outputs the prediction results of the bounding box and category at the corresponding scale. The three detection heads make independent predictions, and the final output is a combination of the prediction results from feature maps of different scales. The prediction results output by all detection heads are fused in the post-processing stage of the network, including non-maximum suppression (NMS) to remove redundant boxes with large overlaps and retain only the boxes with the highest confidence as the final detection results. The final output contains the combination of the prediction results of all detection heads, and the bounding box coordinates and category probability of each object come from the corresponding detection head. The final output results of the detection head are relatively mature technologies in this field and will not be described here.

[0054] Another embodiment of the present application further improves the neck network, such as Figure 3 As shown, a target detection method based on the improved yolov8 model is proposed, and the neck network performs the following operations: The feature maps of three scales extracted by the backbone network are used as the first scale features, second scale features and third scale features input to the neck network; The first scale feature, the second scale feature and the third scale feature are input into the created multi-scale context aggregation module for the first fusion to obtain enhanced features; The enhanced features are concatenated with the third-scale features after the convolution operation, and then passed through the feature conversion module to obtain the converted third-scale features; The enhanced features are upsampled and concatenated with the first-scale features, and then passed through a feature conversion module to obtain the converted first-scale features; The enhanced features, the converted first-scale features, and the converted third-scale features are input into the created multi-scale context aggregation module for a second fusion; After the second fusion, the features are convolved to obtain the second-scale feature map output by the neck network; The features after the second fusion are convolved and then concatenated with the converted third-scale features to obtain the third-scale feature map output by the neck network; The features after the second fusion are upsampled and concatenated with the first-scale features after conversion to obtain the first-scale feature map output by the neck network.

[0055] This example performs two key optimizations on the neck network (Neck), introducing two multi-scale context integration modules (Multi-Scale Context Integration, MSCI) respectively. , , are the outputs of the 4th, 6th, and 9th layers in the backbone network, representing feature maps of 80*80, 40*40, and 20*20, respectively (here , , The Neck network mainly relies on the feature conversion module (C2F module), upsampling and feature concatenation (Concat) operations to fuse features of different scales. This feature fusion lacks in-depth integration of multi-scale context information, especially when facing complex backgrounds and multi-scale targets, which may lead to insufficient performance in detecting small and overlapping targets.

[0056] In this embodiment, the MSCI module is introduced into Neck for the first time in the early stage. The MSCI module integrates the , , The feature maps come from the 4th, 6th and 9th layers of the backbone network, representing feature maps of 80*80, 40*40 and 20*20 respectively. Through the first MSCI module, these feature maps of different scales are fused and processed, and then these enhanced features are further diffused to other layers of the Neck structure.

[0057] The second MSCI module receives feature outputs from Neck, corresponding to the 16th layer (P3), 10th layer (P4), and 13th layer (P5) in Neck. The second MSCI module performs a deeper fusion of these feature maps to ensure that contextual information of different scales interacts more fully in the feature maps. Similar to the first MSCI, the processed features do not directly enter the detection head, but are diffused to other network layers of the Neck structure again.

[0058] The improved Neck significantly enhances the fusion capability of features of different scales through the introduction of two MSCI modules and feature diffusion mechanism, and can better capture the target features of small targets and complex scenes. Compared with the official basic version of the YOLOv8 model, the improved Neck is not only more complex in structure, but also has greatly improved the depth of feature processing and the richness of context information, making the model have stronger multi-scale target detection capabilities.

[0059] Specifically, the multi-scale context aggregation module performs the following operations: Down-sample the input first-scale feature map; perform convolution operation on the input second-scale feature map; up-sample and convolution operation on the input third-scale feature map; then concatenate to obtain the first feature; The first feature performs four depth convolutions with different convolution kernels respectively, and the four features after the depth convolution are added together with the first feature to obtain the second feature; After convolution, the second feature is added to the first feature to obtain the output feature of the multi-scale context aggregation module.

[0060] In order to verify the technical solution of the present application, the present application compares the performance of the technical solution of the present application with that of the basic YOLOv8 through experiments. The experimental data are shown in Tables 1 and 2: Table 1 method index Car Cyclist Pedestrian average value YOLOv8 P 0.919 0.835 0.84 0.864 YOLOv8 R 0.918 0.808 0.66 0.796 YOLOv8 mAP50 0.961 0.864 0.772 0.866 YOLOv8 mAP50-95 0.784 0.575 0.429 0.596 This application P 0.926 0.881 0.856 0.888 This application R 0.933 0.815 0.699 0.815 This application mAP50 0.968 0.878 0.798 0.881 This application mAP50-95 0.807 0.603 0.464 0.624 Table 2 method Params (M) FLOPs (G) YOLOv8 3.01 8.1 This application 2.61 9.6 Among them, car represents vehicles, Cyclist represents cyclists, and Pedestrian represents pedestrians, all of which are categories that need to be identified. P is the precision index, which is used to measure the accuracy of the model's prediction of positive samples; R is the recall rate, which is used to measure the proportion of real targets that the model can find; mAP50 is the average precision value when IoU (intersection over union) is 0.5; mAP50-95 is the average precision calculated under the IOU range from 0.5 to 0.95 (step size is 0.05), Params (M) is the number of parameters, in millions (M), which is usually used to measure the complexity and size of the model. FLOPs (G) is the floating point operation, calculated in gigabytes (Giga), which represents the amount of calculation required by the model during the inference process. Through the analysis of the results, the method of this application has improved the mAP50 index by 1.5% and the mAP50-95 index by 2.8%. The detection accuracy of most categories in the dataset has been improved.

[0061] The experiment tested the performance of the improved model and the official model on an independent test set. The comparative analysis proved that the model proposed in this application has higher detection accuracy. The improved model surpasses the original YOLOv8n model in accuracy, recall rate and multiple mAP indicators. At the same time, the number of model parameters is reduced, making it more portable and suitable for embedded devices. With a slight increase in the amount of calculation, better detection accuracy is obtained, especially the detection ability of pedestrians and vehicles in complex scenes is enhanced.

[0062] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A target detection method based on the improved YOLOv8 model, characterized in that: The target detection method based on the improved YOLOv8 model includes: Build and train a YOLOv8-based target detection network model, which includes a backbone network, a neck network, and a task alignment detection head; The image to be detected is input into the trained object detection network model, and feature maps of three scales are obtained through the backbone network and the neck network; The feature maps of the three scales are input into their respective task alignment detection heads to obtain the final detection results; Among them, the task alignment detection head performs the following operations: Perform one and two convolution operations on the input feature map respectively, and then concatenate them to obtain the interactive features; Convolution operations are performed on the interaction features to generate masks and offsets; The interactive features are passed through a task decomposition module to obtain the positioning branch features; The interactive features are passed through another task decomposition module to obtain classification branch features; Perform convolution operations on the interactive features to obtain dynamic weights; Perform dynamic convolution operation on the mask and offset with the positioning branch features to obtain dynamic features; The classification branch features are fused with the dynamic weights to obtain fused features; Perform localization convolution and classification convolution operations on dynamic features and fusion features respectively to obtain localization output and classification output; The positioning output and classification output are scaled and spliced ​​through the scale processing module to obtain the detection result.

2. The target detection method based on the improved YOLOv8 model according to claim 1, characterized in that: The task decomposition module performs the following operations: Perform global average pooling on the interaction features to obtain the average features; The average features are converted into attention weights through convolution operations; Fuse the attention weight with the base weight of the convolution kernel to obtain the fused weight; The fusion weights are convolved with the interaction features to obtain the output features of the task decomposition module.

3. The target detection method based on the improved YOLOv8 model according to claim 1, characterized in that: The neck network performs the following operations: The feature maps of three scales extracted by the backbone network are used as the first scale features, second scale features and third scale features input to the neck network; The first scale feature, the second scale feature and the third scale feature are input into the created multi-scale context aggregation module for the first fusion to obtain enhanced features; The enhanced features are concatenated with the third-scale features after the convolution operation, and then passed through the feature conversion module to obtain the converted third-scale features; The enhanced features are upsampled and concatenated with the first-scale features, and then passed through a feature conversion module to obtain the converted first-scale features; The enhanced features, the converted first-scale features, and the converted third-scale features are input into the created multi-scale context aggregation module for a second fusion; After the second fusion, the features are convolved to obtain the second-scale feature map output by the neck network; The features after the second fusion are convolved and then concatenated with the converted third-scale features to obtain the third-scale feature map output by the neck network; The features after the second fusion are upsampled and concatenated with the first-scale features after conversion to obtain the first-scale feature map output by the neck network.

4. The target detection method based on the improved YOLOv8 model according to claim 3, characterized in that: The multi-scale context aggregation module performs the following operations: Down-sample the input first-scale feature map; perform convolution operation on the input second-scale feature map; up-sample and convolution operation on the input third-scale feature map; then concatenate to obtain the first feature; The first feature performs four depth convolutions with different convolution kernels respectively, and the four features after the depth convolution are added together with the first feature to obtain the second feature; After convolution, the second feature is added to the first feature to obtain the output feature of the multi-scale context aggregation module.

Images