Improved target detection method based on YOLOv8s

Abstract

The application relates to the technical field of target detection algorithm in the field of automatic driving of passenger cars, in particular to an improved target detection method based on YOLOv8s which can enhance visual perception and improve recognition accuracy, and the method is characterized in that the main network, the neck and the detection head are improved respectively: firstly, in order to improve the feature extraction capability, a reversible main network with a fused LSKA module is adopted to expand the large model architecture; secondly, in order to improve the feature fusion capability, a RepGFPN neck network based on dynamic upsampling is introduced; thereafter, in order to solve the problem of insufficient information interaction of parallel detection heads, a dynamic detection head based on the task alignment idea is proposed, which reduces the parameter quantity while improving the model detection precision.

Description

Technical fields:

[0001] This invention relates to the field of target detection algorithm technology in the field of autonomous driving for passenger vehicles, specifically to an improved target detection method based on YOLOv8s that can enhance visual perception capabilities and improve recognition accuracy. Background technology:

[0002] Visual perception mainly includes technologies such as object detection, instance segmentation, and multi-object tracking. Object detection technology can detect the location of objects of interest in an image and determine their category, ultimately outputting information such as bounding box coordinates, category information, and confidence score. As the underlying infrastructure for many advanced computer vision applications, the quality of object detection significantly impacts the performance of subsequent tasks. In the field of autonomous driving, object detection technology can help passenger vehicles detect and identify traffic signs, traffic lights, pedestrians, and various vehicles within the imaging range of visual sensors, thereby enabling functions such as adaptive cruise control and automatic obstacle avoidance. Object detection technology helps unmanned equipment perform tasks such as reconnaissance and transportation. Unmanned equipment equipped with object detection and other necessary systems can autonomously assess its surroundings, identify enemies, and take appropriate actions. In the field of intelligent security, object detection can be combined with multi-object tracking algorithms to locate and track targets in surveillance videos, providing conditions for subsequent behavior analysis and other tasks, and has important applications in many situations such as airports, train stations, and customs. Intelligent passenger vehicles equipped with sentry mode can monitor the surrounding environment of the vehicle while it is parked, allowing the owner to quickly grasp relevant vehicle information. Currently, due to the mutual occlusion and varying scales of targets in autonomous driving scenarios, higher demands are placed on the detection performance of algorithms.

[0003] Object detection algorithms can be divided into two categories: traditional object detection algorithms and deep learning-based object detection algorithms. Deep learning-based convolutional neural networks can autonomously learn target features and extract a large amount of highly non-linear semantic information, avoiding the limitations of manual feature extraction which lacks specificity. Therefore, they dominate the field of object detection. Meanwhile, the YOLO series of algorithms, with their high accuracy and speed advantages, have brought new breakthroughs to visual perception algorithms, and the continuous emergence of new models has created possibilities for the practical application of these algorithms. YOLOv8s boasts a fast detection speed, but its accuracy still needs improvement when dealing with multi-scale targets and occluded targets. Summary of the Invention:

[0004] This invention addresses the shortcomings and deficiencies of existing technologies by proposing an improved target detection method based on YOLOv8s that enhances accuracy while maintaining real-time performance.

[0005] This invention achieves its purpose through the following measures:

[0006] An improved object detection method based on YOLOv8s is characterized by improvements to the backbone network, neck network, and detection head: First, to improve feature extraction capabilities, a reversible backbone network incorporating LSKA modules is adopted, extending to a larger model architecture; second, to improve feature fusion capabilities, a RepGFPN neck network based on dynamic upsampling is introduced; subsequently, to address the problem of insufficient information exchange between parallel detection heads, a dynamic detection head based on task alignment is proposed, which improves model detection accuracy while reducing the number of parameters.

[0007] The improvements to the backbone network in this invention include: introducing the reversible multi-subnetwork RevCol into the model design, using the reversible subnetwork column as the unit to transmit information, the entire network structure includes multiple subnetwork columns, and adding reversible connections between subnetwork columns. By repeatedly connecting the input to the subnetwork columns, low-level texture details and semantic information are gradually separated.

[0008] In the reversible multi-level fusion module, m is the order of recursion, and the input x of the reversible multi-level fusion module is... t Instead of relying solely on the outputs of the previous two blocks, it now considers the previous m blocks. Therefore, when working backwards, obtaining all m-1 inputs from the first part is sufficient to calculate the input x for the second part. t-m Divide the m features into a group, i.e. (x1, x2, ..., x...) m ),(x m+1 ,x m+2 ,…,x 2m With this improvement, the restriction on feature dimensions is greatly reduced when the value of m is large. It is no longer necessary for the feature map dimensions within a group to be consistent, but only for the feature dimensions between groups to be consistent. Therefore, tensors of different shapes are used to represent features of different semantic levels or different resolutions.

[0009] Using only the low-level features of the current column and the high-level features of the previous column, the simplified process is shown in equations (1) and (2):

[0010] Forward:x t =F t (x t-1 ,x t-m+1 )+γx t-m (1)

[0011] Inverse:x t-m =γ -1 [x t -F t (x t-1 ,x t-m+1 (2)

[0012] The basic structure of RevCol consists of sub-networks composed of m features in each group. The structure in the sub-networks can be replaced by various classic CNN or Transformer models. The RevCol architecture is used to reconstruct the backbone network of YOLOv8s. Considering that the number of parameters and computation will increase when the number of columns is increased, the number of sub-networks is set to 3. The stacking of individual networks is based on the design of the YOLOv8s backbone network. The number of C2f blocks stacked in each column is 1, 2, 2, 1 from the first to the fourth layer. The input image is first divided into patches according to the VIT method, and then input to each column. Each column has 4 levels of feature maps. Multi-resolution features can be extracted from the output of each stage. When performing classification tasks, only the last layer feature map of the last column is used because it has rich semantic information. When performing detection and segmentation tasks, all 4 levels of features of the last column are used to obtain both low-level and high-level semantic information.

[0013] This invention names the reconstructed backbone network RevCol-C2f. In RevCol-C2f, each column maintains a consistent architecture, has the same input but different weights, and generates its own prediction results. Features from low to high level are stored in each column sequentially. Then, through a reversible transformation, the features of the i-th column are propagated to the (i+1)-th column without loss of information. During the propagation process, as complexity and nonlinearity increase, the level of all features gradually increases, and the decoupled representation of the prediction input is given in the last column of the network.

[0014] In this invention, LSKA is introduced into the third max pooling operation of SPPF. After processing by LSKA, the output is fed into the final convolution. LSKA can weight the multi-scale feature map, so that the model can pay more attention to the features related to the target and improve the detection accuracy. Specifically, the output of the large separable kernel attention LSKA is calculated by equations (3) to (6). First, the convolutional layer is initialized. Two convolutional layers are responsible for extracting the horizontal and vertical features of the input feature map respectively to generate a preliminary attention map. Then, spatial dilation convolution with different dilation rates is used to further extract features, covering a larger receptive field and capturing a wider range of contextual information without increasing the computational cost. Finally, the features obtained by fusing through a convolutional layer are used to generate the final attention map. This attention map is multiplied by the original input feature map at the element level. In this way, each element in the original feature map is weighted according to the corresponding parameters of the attention, highlighting important features.

[0015]

[0016] in, This represents the output of a depthwise convolution with a kernel size of (2d-1)×(2d-1);

[0017] SPPF is an important component in the YOLOv8s model. It is used to extract multi-scale features. Through spatial pyramid pooling, SPPF can fuse feature maps of different scales together, thereby enhancing the model's robustness to changes in the target scale.

[0018] In the sampling stage of the neck network, this invention introduces dynamic upsampling (DyUpsample) into RepGFPN to replace the original upsampling module. The improved Neck structure is named GDFPN. Dynamic sampling bypasses dynamic convolution and constructs an upsampling structure from the perspective of point sampling. For a given feature map χ, the output of dynamic sampling is obtained by processing the input and the sample set through the grid sample function. The process of obtaining the sample set can be expressed as equations (7) and (8), that is, using a linear layer and a sigmoid function to obtain the offset O, and the sample set S is the sum of the offset and the original sampling grid G:

[0019] O=σ·sigmoid(linear1(χ))·linear2(χ) (7),

[0020] S=G+O (8), where σ represents the static factor, which is 0.5.

[0021] This invention employs a customized task alignment structure in the detection head: after feature maps from different stages are input into the detection head, they first undergo two 3×3 group convolutions. Group normalization (GroupNorm) has been proven in the FCOS algorithm to improve the performance of detection head localization and classification. Using group convolutions can reduce the amount of computation and parameters, and isolate information exchange between different groups. When calculating task interaction features, classification and localization features are used simultaneously, which allows each task to better perceive the state of the other. However, since task alignment uses a single branch, the two tasks have different goals, i.e., they focus on different features. Task interaction features will inevitably introduce feature conflicts between the two tasks. Therefore, TOOD proposes a layer attention mechanism, which decomposes tasks by dynamically calculating task-specific features. The calculation method of task-specific features for each task is shown in equations (9) and (10):

[0022]

[0023] Among them, wk It is the k-th parameter of the learnable attention layer, w k ∈R N ,

[0024]

[0025] Where δ represents the ReLU activation function, x inter Through the After average pooling, the final classification or localization result is obtained from each X. task As shown in equation (11), Z is obtained. task The scores P and bounding boxes are converted using the sigmoid function and the distance to the bounding box, respectively. Then, the two tasks are further aligned more precisely based on the spatial distribution of P and B.

[0026] Z task =conv2(δ(conv1(X) task ))) (11).

[0027] This invention addresses the shortcomings of the TOOD task alignment strategy in the prediction stage, which is not concise enough and difficult to adapt to the YOLOv8 framework. It proposes a strategy that uses DCNv2 and offset and mask generated from interactive features in the localization branch, and uses interactive features for dynamic feature selection in the classification branch. This approach applies the concept of deformability to process the task alignment output. The offset and mask are obtained from the output of the shared convolution, ensuring the conciseness and portability of the detector head position generation. A portion of the shared convolution output is processed and used together with the task alignment output to complete the dynamic feature selection, thereby obtaining the classification prediction output. After scaling, the final output is obtained.

[0028] Compared with existing technologies, this invention optimizes the YOLOv8s algorithm, further improving its performance in detection tasks. For the backbone, neck, and head structure of the baseline algorithm, a reversible backbone network, a GDFPN fusion network, and a task-aligned dynamic detection head are introduced respectively. The effectiveness of each part of the improved algorithm is verified through ablation experiments on the COCO 20000 validation set. The algorithm's performance is tested on the COCO 20000 and KITTI 2D Object datasets. The results show that the improved algorithm significantly improves accuracy compared to the baseline algorithm, with an overall AP index improvement of more than 2 percentage points, enabling it to better handle complex autonomous driving scenarios. Attached image description:

[0029] Figure 1 This is a schematic diagram of the overall structure of the improved detection model in this invention.

[0030] Figure 2 This is the reversible multi-level fusion module in this invention.

[0031] Figure 3 This is a schematic diagram of the reconstructed backbone network RevCol-C2f in this invention.

[0032] Figure 4 This is a schematic diagram of the attention of a large convolutional kernel in this invention.

[0033] Figure 5 This is a schematic diagram of the SPPF-LSKA structure in this invention.

[0034] Figure 6 This is a schematic diagram of the fusion module structure in this invention, wherein (a) is a CSPStage structure diagram and (b) is a RepConv structure diagram.

[0035] Figure 7 This is a schematic diagram of the DyUpsample module in this invention.

[0036] Figure 8 This is a diagram of the TADDH network structure in this invention.

[0037] Figure 9 This is a schematic diagram of Layer Attention in this invention.

[0038] Figure 10 This is a schematic diagram of the normalized confusion matrix of YOLOv8s and YLGTRv8s in an embodiment of the present invention, wherein (a) is the confusion matrix of YOLOv8s verification result and (b) is the confusion matrix of YLGTRv8s verification result.

[0039] Figure 11 These are the inference experiment results of the detection algorithm in this embodiment of the invention. (a) is the inference result of YOLOv8s, and (b) is the inference result of YLGTRv8s.

[0040] Figure 12 These are the heatmap experimental results in the embodiments of the present invention, wherein Figure (a) is the target detection heatmap of the YOLOv8s algorithm, and Figure (b) is the target detection heatmap of the YLGTRv8s algorithm.

[0041] Figure 13 The figure shows the convergence effect of the validation set in the embodiment of the present invention, where Figure (a) is the loss change curve and Figure (b) is the accuracy change curve.

[0042] Figure 14 This is a schematic diagram of the receptive field of the model in an embodiment of the present invention, where (a) is the receptive field of YOLOv8s and (b) is the receptive field of YLGTRv8s. Detailed implementation method:

[0043] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0044] YOLOv8s boasts a relatively fast detection speed, but its accuracy needs improvement when dealing with multi-scale targets and occluded targets. To enhance model accuracy while maintaining real-time performance, this invention improves the backbone network, neck network, and detection head, naming the improved algorithm YLGTRv8s (YOLOv8s-LSKA-GDFPN-TADDH-RevCol). When not using the reversible backbone network, it is named YLGTv8s. The overall structure of the improved model is as follows: Figure 1 As shown.

[0045] To address the issues of feature over-compression and network design paradigms, the Reversible Column Network (RevCol) introduces the concept of disentangled feature learning into its model design. It proposes using reversible columns as units to transmit information, ensuring feature decoupling while preserving information transmission within the network. The entire network structure comprises multiple sub-networks (columns) with reversible connections between them. By repeatedly feeding input into columns, low-level texture details and semantic information are gradually separated. This approach maintains high detection accuracy during pre-training while preventing the loss of low-level information, leading to better performance in downstream tasks (object detection, instance segmentation, etc.). The RevCol architecture provides a new paradigm for large-scale model design, allowing pure CNN networks to continue evolving towards larger models. It still follows the design paradigm of reversible neural networks and inherits the inherent memory-saving advantage of reversible networks, bringing advantages to the training of large models.

[0046] The process of the Reversible Multi-Level Fusion Module is as follows: Figure 2 As shown.

[0047] m is the order of recursion. The design of the reversible multi-level fusion module adds more x to its input, meaning that x in this module... t Instead of relying solely on the outputs of the previous two blocks, it now considers the previous m blocks. Therefore, when working backwards, obtaining all m-1 inputs from the first part is sufficient to calculate the input x for the second part. t-m Generally, m features are divided into a group, i.e. (x1, x2, ..., x...). m ),(x m+1 ,x m+2 ,…,x 2mWith this improvement, the restriction on feature dimensions is greatly reduced when the value of m is large. It is no longer necessary for the feature map dimensions within a group to be consistent; only the feature dimensions between groups are required to be consistent. Therefore, tensors of different shapes can be used to represent features at different semantic levels or different resolutions. In practical use, too much input will bring a small accuracy gain but consume more GPU resources. Therefore, only the low-level features of the current column and the high-level features of the previous column are used. This simplification will not destroy the reversible performance. The simplified process is shown in equations (1) and (2).

[0048] Forward:x t =F t (x t-1 ,x t-m+1 )+γx t-m (1),

[0049] Inverse:x t-m =γ -1 [x t -F t (x t-1 ,x t-m+1 (2),

[0050] The basic structure of RevCol is a subnetwork composed of m features in each group. The structure in the subnetwork can be replaced by various classic CNN or Transformer models.

[0051] Inspired by the RevCol architecture, this invention incorporates its design principles to reconstruct the YOLOv8s backbone network, aiming to significantly improve the algorithm's accuracy. The reconstructed backbone network is as follows: Figure 3As shown, considering that adding columns to the network will simultaneously increase the number of parameters and computational cost, which will bring difficulties to model training and deployment, this invention sets the number of sub-networks to 3. The stacking of individual networks follows the design of the YOLOv8s backbone network. The number of C2f blocks stacked in each column is 1, 2, 2, 1 from the first to the fourth layer, in order to verify the performance improvement brought by the reversible multiple sub-networks as much as possible. The input image is first processed into patches according to the VIT method, and then fed into each column. Each column has 4 levels of feature maps, which can easily extract multi-resolution features from the output of each stage. When performing classification tasks, only the last layer feature map of the last column is used because it has rich semantic information. When performing detection and segmentation tasks, all 4 levels of features of the last column are used to obtain both low-level and high-level semantic information. This invention names the reconstructed backbone network RevCol-C2f. The architecture of each column in RevCol-C2f is consistent, with the same input but different weights, and each column generates its own prediction results. Features are stored sequentially in each column from low to high level. Then, a reversible transformation propagates the features of the i-th column to the (i+1)-th column without loss of information. During propagation, the level of all features gradually increases with the increase of complexity and nonlinearity. The last column of the network predicts the decoupled representation of the input. The reversible branch of the RevCol architecture increases the model dimensionality, which helps improve the accuracy of the algorithm. This is of great significance for the recognition of small targets in autonomous driving environments. However, the reversible architecture also inevitably increases the model inference cost.

[0052] A major reason for the success of VIT is its ability to model long-term dependencies in the input image, a capability that CNN networks can replicate by using large receptive fields with attention mechanisms. Vision Attention Networks (VANs) with Large Kernel Attention (LKA) modules have demonstrated superior performance on various vision-based tasks. The LKA module employs standard depthwise convolutions with small receptive fields to capture local dependencies and compensate for gridding issues, followed by dilated depthwise convolutions with large receptive field kernels to model long-range dependencies. However, the increased computational and memory consumption of these deep convolutional layers with increasing kernel size reduces the effectiveness of the model as kernel size increases.

[0053] To address this issue, researchers have proposed Large Separable Kernel Attention (LSKA). LSKA first decomposes the deep convolution with a large kernel into deep convolutions with smaller kernels, and further decomposes the two-dimensional kernels of the deep convolutional layers into cascaded horizontal and vertical one-dimensional kernels. Then, it performs a spatially dilated depth-wise convolution with a considerably large kernel size. Unlike the standard LKA design, this decomposition strategy allows the direct use of the large kernels of the deep convolutional layers in the attention module without requiring any additional blocks. The structure of the LSKA module is as follows: Figure 4 As shown.

[0054] The output of LSKA can be calculated using equations (3) to (6). First, the convolutional layers are initialized, and two convolutional layers are used to extract the horizontal and vertical features of the input feature map, respectively, to generate a preliminary attention map. Then, spatial dilation convolutions with different dilation rates are used to further extract features, covering a larger receptive field and capturing a wider range of contextual information without increasing computational cost. Finally, the features obtained are fused through a convolutional layer to generate the final attention map. This attention map is multiplied element-wise with the original input feature map, so that each element in the original feature map is weighted according to the corresponding attention parameters, highlighting important features.

[0055]

[0056] A C =W 1×1 *Z C (5),

[0057]

[0058] in, This represents the output of a depthwise convolution with a kernel size of (2d-1)×(2d-1).

[0059] SPPF is a crucial component in the YOLOv8s model, used for extracting multi-scale features. Through spatial pyramid pooling, SPPF fuses feature maps of different scales, enhancing the model's robustness to changes in target scale. However, while SPPF improves multi-scale feature extraction to some extent, there is still room for improvement. Inspired by LSKA, this invention introduces LSKA after the third max pooling operation of SPPF, with the LSKA-processed output fed into the final convolution. LSKA weights the multi-scale feature maps, allowing the model to focus more on target-related features, thus improving detection accuracy. The improved SPPF structure is shown below. Figure 5As shown.

[0060] GDFPN Feature Fusion Network:

[0061] In traditional Feature Pyramid Networks (FPNs), multi-scale fusion aims to aggregate features output from different stages of the backbone network, thereby enhancing the expressive power of the output features and improving model performance. FPNs introduce a top-down path to fuse features. Considering the limitations of unidirectional information flow, Path Aggregation Networks (PANs) add an additional bottom-up path, but this increases computational cost. To reduce computational burden, the YOLO series networks choose an FPN+PAN structure with Cross Stage Partial Connections (CSPs) for feature fusion. To better meet the accuracy and speed requirements of real-time detection, DAMO-YOLO proposes a novel RepGFPN based on GFPN (Global Feature Pyramid Networks). RepGFPN mainly includes the following improvements: different numbers of channels are used for features at different scales, so as to flexibly control the expressive power of high-level and low-level features under the constraint of lightweight computation; the original convolution-based feature fusion is improved into a CSPStage structure, and the ideas of reparameterization and efficient long-range aggregation network (ELAN) are introduced to improve the model accuracy without increasing the amount of computation.

[0062] CSPStage structure as follows Figure 6 As shown in (a), one branch has a residual block consisting of N RepConvs, and the RepConv structure is as follows. Figure 6 As shown in (b), its characteristic is structural reparameterization, employing two branches during training and fusing multiple branches together during inference, significantly reducing inference time. To improve the multi-scale fusion capability of the algorithm in this invention, RepGFPN is introduced as the Neck part of the algorithm, and improvements are made accordingly.

[0063] During the neck network sampling phase, this invention introduces dynamic upsampling (DyUpsample) into RepGFPN to replace the original upsampling module. The improved neck structure is named GDFPN, ​​and its structure is as follows: Figure 1As shown, ordinary upsampling is achieved using transpose convolution, which can enlarge the size of the input feature map while preserving the spatial relationships of the features. This increases the feature map resolution and enriches spatial information, helping to improve the performance and accuracy of object detection. Therefore, lightweight and efficient upsampling structures are crucial for Neck networks. Although recent dynamic upsampling structures based on convolutional kernels have achieved remarkable performance, these modules introduce dynamic convolution and additional subnetworks for generating the dynamic convolutional kernels, resulting in a significant computational burden and limiting their application scenarios. Dynamic sampling aims to solve this problem by bypassing dynamic convolution and constructing an upsampling structure from the perspective of point sampling. This saves resources and is easy to implement. The module structure is shown below. Figure 7 As shown.

[0064] For a given feature map χ, the output of dynamic sampling is obtained by processing the input and the sample set through the grid sample function. The process of obtaining the sample set can be expressed as equations (7) and (8), that is, using a linear layer and the sigmoid function to obtain the offset O, and the sample set S is the sum of the offset and the original sampling grid G.

[0065] O=σ·sigmoid(linear1(χ))·linear2(χ) (7),

[0066] S=G+O (8), where σ represents the static factor, which is generally chosen as 0.5.

[0067] Object detection is often represented as a multi-task learning problem that jointly optimizes object classification and localization. Most current one-stage detectors perform object classification and localization in parallel and independently through two independent branches (i.e., the head network). Due to the different learning mechanisms for classification and localization, this design with two parallel branches leads to different spatial distributions of features learned by the two tasks. When using two separate branches for prediction, this can result in a certain degree of misalignment.

[0068] To address these issues, researchers proposed a Task-aligned One-stage Object Detection (TOOD). Addressing the characteristic of traditional one-stage object detection where classification and localization are implemented in parallel with two separate branches, a task alignment head was designed to enhance the interactivity between the two tasks. The task alignment head computes task interaction features and makes predictions using a novel Task-aligned Predictor (TAP). It then aligns the spatial distributions of the two predictions based on information provided by task alignment learning. This allows the two tasks to work collaboratively, thereby adjusting their predictions more accurately. Furthermore, to further overcome the misalignment problem, TOOD introduces Task-aligned Learning (TAL) to define the distance between the optimal anchors for the two tasks. Sample allocation collects training samples (positive or negative samples) by calculating the task alignment degree for each anchor, while the task alignment loss gradually unifies the optimal anchor to predict classification and localization results during training. During inference, the model can retain the bounding box with the highest classification score and the most accurate localization.

[0069] Inspired by the TOOD concept, this invention further proposes a Task-aligned Dynamic Detection Head (TADDH), whose network structure is as follows: Figure 8 As shown, TOOD only performs task alignment in the label assignment strategy, while this invention also adopts a customized task alignment structure in the detection head. Existing object detector heads typically use independent classification and localization branches, which leads to a lack of interaction between the two tasks. TADDH learns task interaction features from multiple convolutional layers through a feature extractor to obtain joint features. The localization branch uses DCNv2 and interaction features to generate DCNv2 offset and mask, and the classification branch uses interaction features for dynamic feature selection. By using shared convolutions, the number of parameters is significantly reduced, which is more conducive to training and deploying the model on resource-constrained devices. While using shared convolutions, to address the problem of inconsistent target scales detected by each detection head, a scale layer is used to scale the features.

[0070] After the feature maps from different stages are input into the detection head, they first undergo two 3×3 grouped convolutions. Group normalization (GroupNorm) has been proven in the FCOS algorithm to improve the performance of detection head localization and classification. Using grouped convolutions can reduce the amount of computation and parameters, and isolate information exchange between different groups.

[0071] When calculating task interaction features, both classification and localization features are used simultaneously, allowing each task to better perceive the other's state. However, since task alignment uses a single branch, and the two tasks have different objectives (i.e., focusing on different features), task interaction features inevitably introduce feature conflicts between the two tasks. Therefore, TOOD proposes a layer attention mechanism, such as... Figure 9 As shown, the task is decomposed by dynamically calculating task-specific features. This design is continued in this invention. The calculation method of task-specific features for each task is shown in equations (9) and (10).

[0072]

[0073] Among them, w k It is the k-th parameter of the learnable attention layer, w k ∈R N .

[0074]

[0075] Where δ represents the ReLU activation function, x inter Through the The average pooling is then performed.

[0076] Ultimately, the classification or localization results are derived from each X. task The result is shown in equation (11). The obtained Z... task The scores P and bounding boxes B are converted using the sigmoid function and the distance to the bounding box, respectively. Then, the two tasks are further aligned more precisely based on the spatial distribution of P and B.

[0077] Z task =conv2(δ(conv1(X) task ))) (11),

[0078] In the prediction phase, TOOD's task alignment strategy is not concise enough and is difficult to adapt to the YOLOv8 framework. Therefore, this invention proposes a strategy where the localization branch uses DCNv2 and offsets and masks generated from interactive features, while the classification branch uses interactive features for dynamic feature selection. The deformable concept is applied to process the task alignment output; the offset and mask are obtained from the output of the shared convolution, ensuring the conciseness and portability of the detector head position generation. A portion of the shared convolution output is processed and used together with the task alignment output to perform dynamic feature selection, thereby obtaining the classification prediction output, which is then scaled to obtain the final output. Example:

[0079] The following is an experimental analysis of the present invention:

[0080] The hardware conditions and environment configuration for this experiment are shown in Table 1.

[0081] Table 1 Equipment and Environmental Configuration

[0082]

[0083] After comprehensively considering the amount of data, the breadth of use, and practical considerations, this project adopted MS COCO 2017 as the basic experimental dataset for the object detector. The MS COCO 2017 dataset contains a large number and variety of images, most of which are unnecessary for autonomous driving scenarios. This invention first filters the images in this dataset, focusing on extracting images from the main categories that frequently appear in autonomous driving scenarios, such as pedestrians, vehicles, and traffic lights. Considering the diversity of scenarios, this invention retains categories including motorcycles, bicycles, and pet dogs. To measure the algorithm's ability to perceive small targets and to increase the difficulty of the dataset, two smaller target categories, bottles and mobile phones, were added. The features of these two target categories differ significantly from those of pedestrians and vehicles, allowing for better measurement of the model's generalization ability. To reduce false positives, some background images were added to the extracted dataset, resulting in a dataset named COCO 20000. COCO 20000 is divided into a training set and a validation set, with specific parameters shown in Table 2.

[0084] Table 2 COCO 20000 dataset

[0085]

[0086] In addition, to verify the algorithm's performance in real-world driving environments, this invention conducted supplementary experiments on the KITTI 2D Object dataset. 7481 images from the dataset were selected and divided into training and validation sets in an 8:2 ratio. The categories were divided into three main types: car, cyclist, and pedestrian. The car category encompasses various vehicle types such as truck and van. Specific parameters of this dataset are shown in Table 3.

[0087] Table 3 KITTI 2D Object Dataset

[0088]

[0089] The parameter settings for training and validation of this invention are shown in Table 4. For the optimizer, the SGD optimizer, commonly used in YOLO series algorithms, was adopted, with an initial learning rate of 0.01 and a final learning rate of 0.0001. During model training, the input size was set to 640×640, and Mosaic data augmentation was used to preprocess the images. Mosaic integrates multiple augmentation methods such as rotation, cropping, and blending, which helps prevent overfitting. Data augmentation was disabled in the last 10 training rounds to better fit the features.

[0090] Table 4 Experimental Parameter Settings

[0091]

[0092] Analysis of Target Detection Experimental Results

[0093] The experimental results of the improved algorithm in this example are based on a reversible backbone network with four sub-branches. After the model was trained, it was evaluated on the validation set, and the results are shown in Table 5. The experimental results show that the improved YLGTRv8s model has an inference speed of 3.9ms and an FPS of 256, meeting the real-time requirements of the scenario. Meanwhile, AP... 50 With AP 50-95 Compared to YOLOv8s, the improvements of 3.4% and 2.6% are significant for targets that are difficult to detect in complex traffic conditions. The ablation experiments in Section 4.4 show that the increase in the number of parameters and computational cost of the improved algorithm mainly comes from the reversible backbone network. In some scenarios with extremely high real-time requirements, a balance between accuracy and speed can be achieved by adjusting the number of branches in the RevCol-C2f network. Without adding a reversible backbone network, the improved algorithm YLGTv8s still achieves leading accuracy while better controlling inference latency.

[0094] Table 5. Comparison of Object Detection Algorithms (YOLO Evaluation Metrics)

[0095]

[0096] To better evaluate the performance of the algorithm of this invention, a representative algorithm was selected as a control experiment. During training, the control algorithm was trained for 200 rounds according to the parameters provided by the official documentation. The mmdetection open-source detection framework was used during training, and the COCO 20000 dataset constructed in this invention was used. The training results were evaluated using the COCO metric, as shown in Table 6.

[0097] Table 6. Comparison Experiment of Target Detection Algorithms (COCO Evaluation Index)

[0098]

[0099]

[0100] Because the COCO index calculation does not use linear interpolation, the performance metrics of YLGTRv8s calculated in this way will deviate from the results in Table 5. The improved algorithm of this invention improves all accuracy metrics compared to the original algorithm, and performs better in detecting targets of different scales, with detection accuracy for small, medium, and large targets all increasing by more than 2 percentage points. Furthermore, in comparisons of detection performance with various representative detection algorithms on the COCO 20000 validation set, the improved algorithm YLGTRv8s significantly outperforms the comparison algorithms in overall accuracy, and its performance even surpasses that of the Transformer-based RT-DETR-R18 network with a similar model size.

[0101] To visually demonstrate the algorithm's detection performance for various targets, a confusion matrix of the model's evaluation results on the COCO 20000 validation set was plotted, as shown below. Figure 10 As shown.

[0102] The sum of each row in the confusion matrix represents the actual number of samples in that category, and the sum of each column represents the number of samples predicted by the model as belonging to that category. In this example, the confusion matrix is ​​normalized; the darker the color of the boxes in the matrix, the closer the value is to 1, indicating better detection performance for that category. For the improved algorithm, we hope that the proportion of correctly predicted samples in its confusion matrix is ​​higher than that of the original algorithm. As can be seen from the figure, the values ​​on the diagonal of the confusion matrix of YLGTRv8s are improved to varying degrees compared to YOLOv8s, with improvements of 0.05, 0.06, and 0.13 for the pedestrian, car, and traffic light categories, respectively, compared to the original algorithm. Because there are more pedestrian samples in the training set, the algorithm also has a higher accuracy rate for predicting this category on the validation set. For the targets of interest in this invention, such as cars and motorcycles, the accuracy rate also reaches above 0.5.

[0103] To more comprehensively evaluate the performance of the improved algorithm, experiments were conducted on the KITTI 2D Object dataset. This dataset mainly consists of human and vehicle targets and is specifically designed for autonomous driving scenarios. Its category variations are less than those of COCO 20000, making it relatively easier. The experimental results are shown in Table 7. The AP of the improved algorithm across all categories... 50 The accuracy rate reached 92.9%, which can significantly reduce false detections in practical applications and further verifies that the algorithm of this invention can effectively solve the multi-scale target perception problem in autonomous driving scenarios.

[0104] Table 7 Experimental Results of the KITTI Dataset

[0105]

[0106] To more intuitively demonstrate the detection performance in real-world scenarios, this example selects representative images for algorithm inference experiments. The input size is uniformly set to 640×640, the same size used during training. The inference results are as follows: Figure 11 As shown, the original image size is 2046×1179. The number of people, bicycles, cars, motorcycles, traffic lights, and buses that humans can distinguish in the image are 13, 3, 6, 6, 4, and 1, respectively. The number of each category detected using the YOLOv8s algorithm is 4, 1, 3, 2, 3, and 0, while the number detected by the YLGTRv8s algorithm is 5, 3, 5, 2, 4, and 0. Compared to YOLOv8s, except for the severely occluded bus which neither algorithm detected, the number of detected categories and the confidence level are improved.

[0107] The inference results show that the improved algorithm effectively enhances the detection performance for small and occluded targets. For small target detection, focus on the green traffic light on the far left of the image; YOLOv8s failed to detect this target. For occluded targets, focus on the motorcycle and pedestrian to the left of the red vehicle. When detecting a pedestrian riding a motorcycle, the YOLOv8s algorithm sometimes categorizes the entire motorcycle as a single class, while YLGTRv8s can detect both classes of targets separately. Figure 12 Heatmaps of the output layers of the YOLOv8s algorithm and its improved version are presented. The heatmaps are calculated using features from three branches of the input detection head in the algorithm.

[0108] In the heatmap, we want the areas with a high degree of overlap with the target to be the parts that the model focuses on. As can be seen from the rightmost traffic light in the figure, the red area in the YLGTRv8s heatmap has a higher degree of overlap with the shape of the original traffic light. This means that the improved model can better focus on the overall features of the target, thus showing better detection and recognition performance.

[0109] The Neck, Head, and Backbone structures were added to the baseline algorithm, and the experimental results are shown in Table 8. C1 represents the addition of GDFPN, ​​C1-2 represents the addition of both GDFPN and TADDH, and C1-3 represents the addition of GDFPN, ​​TADDH, and RevC2f-LSKA. GDFPN, ​​due to its introduction of dynamic upsampling and the reparameterized CSPStage structure, makes the model lightweight and efficient. With only a 0.1 increase in inference latency, GDFPN improved the AP by 1.3 percentage points. 50 Increased by 0.9 percentage points in AP 50-95 Improvement. Based on this, add TADDH and AP. 50It was further increased by 1.1, AP 50-95 It improved by 0.9 percentage points. The RevC2f-LSKA structure, with its reversible structure and large separable convolutional kernel attention mechanism, brings excellent feature extraction and fusion capabilities to the model, resulting in high AP. 50-95 It reached 40.7%.

[0110] Table 8 Ablation Experiment Results

[0111]

[0112] To more intuitively demonstrate the convergence of different algorithms during the ablation experiment, convergence diagrams for each algorithm during training were plotted, and the validation set loss curves are shown below. Figure 13 As shown in (a), all algorithms used in the ablation experiments converged on the validation set. Since the algorithms employed three different loss functions, they are plotted separately. The red curve represents the improved algorithm YLGTRv8s proposed in this invention, which achieved faster and lower loss function convergence. The accuracy variation curve is shown below. Figure 13 As shown in (b), the differences in accuracy among the various algorithms in the ablation experiment are clearly visible through the images, demonstrating the significant advantage of the improved algorithm. Furthermore, we note the AP... 50-95 The curve has a very slight downward bend at the end, which is due to the model starting to overfit. At this point, it proves that the model has fully converged and there is no need to increase the number of training rounds.

[0113] The size of the receptive field represents the range of image perception of the model. Increasing the receptive field can improve the model's ability to understand images and extract higher-level semantic features. To verify the changes in the receptive field of different output layers of the model, schematic diagrams of the receptive fields of YOLOv8s and YLGTRv8s were drawn respectively. The receptive field calculation method is based on RepLKNet.

[22] The Chinese method selects the first 50 images from the training set for calculation and normalizes the results. In the image, the darker the green, the closer the value is to 1, indicating a better understanding of the region by the model. The receptive field of YOLOv8s is as follows: Figure 14 As shown in (a), the selected output layers from left to right are the P3 stage, P4 stage, and the last layer of the YOLOv8s backbone network. It can be seen that as the network deepens, the receptive field of the model gradually increases. Through the fusion of different layers, the model can detect targets at different scales. The receptive field of YLGTRv8s is shown below. Figure 14 As shown in (b), the output layers selected from left to right in the figure are P3, P4, and the last layer of the RevC2f-LSKA network. It can be seen that the improved model has a larger and denser receptive field. This is because the SPPF-LSKA module uses large separable convolutional kernel attention, where dilated convolutions increase the scale of the model's receptive field. Figure 14(b) The change from the middle image to the right image intuitively explains the performance improvement of the improved algorithm.

[0114] This invention optimizes the YOLOv8s algorithm, proposing an improved visual object detection algorithm that further enhances its performance on detection tasks. For the three-part structure of the baseline algorithm (backbone, neck, and head), an invertible backbone network, a GDFPN fusion network, and a task-aligned dynamic detection head are introduced, respectively. Ablation experiments on the COCO 20000 validation set verify the effectiveness of each part of the improved algorithm. Performance tests on the COCO 20000 and KITTI 2D Object datasets show that the improved algorithm significantly improves accuracy compared to the baseline algorithm, with an overall AP (Average Precision) improvement of more than 2 percentage points, enabling it to better handle complex autonomous driving scenarios.

Claims

1. An improved target detection method based on YOLOv8s, characterized in that, Improvements were made to the backbone network, neck network, and detection head: First, a reversible backbone network incorporating LSKA modules was adopted to extend the model to a larger architecture; second, a RepGFPN neck network based on dynamic upsampling was introduced. Subsequently, a dynamic detection head based on the idea of ​​task alignment improved the model's detection accuracy while reducing the number of parameters; Improvements to the backbone network include: introducing the reversible multi-subnetwork RevCol into the model design, using the reversible subnetwork column as the unit to transmit information, the entire network structure includes multiple subnetwork columns, and adding reversible connections between subnetwork columns. By repeatedly feeding the input into the subnetwork columns, low-level texture details and semantic information are gradually separated. Among them, the reversible multi-level fusion module m It is the order of recursion, the input of the invertible multi-level fusion module. No longer relying solely on the output of the previous two blocks, but rather... m One, when working backwards, only need to obtain all of the first part. m The second part of the input can be calculated with -1 input. x t-m ,Bundle m Each feature is divided into a group, namely ( x 1, x 2,…, x m ), ( x m+1 , x m+2 ,…, x 2m Tensors of different shapes can be used to represent features at different semantic levels or resolutions. Using only the low-level features of the current column and the high-level features of the previous column, the simplified process is shown in equations (1) and (2): (1)， (2)， Each group m The basic structure of RevCol consists of sub-networks composed of features. The structure in the sub-networks can be replaced by various classic CNN or Transformer models. The RevCol architecture is used to reconstruct the backbone network of YOLOv8s. The number of sub-networks is set to 3. The stacking of individual networks is based on the design of the YOLOv8s backbone network. The number of C2f blocks in each column is 1, 2, 2, 1 from the first layer to the fourth layer. The input image is first divided into patches according to the VIT method, and then input to each column. Each column has 4 levels of feature maps. Multi-resolution features can be extracted from the output of each stage. When performing classification tasks, only the last layer feature map of the last column is used. When performing detection and segmentation tasks, all 4 levels of features of the last column are used to obtain low-level and high-level semantic information at the same time. A custom task-aligned structure is adopted in the detection head: after the feature maps from different stages are input into the detection head, they first pass through two 3×3 group convolutions. Group normalization (GroupNorm) has been proven in the FCOS algorithm to improve the performance of detection head localization and classification. Using group convolution can reduce the amount of computation and parameters, and isolate the information exchange between different groups. When calculating the task interaction features, the classification and localization features are used at the same time. The task is decomposed by dynamically calculating the task-specific features. The calculation method of the task-specific features for each task is shown in Equations (9) and (10): (9)， in, It is the first of the learnable attention layers. k One parameter, , (10)， in, Represents the ReLU activation function. Through the After average pooling, the final classification or localization result is obtained from each As shown in equation (11), the result is obtained. The scores P and bounding boxes are converted using the sigmoid function and the distance to the bounding box, respectively. Then, the two tasks are further aligned more precisely based on the spatial distribution of P and B. (11)。 2. The improved target detection method based on YOLOv8s according to claim 1, characterized in that, The reconstructed backbone network is named RevCol-C2f. In RevCol-C2f, each column maintains a consistent architecture, receives the same input, but has different weights and generates its own predictions. Features from low to high levels are sequentially stored in each column, and then a reversible transformation is used to transform the first-order features into the second-order features without loss of information. i The feature propagation of the first column to the second column... i +1, during the propagation process, as complexity and nonlinearity increase, the level of all features gradually increases, predicting the decoupled representation of the input in the last column of the network.

3. The improved target detection method based on YOLOv8s according to claim 1, characterized in that, After introducing LSKA into the third max pooling operation of SPPF, the output after LSKA processing is fed into the final convolution. LSKA can perform weighted processing on multi-scale feature maps, so that the model can pay more attention to the features related to the target and improve the detection accuracy. Specifically, the output of the large separable kernel attention LSKA is calculated by equations (3) to (6). First, the convolutional layer is initialized, and two convolutional layers are responsible for extracting the horizontal and vertical features of the input feature map respectively to generate a preliminary attention map. Then, spatial dilation convolution with different dilation rates is used to further extract features. Finally, the features obtained are fused through a convolutional layer to generate the final attention map. This attention map is multiplied by the original input feature map at the element level, so that each element in the original feature map is weighted according to the corresponding parameters of the attention, highlighting important features. (3)， (4)， (5)， (6)， in, This represents the output of a depthwise convolution, with a kernel size of (2... d -1)×(2 d -1); SPPF is an important component in the YOLOv8s model. It is used to extract multi-scale features. Through spatial pyramid pooling, SPPF can fuse feature maps of different scales together, thereby enhancing the model's robustness to changes in the target scale.

4. The improved target detection method based on YOLOv8s according to claim 1, characterized in that, In the neck network sampling stage, dynamic upsampling (DyUpsample) is introduced into RepGFPN to replace the original upsampling module. The improved neck structure is named GDFPN. Dynamic sampling bypasses dynamic convolution and constructs an upsampling structure from the perspective of point sampling. For a given feature map... The output of dynamic sampling is obtained by processing the input and sample set through the grid sample function. The process of obtaining the sample set can be expressed as equations (7) and (8), that is, using a linear layer and the sigmoid function to obtain the offset. O sampling set S That is, the offset and the original sampling grid. G The sum of: (7)， (8)， in, This represents the static factor, and we choose 0.

5.

5. The improved target detection method based on YOLOv8s according to claim 1, characterized in that, In the prediction phase, the localization branch uses DCNv2 and offset and mask generated from interactive features, while the classification branch uses interactive features to perform a dynamic feature selection strategy. The offset and mask are obtained from the output of the shared convolution. A portion of the output of the shared convolution is processed and aligned with the task output to jointly complete the dynamic feature selection, thereby obtaining the classification prediction output. After scaling, the final output is obtained.

Images