Vehicle and pedestrian detection method based on improved YOLOv8

Abstract

The invention discloses a vehicle and pedestrian detection method based on improved YOLOv8, and the method comprises the following steps: introducing a self-adaptive SE attention mechanism into a C2f module, and enhancing the extraction of key features; a double-path feature separation and fusion strategy is introduced into the C2f module, so that the detection precision of the model is improved; dynamic convolution optimization feature extraction and lightweight processing are introduced into the Bottleneck module, and global information collection is enhanced. According to the invention, by introducing an adaptive SE attention mechanism, a dual-path feature separation and fusion strategy and dynamic convolution optimization feature extraction and lightweight processing, the detection performance and calculation efficiency of the model are significantly improved; the method has remarkable advantages in the aspects of precision, calculation efficiency and model lightweight, and has wide application prospects in the fields of intelligent traffic monitoring, automatic driving systems, public safety and the like.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of intelligent transportation, automatic driving and public safety, and in particular to a vehicle and pedestrian detection method based on an improved YOLOv8. BACKGROUND

[0002] In the fields of intelligent transportation, automatic driving and public safety, vehicle and pedestrian detection technology plays a crucial role. With the rapid development of urban transportation and the increasing complexity of road conditions, high-precision and low-latency target detection has become a core requirement for intelligent transportation systems and autonomous driving technology. Traditional vehicle and pedestrian detection methods often rely on manually designed features and rules, which are difficult to adapt to complex and dynamic road environments, and perform poorly under different lighting, weather and occlusion conditions. In recent years, deep learning methods, especially convolutional neural networks, have significantly improved the accuracy and robustness of image recognition, becoming the main technical path to solve this problem. YOLO (You Only Look Once) as an efficient target detection algorithm, due to its excellent detection speed and accuracy, has been widely used in real-time target detection. YOLOv8, as a newer version of this series, has improved detection accuracy and speed compared to previous generations, but there is still room for optimization in multi-scale target detection, feature extraction in complex backgrounds, and detection performance in overlapping target situations. Traditional convolutional neural networks are limited in feature extraction due to fixed receptive fields, making it difficult to effectively capture long-distance dependencies, resulting in decreased detection performance in complex scenarios. Therefore, it is necessary to propose a method with higher detection accuracy. SUMMARY

[0003] The purpose of the present application is to provide a vehicle and pedestrian detection method based on an improved YOLOv8, aiming to improve detection accuracy and real-time performance, and provide a more efficient and stable detection solution for intelligent transportation and autonomous driving applications.

[0004] Technical solution: The vehicle and pedestrian detection method based on the improved YOLOv8 comprises the following steps:

[0005] (1) Introduce an adaptive SE attention mechanism in the C2f module to enhance the extraction of key features;

[0006] (2) Introduce a double-path feature separation and fusion strategy in the C2f module to improve the detection accuracy of the model;

[0007] (3) Introduce dynamic convolution optimization feature extraction and lightweight processing to the Bottleneck module to enhance the collection of global information.

[0008] Further, the step (1) is in channel feature compression, given an input feature map X, which has a shape of BxCxHxW,

[0009] Firstly, global average pooling is performed to extract global information of each channel, and the formula (1) is as follows:

[0010]

[0011] wherein s c represents the global average pooling value of channel c; X c (i,j) is the pixel value of the input feature map on channel c; H and W are the height and width of the feature map, respectively;

[0012] Secondly, two fully connected layers are used to calculate the channel attention weight, which is represented by formula (2):

[0013] z = σ (W2·δ (W1·s) ) (2)

[0014] wherein W1 is the first fully connected layer, W2 is the second fully connected layer, and r is the channel compression ratio. δ is the SiLU activation function.

[0015] Channel feature re-labeling re-applies the calculated channel attention weight z to the input feature X to adjust the channel response, which is represented by formula (3):

[0016]

[0017] wherein z c is the attention weight of channel c, is the adjusted feature map. After formula (3), the features of high-weight channels are enhanced, while the features of low-weight channels are suppressed.

[0018] Further, the step (1) adds an adaptive compression ratio to the SE module in the improved C2f module and applies it to the initial channel processing part.

[0019] Further, the adaptive compression ratio can adjust the compression degree according to the number of channels, and its expression is formula (4):

[0020]

[0021] wherein 2C' is the input channel number of the SE module. Even if the input channel number is small, formula (4) can ensure that the dimension after channel reduction will not be lower than 4.

[0022] Further, the step (2) divides the input feature into two different paths, respectively performs independent feature extraction, and finally fuses them.

[0023] Further, the step of dividing the input feature into two different paths comprises:

[0024] The input feature X is transformed in channel by 1x1 convolution, and then divided into two different branches, represented as formula (5):

[0025]

[0026] Wherein, X' is the channel number adjusted by 1x1 convolution; X1 and X2 represent two feature subsets after division, and each channel number is C';

[0027] Independent feature extraction is performed respectively, different processing methods are adopted for X1 and X2, deep feature extraction is performed in the main path, bottleneck structure is adopted for convolution calculation to extract deep feature information, and light feature extraction is adopted in the auxiliary path, and calculation is performed through dynamic convolution;

[0028] The features Y1 and Y2 processed through different paths need to be fused, the fusion operation adopts channel splicing and residual connection, and the calculation formula (6) is represented as:

[0029]

[0030] Through formula (6) residual connection, the network learns more stable feature information, and reduces the gradient vanishing problem.

[0031] Further, the step (3) dynamic convolution dynamically adjusts the convolution kernel according to the input image content, realizes more adaptive feature extraction, and the calculation formula (7) is:

[0032]

[0033] Wherein, K is the number of multiple dynamic convolution kernels, each convolution kernel W i is responsible for extracting different scale or feature representation; α i is a dynamic weight, α = Softmax(f(X)) is calculated by input feature through an attention network; X*W i represents that different convolution kernels are used for convolution operation on the input feature, and finally all convolution results are weighted and summed.

[0034] Further, the improved bottleneck module of the step (3) adopts dynamic convolution instead of standard convolution, the input feature is first reduced in dimension by 1x1 convolution, the calculation complexity is reduced, the calculation amount of 3x3 convolution layer is reduced, the channel dimension is adjusted, and different convolution kernels are used for learning different levels of features through multiple branch dynamic convolution, and the expression (8) is:

[0035] Yi = X' * W i , i = 1, 2, 3 (8)

[0036] The weight distribution is calculated by a lightweight MLP. After the weight distribution is completed, the most suitable feature component is adaptively selected using equation (7). Finally, the bottleneck optimization is completed through residual connection, which is represented by equation (9):

[0037] Y output = Y + X (9)

[0038] where X' = Conv 1×1 (X), W i represents a convolution kernel.

[0039] Further, the step (3) removes the channels that contribute less to target detection before convolution calculation through pruning strategy. L1 norm pruning is adopted, and ordinary 3x3 convolution is replaced by grouped convolution to retain sufficient feature expression ability.

[0040] Further, in the dynamic convolution bottleneck of the step (3), a variable receptive field mechanism is introduced, so that the model can adaptively select the appropriate convolution kernel size. For small target detection, a 3x3 small convolution kernel is used to extract detailed features. For large target detection, a 5x5 large convolution kernel is used to extract global information. Through parallel multi-scale branch combination, the model is more stable in different detection scenarios. After improvement, weighted residual is adopted, which is represented by equation (10):

[0041] Y output = λY + (1-λ)X (10)

[0042] where λ is the weight of the input feature.

[0043] Beneficial effects: Compared with the prior art, the present application has the following obvious advantages: the present application introduces an adaptive SE attention mechanism, introduces a double-path feature separation and fusion strategy, and optimizes feature extraction and lightweight processing through dynamic convolution, significantly improving the detection performance and computational efficiency of the model; the adaptive SE module is introduced, the channel attention mechanism is used to adaptively weight the features, which significantly enhances the expression ability of different channels, so that the network can more accurately capture key target features, thereby improving the model detection performance; the double-path feature separation and fusion strategy is used, combined with dynamic convolution optimization and residual connection, to enhance the multi-scale feature expression ability, significantly improve the detection accuracy, robustness and computational efficiency; in view of the limitations of fixed convolution kernel, a dynamic convolution optimization strategy is proposed, which enhances the multi-scale feature extraction ability by adaptively adjusting the convolution weight, and combines lightweight design to reduce the computational complexity, significantly improves the detection adaptability in complex scenes; the improved YOLOv8 method has obvious advantages in precision, computational efficiency and model lightweight, and has wide application prospects, such as intelligent traffic monitoring, automatic driving system and public safety. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is the model structure diagram of the present application;

[0045] Figure 2 is the flowchart of the method of the present application;

[0046] Figure 3 is the PR curve comparison chart. DETAILED DESCRIPTION

[0047] The technical solutions of the present application will be further described below in combination with the drawings.

[0048] As shown in Figure 1 , 2 , in order to solve the problems of low detection accuracy, large computational overhead and insensitivity to small targets in existing target detection technology, the present application discloses a YOLOv8 detection method based on improved C2f module, which improves feature expression ability through double-path processing, and improves the detection ability of the model to targets of different scales by combining channel splicing and residual connection mechanism, thereby improving the practical applicability of the model, including the following steps:

[0049] (1) Introduce an adaptive SE attention mechanism in the C2f module to enhance the extraction of key features;

[0050] In the C2f module, the SEBlock (Squeeze-and-Excitation Block) is introduced to enhance the expression ability of channel features. The SE module obtains the global information of each channel through global average pooling, and performs channel attention calculation through two fully connected layers, so as to adaptively weight the features. In the improvement of the C2f module of the application, an adaptive compression ratio is added to the SE module and applied to the initial channel processing part, so as to enhance the expression ability of different channel features and enable the network to more effectively capture the key features of the target.

[0051] The SE module mainly consists of three steps of compression, excitation and re-labeling. In the channel feature compression, given the input feature map X, the shape of which is BxCxHxW,

[0052] Firstly, global average pooling is performed to extract the global information of each channel, and the formula (1) is as follows:

[0053]

[0054] Wherein, s c represents the global average pooling value of channel c; X c (i, j) is the pixel value of the input feature map on channel c; H and W are the height and width of the feature map, respectively;

[0055] Secondly, two fully connected layers are used to calculate the channel attention weight, which is represented by formula (2):

[0056] z = σ (W2·δ (W1·s) ) (2)

[0057] Wherein, is the first fully connected layer, is the second fully connected layer, r is the channel compression ratio, and δ is the SiLU activation function;

[0058] The channel feature re-labeling re-applies the calculated channel attention weight z to the input feature X to adjust the channel response, which is represented by formula (3):

[0059]

[0060] Wherein, z c is the attention weight of channel c, is the adjusted feature map. After formula (3), the features of high weight channels are enhanced, and the features of low weight channels are suppressed.

[0061] In the improved C2f module, an adaptive compression ratio is added to the SE module and applied to the initial channel processing part.

[0062] The adaptive compression ratio can adjust the degree of compression according to the number of channels, and the expression is formula (4):

[0063]

[0064] Wherein, 2C' is the input channel number of the SE module, that is, even if the input channel number is small, formula (4) can ensure that the dimension after channel reduction will not be lower than 4. In this way, information loss and waste of computing resources can be prevented, a higher channel dimension is reserved, which is helpful for transmitting more feature information in the network and improving the recognition ability of the model.

[0065] Through adjustment and optimization for specific network structures, the adaptive SE module can automatically learn which channels are more important and enhance them, improve the discrimination ability of different target features, reduce background interference, so that the network can focus more on key features, thereby improving detection stability. Compared with the traditional SE module, the improved adaptive compression ratio method only introduces a small amount of full connection calculation, which can reduce information loss while ensuring computing efficiency, and improve the performance of the network in processing different tasks, and is more suitable for lightweight models such as YOLOv8.

[0066] (2) Introducing a double-path feature separation and fusion strategy in the C2f module to improve the detection accuracy of the model;

[0067] In the original C2f module, the feature flow is processed by a single path, that is, 1×1 convolution is used for preliminary channel compression, then a plurality of bottleneck modules are used for feature extraction, and finally 1×1 convolution is used for channel restoration. However, there are problems such as information flow limitation, insufficient feature mixing, and calculation redundancy. To solve the above problems, the present application improves the C2f structure and adopts a double-path feature separation and fusion strategy to improve the feature expression ability.

[0068] The core idea of the improvement is to divide the input features into two different paths, independently extract features, and fuse them in the last stage to improve the diversity and effectiveness of the information flow. The input features are divided into two different paths, independently extracted, and fused in the last stage.

[0069] The input features are divided into two different paths, including:

[0070] The input features X are subjected to 1×1 convolution for channel transformation, and then divided into two different branches, represented by formula (5):

[0071]

[0072] Wherein, X' is the channel number adjusted by 1×1 convolution; X1 and X2 represent two feature subsets after division, and the channel number of each is C'.

[0073] Independent feature extraction is performed respectively, different processing methods are adopted for X1 and X2, deep feature extraction is performed on the main path, Bottleneck structure is adopted for convolution calculation, and deep feature information is extracted; the auxiliary path adopts light feature extraction, and calculation is performed through dynamic convolution;

[0074] The features Y1 and Y2 processed through different paths need to be fused, the channel splicing and residual connection are adopted for the fusion operation, and the calculation formula (6) is represented as:

[0075]

[0076] Through the residual connection of formula (6), the network learns more stable feature information, and the gradient vanishing problem is reduced.

[0077] Compared with the traditional C2f structure, the double-path structure can capture different levels of features respectively, ensuring the diversity of information; the dynamic convolution is adopted to optimize the auxiliary path, improving the target detection ability of the model in complex background; through channel splicing and residual connection, the deep features and shallow features are fully fused, improving the detection precision. This improvement can effectively improve the performance of YOLOv8 in vehicle and pedestrian detection tasks, making it more robust and computationally efficient in complex environments.

[0078] (3) Dynamic convolution optimization feature extraction and light processing are introduced to the Bottleneck module to enhance the collection of global information.

[0079] In the YOLOv8 network, the Bottleneck module is the main feature extraction unit, and its structure includes 1x1 convolution, 3x3 convolution and residual connection. However, the fixed convolution kernel weight cannot be adaptively adjusted according to the different features of the input image, which limits the adaptability to different scales and backgrounds in complex scenes; the path feature extraction method may cause excessive emphasis on local information and insufficient use of global information, affecting the detection accuracy. In order to solve the above problems, the Bottleneck module is improved in the present application, dynamic convolution and light calculation optimization strategy are adopted to improve the feature extraction ability and reduce the calculation cost.

[0080] The weight of the traditional convolution is fixed in the entire inference process, while the dynamic convolution can dynamically adjust the convolution kernel according to the input image content, so as to realize more adaptive feature extraction, and the calculation formula (7) is:

[0081]

[0082] Wherein, K is the number of multiple dynamic convolution kernels, each convolution kernel W iResponsible for extracting different scales or feature representations; a i is a dynamic weight, a = Softmax(f(X)) is calculated by input features through an attention network; X*W i represents using different convolution kernels to convolve the input features, and finally summing all the convolution results with weights. Equation (7) uses multiple convolution kernels to learn features of different scales and different semantic levels, improving the robustness of target detection; and through adaptive weight adjustment, the model can dynamically adjust the calculation method according to the different input features, improving the feature extraction capability.

[0083] The improved Bottleneck module uses dynamic convolution instead of standard convolution. The input features are first reduced in dimension through 1x1 convolution, reducing the computational complexity and the calculation amount of the 3x3 convolution layer, and adjusting the channel dimension. The multi-branch dynamic convolution learns features of different levels through different convolution kernels, and the expression (8) is:

[0084] Y i = X' * W i , i = 1, 2, 3 (8)

[0085] The weight distribution is calculated through a lightweight MLP, after the weight distribution is completed, the most suitable feature component is selected adaptively using equation (7), and finally, the Bottleneck optimization is completed through residual connection, represented as equation (9):

[0086] Y output = Y + X (9)

[0087] Where X' = Conv 1×1 (X), W i represents the convolution kernel. In order to reduce the computational complexity, channel pruning and grouped convolution strategies are used in the dynamic convolution Bottleneck module. Before convolution calculation, the channels that contribute less to target detection are removed through pruning strategy to reduce redundant calculation amount. L1 norm pruning is used, and ordinary 3x3 convolution is replaced by grouped convolution to retain sufficient feature expression ability.

[0088] In the dynamic convolution Bottleneck, a variable receptive field mechanism is further introduced, so that the model can adaptively select the appropriate convolution kernel size. For small target detection, use 3x3 small convolution kernel to extract detailed features; for large target detection, use 5x5 large convolution kernel to extract global information; through parallel multi-scale branch combination, the model is more stable in different detection scenarios. The traditional residual connection directly adds the input and output features as equation (9), and the improved one uses weighted residual, which can be represented as equation (10):

[0089] Y output= λY + (1 - λ)X (10)

[0090] where λ is the weight of the input feature, and formula (10) adjusts the weight of the input feature to enable the network to adaptively select and strengthen key features, thereby improving the learning efficiency and generalization ability of the model. Compared with traditional residual connection, weighted residual effectively avoids overfitting, accelerates the convergence speed of training, and improves the expression ability of the network, so that the model can pay more attention to task-related features when processing complex tasks, ensuring the full use of deep features while avoiding excessive information expansion.

[0091] Table 1: Comparison of ablation experiment results

[0092]

[0093] As shown in Table 1, the improved C2f module YOLOv8 detection method is significantly better than the traditional YOLOv8 detection method. As shown in Table 1, the improved C2f module YOLOv8 detection method is significantly better than the traditional YOLOv8 detection method. Figure 3 As shown in Table 1, the improved C2f module YOLOv8 detection method is significantly better than the traditional YOLOv8 detection method. As shown in Table 1, the improved C2f module YOLOv8 detection method is significantly better than the traditional YOLOv8 detection method. As can be seen from the PR curve of figure (a), the detection performance of truck and car categories is good, indicating that the model can stably recognize these targets; the precision of biker and traffic light is moderate, and there may be some false positives or missed detection; the performance of pedestrian is the worst, indicating that the detection effect on pedestrians is relatively weak, which may be affected by factors such as pedestrian posture change and occlusion. The overall mAP@0.5 is 0.808, it is not difficult to see that the YOLOv8 model has good comprehensive detection performance under the threshold of 0.5IoU, but there is still room for improvement, especially in the pedestrian category. As can be seen from the PR curve of figure (b), after the improvement of C2F module, the detection accuracy of the model for each category is improved, and the improvement of pedestrian detection precision is the most obvious, indicating that the improved model has significantly enhanced generalization ability for the pedestrian category, and the detection stability has been improved. The overall mAP@0.5 precision is 0.869, indicating that it performs better in the overall precision-recall trade-off in the comprehensive target detection task, and the detection effect of each target is improved, which can more reliably detect a variety of targets in practical applications, reducing the situation of missed detection and false detection.

Claims

1. A vehicle and pedestrian detection method based on an improved YOLOv8, characterized in that, Includes the following steps: (1) An adaptive SE attention mechanism is introduced into the C2f module to enhance the extraction of key features; (2) Introduce a dual-path feature separation and fusion strategy in the C2f module to improve the detection accuracy of the model; (3) Dynamic convolution is introduced into the Bottleneck module to optimize feature extraction and lightweight processing, thereby enhancing the collection of global information.

2. The vehicle and pedestrian detection method based on improved YOLOv8 according to claim 1, characterized in that, In step (1) of channel feature compression, given an input feature map X with shape B×C×H×W, First, global average pooling is performed to extract global information for each channel, and its formula (1) is as follows: Among them, s c X represents the global average pooling value for channel c; c (i, j) represents the pixel value of the input feature map in channel c; H and W are the height and width of the feature map, respectively. Secondly, the channel attention weights are calculated after two fully connected layers, as expressed in equation (2): z=σ(W2·δ(W1·s)) (2) in, As the first fully connected layer, This is the second fully connected layer, where r is the channel compression ratio and δ is the SiLU activation function; Channel feature recalibration reapplies the calculated channel attention weights z to the input features X to adjust the channel response, as expressed in equation (3): Among them, z c It is the attention weight of channel c. It is the adjusted feature map. After Equation (3), the features of the high-weight channel are enhanced, while the features of the low-weight channel are suppressed.

3. The vehicle and pedestrian detection method based on improved YOLOv8 according to claim 1, characterized in that, Step (1) adds an adaptive compression ratio to the SE module in the improved C2f module and applies it to the initial channel processing section.

4. The vehicle and pedestrian detection method based on improved YOLOv8 according to claim 3, characterized in that, The adaptive compression ratio can adjust the degree of compression according to the number of channels, and its expression is Equation (4): Where 2C′ is the number of input channels of the SE module. Even if the number of input channels is small, Equation (4) can ensure that the dimension after channel reduction will not be less than 4.

5. The vehicle and pedestrian detection method based on improved YOLOv8 according to claim 1, characterized in that, Step (2) divides the input features into two different paths, extracts features independently, and then fuses them in the final stage.

6. The vehicle and pedestrian detection method based on improved YOLOv8 according to claim 5, characterized in that, The process of dividing the input features into two different paths includes: The input feature X undergoes a 1×1 convolution for channel transformation, and is then divided into two different branches, as shown in equation (5): Where X′ is the number of channels adjusted by 1×1 convolution; X1 and X2 represent two feature subsets after partitioning, each with C′ channels; Independent feature extraction is performed separately for X1 and X2, with different processing methods. The main path performs deep feature extraction using a Bottleneck structure for convolution calculation to extract deep feature information; the auxiliary path uses lightweight feature extraction, calculated through dynamic convolution. Features Y1 and Y2, after being processed by different paths, need to be fused. The fusion operation uses channel splicing and residual connection, and the calculation formula (6) is expressed as follows: By using residual connections in equation (6), the network learns more stable feature information and reduces the gradient vanishing problem.

7. The vehicle and pedestrian detection method based on improved YOLOv8 according to claim 1, characterized in that, The dynamic convolution step (3) dynamically adjusts the convolution kernel according to the content of the input image to achieve more adaptive feature extraction. The calculation formula (7) is as follows: Where K is the number of multiple dynamic convolutional kernels, and each convolutional kernel W i Responsible for extracting representations at different scales or features; α i The weights are dynamic, α = Softmax(f(X)) is calculated from the input features through an attention network; X*W i This means using different convolution kernels to perform convolution operations on the input features, and finally summing all the convolution results by weight.

8. The vehicle and pedestrian detection method based on improved YOLOv8 according to claim 1, characterized in that, The improved Bottleneck module in step (3) uses dynamic convolution instead of standard convolution. The input features are first reduced in dimensionality by 1×1 convolution to reduce computational complexity and the computational cost of 3×3 convolutional layers. The channel dimension is adjusted, and multi-branch dynamic convolution learns features at different levels through different convolution kernels. The expression (8) is: Y i =X′*W i ,i=1,2,3 (8) Weight assignment is calculated using a lightweight MLP. After weight assignment, the most suitable feature components are adaptively selected using equation (7). Finally, Bottleneck optimization is completed through residual connections, as expressed in equation (9): AND output =Y+X (9) Where, X′=Conv 1×1 (X), W i This represents the convolution kernel.

9. The vehicle and pedestrian detection method based on improved YOLOv8 according to claim 1, characterized in that, In step (3), before the convolution calculation, channels that contribute less to target detection are removed by a pruning strategy. L1 norm pruning is used, and ordinary 3×3 convolution is replaced by grouped convolution to retain sufficient feature expression capabilities.

10. The vehicle and pedestrian detection method based on improved YOLOv8 according to claim 1, characterized in that, In step (3), a variable receptive field mechanism is introduced in the dynamic convolution Bottleneck, which enables the model to adaptively select an appropriate convolution kernel size. For small target detection, a 3×3 small convolution kernel is used to extract detailed features. For large target detection, a 5×5 large convolutional kernel is used to extract global information; by combining parallel multi-scale branches, the model becomes more stable in different detection scenarios. The improved model uses weighted residuals, expressed as Equation (10): Y output =λY+(1-λ)X (10) Where λ is the weight of the input feature, and X and Y are the input feature values.

Images