Pedestrian target detection method and system based on SVDO-YOLO network
By introducing a two-way multi-layer fusion network and dynamic convolution into the YOLOv8 network, the backbone network is optimized, and the scale changes and dense occlusion problems of YOLOv8 in pedestrian detection are solved, improving detection accuracy and efficiency.
Patent Information
- Application Number
- CN202510350398.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-24
- Publication Date
- 2025-07-08
AI Technical Summary
The YOLOv8 algorithm faces the difficulty of pedestrian detection at small scale and the detection challenges in dense occlusion scenarios in pedestrian detection tasks, resulting in insufficient detection accuracy and robustness.
Based on the YOLOv8 network architecture, a two-way multi-layer fusion network is designed. By building a dynamic intensive feature extraction module and dynamic convolution, combined with the MobileNetV3 network, the backbone network is optimized, and the feature extraction and fusion capabilities are improved, and scale changes and dense occlusion problems are solved.
The detection accuracy of small-scale pedestrians and densely blocked pedestrians is improved, the calculation complexity and parameter quantity are reduced, and the adaptability and efficiency of the model are enhanced.
Smart Images
Figure CN120279579A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of pedestrian target detection, and particularly relates to a pedestrian target detection method and system based on the SVDO-YOLO network. Background Art
[0002] The statements in this part only provide background technical information related to the present invention and do not necessarily constitute prior art.
[0003] Pedestrian detection has always been a hot issue attracting much attention in the field of computer vision, and the depth and breadth of its research are continuously expanding. The value of pedestrian detection technology lies not only in the technology itself but also in its broad application prospects. When combined with technologies such as pedestrian tracking and pedestrian re-identification, this technology can significantly improve the performance and practicality of intelligent systems. Pedestrian detection plays an indispensable role in autonomous driving systems, intelligent robot fields, and intelligent video surveillance systems. In addition, in many fields such as human behavior analysis, passenger flow statistics systems, and intelligent transportation, pedestrian detection technology also demonstrates great application potential and value.
[0004] As a mainstream deep learning one-stage detection algorithm, the YOLO series of algorithms has been widely used in various scenarios. Among them, YOLOv8 has become the first choice for many applications due to its high efficiency and comprehensiveness.
[0005] Although the YOLOv8 algorithm performs well in many aspects, it still faces some limitations when dealing with pedestrian detection tasks, resulting in its performance not being fully exerted. The changes in shooting distance and perspective make pedestrians present diverse scales in the image. Large-scale pedestrians are relatively easy to detect due to rich information, but small-scale pedestrians often face the dilemma of missed detection due to low pixels, blurred contours, and scarce information. In addition, the occlusion problem in crowded scenes is also a major challenge. When multiple pedestrians gather closely, the occlusion phenomenon between them is inevitable, which not only increases the difficulty of detection but also easily causes the algorithm to confuse the occluder with the pedestrian itself or completely miss the occluded pedestrian. These problems not only test the recognition accuracy of the algorithm but also highlight the urgent need to improve the robustness and adaptability in complex scenarios. Summary of the Invention
[0006] To overcome the deficiencies of the above-mentioned prior art, the present invention provides a pedestrian target detection method and system based on the SVDO-YOLO network. Based on the YOLOv8 network architecture, a bidirectional multi-layer fusion network is designed to specifically solve the problems of scale change and dense occlusion in the field of pedestrian detection.
[0007] To achieve the above object, one or more embodiments of the present invention provide the following technical solutions:
[0008] The first aspect of the present invention provides a pedestrian target detection method based on the SVDO - YOLO network, including:
[0009] Obtain pedestrian target images;
[0010] Pre - process the pedestrian target images to obtain pre - processed pedestrian target images;
[0011] Construct a pedestrian target detection model, and input the pre - processed pedestrian target images into the pedestrian target detection model for training to obtain a trained pedestrian target detection model;
[0012] Use the trained pedestrian target detection model to detect the pedestrian target to be detected;
[0013] Among them, when inputting the pre - processed pedestrian target images into the pedestrian target detection model for training, the pedestrian target detection model is based on the YOLOv8 network architecture, constructs a new bidirectional multi - layer fusion network based on the bidirectional feature pyramid network, constructs a dynamic dense feature extraction module to replace the second C2f module in the backbone network and the neck network C2f module, and constructs a dense feature extraction module to replace the C2f module in the head network. The specific process is as follows:
[0014] Input the target images into the improved backbone network for feature extraction and enhancement to obtain multi - scale key feature maps;
[0015] Input the multi - scale key features into the neck network for feature fusion and enhancement to obtain the final output feature maps;
[0016] Input the final output feature maps into the head network for detection to obtain pedestrian target detection results.
[0017] As an implementation, inputting the target images into the improved backbone network for feature extraction and enhancement to obtain multi - scale key feature maps, the specific process is as follows:
[0018] In the backbone network, input the pictures into two convolutional layers in sequence for downsampling to obtain low - level feature maps;
[0019] Input the low - level features into the C2f module for feature aggregation and compression to obtain aggregated features;
[0020] Input the aggregated features into a convolutional layer for further downsampling and dimensionality reduction to obtain deep - layer feature representations;
[0021] Input the deep - layer features into the first dynamic dense feature extraction module for dynamic feature extraction to obtain refined features;
[0022] Input the refined features into 9 MobileNetV3 modules in sequence for feature enhancement to obtain multi - scale key features.
[0023] As an implementation, the deep features are input into the first dynamic dense feature extraction module to perform dynamic feature extraction to obtain refined features. Each dynamic dense feature extraction module includes two convolutional layers, a segmentation module, and multiple dynamic-bottleneck modules. The specific process is as follows:
[0024] The deep features are input into the convolutional layer to expand the number of channels, obtaining expanded features.
[0025] The expanded features are input into the segmentation module to obtain two feature groups.
[0026] The first feature group is sequentially input into multiple dynamic-bottleneck modules to obtain deeper features respectively;
[0027] The deeper features are concatenated with the second feature group to obtain fused features;
[0028] The fused features are input into the convolutional layer to obtain refined features.
[0029] As an implementation, the refined features are sequentially input into 9 MobileNetV3 modules. The MobileNetV3 module includes three convolutional layers, a squeeze-and-excitation module;
[0030] In the first MobileNetV3 module, the refined features are sequentially input into two convolutional layers to obtain dimension-increased lightweight features;
[0031] The dimension-increased lightweight features are input into the squeeze-and-excitation module for channel weighting to obtain weighted activation features;
[0032] The weighted activation features are input into the third convolutional layer to obtain multi-scale key features.
[0033] As an implementation, the multi-scale key features are input into the neck network for feature fusion and enhancement to obtain the final output feature map. The specific process is as follows:
[0034] In the neck network, the aggregated features output by the C2f module, the refined features output by the first dynamic dense feature extraction module, and the multi-scale key features output by the fourth MobileNetV3 module are input into the first network to obtain enhanced fused features;
[0035] According to the enhanced fused features, cross-layer fused features are obtained through the processing of the second network;
[0036] According to the cross-layer fused features, the final input features are obtained through the processing of the third network.
[0037] As an implementation, the aggregated features output by the C2f module, the refined features output by the first dynamic dense feature extraction module, and the multi-scale key features output by the fourth MobileNetV3 module are input into the first network. The specific process is as follows:
[0038] The multi-scale key features obtained from the fourth MobileNetV3 module are simultaneously input into the first SPPF module to obtain the first pooled feature;
[0039] The first pooled feature is upsampled and concatenated with the refined features output by the first dynamic dense feature extraction module to obtain the first fused feature;
[0040] The first fused feature is input into the second dynamic dense feature extraction module for deep feature extraction to obtain prominent features;
[0041] The prominent features are upsampled and concatenated with the aggregated features output by the C2f module to obtain the second fused feature;
[0042] The second fused feature is input into the third dynamic dense feature extraction module for deep feature extraction to obtain enhanced fused features.
[0043] As an implementation, according to the enhanced fused features, cross-layer fused features are obtained through the processing of the second network. The specific process is as follows:
[0044] The multi-scale key features output by the ninth MobileNetV3 module are input into the second SPPF module to obtain the second pooled feature;
[0045] The second pooled feature is upsampled and concatenated with the key features output by the fourth MobileNetV3 module to obtain the third fused feature;
[0046] The third fused feature is input into the fourth dynamic dense feature extraction module for feature enhancement to obtain the first enhanced feature;
[0047] The first enhanced feature is upsampled and concatenated with the refined features of the first dynamic dense feature extraction module and the refined features output by the second dynamic dense feature extraction module to obtain the fourth fused feature;
[0048] The fourth fused feature is input into the fifth dynamic dense feature extraction module for feature extraction to obtain integrated features;
[0049] The integrated features are upsampled and concatenated with the aggregated features output by the C2f module and the refined features output by the third dynamic dense feature extraction module to obtain the fifth fused feature;
[0050] The fifth fused feature is input into the dense feature extraction module to obtain cross-layer fused features.
[0051] As an implementation manner, according to the cross-layer fusion feature, the final input feature is obtained through the third network processing. The specific process is as follows:
[0052] The cross-layer fusion feature output by the dense feature extraction module is input into the convolutional layer for downsampling to obtain the deep feature;
[0053] The deep feature is concatenated with the refined feature output by the second dynamic dense feature extraction module and the refined feature output by the fourth dynamic dense feature extraction module to obtain the sixth fusion feature;
[0054] The sixth fusion feature is input into the sixth dynamic dense feature extraction module for feature extraction to obtain the second enhanced feature;
[0055] The second enhanced feature is input into the DynamicConv module for dynamic downsampling to obtain the dynamic fusion feature;
[0056] The dynamic fusion feature is input into the seventh dynamic dense feature extraction module for dynamic feature extraction to obtain the final output feature.
[0057] As an implementation manner, the second enhanced feature is input into the DynamicConv module for dynamic downsampling to obtain the dynamic fusion feature. The specific process is as follows:
[0058] The second enhanced feature is input into the attention module to obtain the attention-weighted feature;
[0059] The attention-weighted feature is input into multiple convolutional layers to obtain branch features respectively;
[0060] The branch features are concatenated to obtain the comprehensive feature;
[0061] The obtained comprehensive feature and the refined feature output by the sixth dynamic dense feature extraction module are input into the convolutional layer and normalized to obtain the dynamic fusion feature.
[0062] The second aspect of the present invention provides a pedestrian target detection system based on the SVDO-YOLO network, including:
[0063] A data acquisition module for acquiring pedestrian target images;
[0064] A data preprocessing module for preprocessing the pedestrian target image to obtain the preprocessed pedestrian target image;
[0065] A model construction and training module for constructing a pedestrian target detection model and inputting the preprocessed pedestrian target image into the pedestrian target detection model for training to obtain the trained pedestrian target detection model;
[0066] Among them, the preprocessed pedestrian target image is input into the pedestrian target detection model for training. The pedestrian target detection model is based on the YOLOv8 network architecture and constructs a new bidirectional multi-layer fusion network based on the bidirectional feature pyramid network. A dynamic dense feature extraction module is constructed to replace the second C2f module in the backbone network and the C2f module in the neck network, and a dense feature extraction module is constructed to replace the C2f module in the head network. The specific process is as follows:
[0067] The target image is input into the improved backbone network for feature extraction and enhancement to obtain a multi-scale key feature map;
[0068] The multi-scale key features are input into the neck network for feature fusion and enhancement to obtain the final output feature map;
[0069] The final output feature map is input into the head network for detection to obtain the pedestrian target detection result;
[0070] The model detection module is used to detect the pedestrian target to be detected by using the trained pedestrian target detection model.
[0071] The above one or more technical solutions have the following beneficial effects:
[0072] In this embodiment, to address the problem of pedestrian scale variation, an architecture called Bidirectional Multi-Layer Fusion Network (BMFN) is proposed. By designing multiple feature fusion paths and implementing bidirectional feature fusion, this architecture can enhance the algorithm's ability to extract and utilize feature information at different levels, thereby improving its adaptability to scale changes and effectively solving the challenges brought by pedestrian scale variation.
[0073] In this embodiment, to tackle the problem of dense occlusion, the Bottleneck structure in the C2f module is improved by introducing the DenseNet network architecture, and a Dense Feature Extraction Module (DeFM) and a Dynamic Dense Feature Extraction Module (DyFM) are designed. This improvement enhances the efficiency of feature reuse and propagation, enabling the model to capture key feature information more accurately. At the same time, Dynamic Convolution is incorporated into the algorithm structure, effectively reducing the computational complexity of the SVDO-YOLO algorithm.
[0074] In this embodiment, to address the potential increase in the number of parameters and computational complexity after algorithm improvement, the MobileNetV3 network architecture is introduced to optimize the Backbone architecture. This measure not only significantly reduces the number of parameters and computational complexity of the SVDO-YOLO algorithm but also further improves the deployment convenience and running efficiency of the model in practical applications.
[0075] Advantages of additional aspects of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0076] The accompanying drawings forming a part of this specification are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation to the present invention.
[0077] Figure 1 It is a schematic diagram of the pedestrian target detection solution based on the SVDO - YOLO network for the first embodiment;
[0078] Figure 2 It is a flowchart of the pedestrian target detection method based on the SVDO - YOLO network for the first embodiment;
[0079] Figure 3 It is an architecture diagram of the SVDO - YOLO network for the first embodiment;
[0080] Figure 4 It is a structure diagram of the Bidirectional Multi - layer Fusion Network (BMFN) for the first embodiment;
[0081] Figure 5 It is a structure diagram of the Dense Feature Extraction Module (DeFM) for the first embodiment;
[0082] Figure 6 It is a structure diagram of the Dynamic Dense Feature Extraction Module (DyFM) for the first embodiment;
[0083] Figure 7 It is a structure diagram of the Dynamic Convolution for the first embodiment;
[0084] Figure 8 It is a structure diagram of the MobileNetV3 network for the first embodiment;
[0085] Figure 9 It is a comparison diagram of the pedestrian scale change detection effect for the first embodiment;
[0086] Figure 10 It is a comparison diagram of the pedestrian dense occlusion detection effect for the first embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0087] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.
[0088] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention.
[0089] In the case of no conflict, the embodiments in the present invention and the features in the embodiments may be combined with each other.
[0090] Embodiment 1
[0091] This embodiment discloses a pedestrian target detection method based on the SVDO - YOLO network.
[0092] To describe this embodiment more clearly, the implementation process of pedestrian target detection based on the SVDO - YOLO network can be specifically described as follows:
[0093] The pedestrian target detection method based on the SVDO - YOLO network includes:
[0094] S1. Obtain a pedestrian target image;
[0095] S2. Preprocess the pedestrian target image to obtain a preprocessed pedestrian target image;
[0096] S3. Build a pedestrian target detection model, and input the preprocessed pedestrian target image into the pedestrian target detection model for training to obtain a trained pedestrian target detection model;
[0097] S4. Use the trained pedestrian target detection model to detect the pedestrian target to be detected.
[0098] As Figure 2 、 Figure 3 shown, in step S1, a pedestrian target image is obtained.
[0099] In this embodiment, pedestrian target images with multi - scales and dense occlusions are obtained from the Crowdhuman dataset and the Widerperson dataset to construct a new pedestrian dataset; the dataset is divided into a training set, a validation set, and a test set according to the ratio of 7:2:1.
[0100] As Figure 2 、 Figure 3 shown, in step S2, the pedestrian target image is preprocessed to obtain a preprocessed pedestrian target image.
[0101] First, the image is scaled to 640×640 proportionally, then normalized and standardized, and finally data augmentation is performed through techniques such as random cropping, rotation, and Mosaic augmentation.
[0102] As Figure 2 、 Figure 3As shown, in step S3, a pedestrian target detection model is constructed, and the preprocessed pedestrian target image is input into the pedestrian target detection model for training to obtain the trained pedestrian target detection model.
[0103] S3-1. Construct a pedestrian target detection model.
[0104] In this embodiment, a Bidirectional Multi-Layer Fusion Network (BMFN) is designed to improve the YOLOv8 network architecture as the core, and an SVDO-YOLO network model is constructed.
[0105] Based on the YOLOv8 network architecture, the idea of the Bidirectional Feature Pyramid Network (BiFPN) is integrated to construct the BMFN, as shown in the appendix. Figure 4 As shown. While retaining the YOLOv8 detection framework, the BMFN additionally introduces an upsampling branch from the P4 layer of the backbone network, adds a detection layer with a resolution of 160×160, and optimizes the detection scale layout, canceling the 20×20 low-resolution detection points.
[0106] In addition, a DeFM is designed to replace the C2f module before the 160×160 detection head, a DyFM is designed to replace the second C2f module in the backbone network and the remaining C2f modules in the neck network, and dynamic convolution is used to replace the last-layer downsampling; as shown in the appendix. Figure 5 As shown, in the designed DeFM, a Dense_Bottleneck is used to replace the Bottleneck component in the C2f module; as shown in the appendix. Figure 6 As shown, in the designed DyFM, a Dynamic_Bottleneck is used to replace the Bottleneck component in the C2f module; the structure of dynamic convolution is shown in the appendix. Figure 7 As shown, it is integrated into the last-layer downsampling.
[0107] Finally, the P4 and P5 layers of the backbone network are replaced with the MobileNetV3 module, and the MobileNetV3 network structure is shown in the appendix. Figure 8 As shown.
[0108] (1) Based on the architecture of the YOLOV8 network model, construct a Bidirectional Multi-Layer Fusion Network (BMFN).
[0109] Based on the YOLOv8 network architecture, the BMFN is designed by combining the Bidirectional Feature Pyramid Network (BiFPN). While retaining the YOLOv8 detection framework, the BMFN additionally introduces an upsampling branch from the P4 layer, adds a detection layer with a resolution of 160×160, and optimizes the detection scale layout, canceling the 20×20 low-resolution detection points.
[0110] The designed BMFN effectively reduces the problem of small-scale target feature loss caused by excessive downsampling, thereby improving the model's detection sensitivity and accuracy for small-scale targets. In addition, BMFN fully utilizes the advantages of BiFPN and achieves seamless connection and efficient use of feature information at different levels through its bidirectional feature fusion mechanism, which not only improves the overall quality of the feature map, but also enables the model to more comprehensively capture and analyze the multi-scale key features of pedestrians.
[0111] Replace the architecture of the YOLOV8 network model with BMFN.
[0112] (2) Construct a dense feature extraction module and a dynamic dense feature extraction module, and replace the second C2f module in the backbone network and the C2f module of the neck network with the dynamic dense feature extraction module, replace the last layer downsampling with dynamic convolution, and replace the C2f module of the head network with the dense feature extraction module.
[0113] In the C2f module, the bottleneck module is improved by using a deep convolutional neural network structure to design a dense feature extraction module (DeFM). Furthermore, dynamic convolution is integrated into DeFM to construct a dynamic dense feature extraction module (DyFM), and dynamic convolution is integrated into the last layer of downsampling.
[0114] Specifically, we use DenseNet to reconstruct the Bottleneck module in the C2f module and design a new Dense_Bottleneck. Furthermore, we introduce dynamic convolution to replace the convolution in the Dense_Bottleneck module and design Dynamic_Bottleneck. Dynamic convolution can dynamically generate and adjust the convolution kernel weights, making it more flexible to adapt to target objects of different scales, while effectively reducing the computational burden of the model.
[0115] In the designed DeFM, Dense_Bottleneck is used to replace the Bottleneck component in the C2f module. DeFM can capture more features of densely occluded pedestrians by maximizing the flow and reuse of feature information.
[0116] In the designed DyFM, Dynamic_Bottleneck is used to replace the Bottleneck component in the C2f module, and dynamic convolution is integrated into the final layer downsampling. This improvement not only maintains the advantage of feature information utilization, but also significantly reduces the computational cost, showing stronger adaptability and higher detection efficiency when dealing with complex scenes such as densely occluded pedestrians.
[0117] Replace the C2f module before the 160×160 detection head with DeFM, replace the second C2f module in the backbone network and the remaining C2f modules in the neck network with DyFM, and replace the last-layer downsampling with dynamic convolution.
[0118] (3) Introduce the MobileNetV3 network architecture into the backbone network.
[0119] Introduce the MobileNetV3 network, which ingeniously combines 1×1 convolution, depthwise convolution, and the Squeeze-and-Excitation (SE) attention mechanism. The input image first undergoes 1×1 convolution for channel expansion, and then depthwise convolution is performed in the high-dimensional space to extract features. The SE mechanism applies global average pooling on these feature maps and calculates the weights of each channel through two fully connected layers with different activation functions. These weights are then used to recalibrate the original feature maps, enabling the model to focus more on key information.
[0120] Replace the P4 and P5 layers of the backbone network with MobileNetV3 modules.
[0121] After the above steps, a pedestrian target detection model based on the SVDO-YOLO network is constructed.
[0122] S3-2. Input the preprocessed pedestrian target images into the pedestrian target detection model for training to obtain the trained pedestrian target detection model.
[0123] Input the 640×640 RGB image tensor normalized to [0,1] into the pedestrian detection model and train it in combination with the annotated bounding boxes and class labels. The model learns features by optimizing the combined loss (classification, localization, confidence) and outputs a tensor containing the predicted box coordinates, confidence, and class probabilities. In the post-processing stage, the coordinates are decoded to the original image size, and low-confidence predictions are filtered, ultimately achieving high-precision pedestrian target detection and localization.
[0124] Divide the preprocessed pedestrian detection dataset into a training set, a validation set, and a test set according to the ratio of 7:2:1.
[0125] Set the number of training epochs to 200, and the number of input images per time is 16.
[0126] Set up the experimental environment, set the experimental parameters, and input the images into SVDO-YOLO.
[0127] Among them, the experimental environment of this example is built, and the configuration environment in this embodiment is as follows: the operating system is Ubuntu 22.04, the GPU is RTX 4090 (24G), the CPU is AMD EPYC 9654, Pytorch 2.1.0, CUDA 12.1, and Python 3.10.
[0128] The specific training process of the SVDO-YOLO pedestrian target detection model is as follows:
[0129] S3-2-1. Input the target image into the improved backbone network for feature extraction and enhancement to obtain multi-scale key feature maps.
[0130] As Figure 3 shown, the specific process is as follows:
[0131] (1) In the backbone network, input the picture into two convolutional layers in sequence for downsampling to obtain low-level feature maps.
[0132] (2) Input the low-level features into the C2f module for feature aggregation and compression to obtain aggregated features.
[0133] (3) Input the aggregated features into the convolutional layer for further downsampling and dimensionality reduction to obtain deep feature representations.
[0134] (4) Input the deep features into the first dynamic dense feature extraction module for dynamic feature extraction to obtain refined features.
[0135] As Figure 6 shown, each dynamic dense feature extraction module includes two convolutional layers, a segmentation module, and multiple dynamic-bottleneck modules. The specific process is as follows:
[0136] 1) Input the deep features into the convolutional layer to expand the number of channels to obtain extended features.
[0137] 2) Input the extended features into the segmentation module to obtain two feature groups.
[0138] 3) Input the first feature group into multiple dynamic-bottleneck modules in sequence to obtain deeper features respectively.
[0139] The dynamic-bottleneck module includes four DynamicConv layers, and the output of each previous DynamicConv layer is concatenated as the input of the next DynamicConv layer.
[0140] 4) Concatenate the deeper features with the second feature group to obtain fused features.
[0141] 5) Input the fused features into the convolutional layer to obtain refined features.
[0142] (5) Input the refined features into 9 MobileNetV3 modules in sequence to obtain multi-scale key features through feature enhancement.
[0143] As Figure 8 shown, in the MobileNetV3 module, it includes three convolutional layers, a squeeze-and-excitation module.
[0144] Input the refined features into 9 MobileNetV3 modules in sequence, where the MobileNetV3 module includes three convolutional layers, a squeeze-and-excitation module;
[0145] In the first MobileNetV3 module, input the refined features into two convolutional layers in sequence to obtain upsampled lightweight features;
[0146] Input the upsampled lightweight features into the squeeze-and-excitation module for channel weighting to obtain weighted activation features;
[0147] Input the weighted activation features into the third convolutional layer to obtain multi-scale key features.
[0148] Among them, the squeeze-and-excitation module includes a squeeze module, an excitation module, an average pooling layer, and a fully connected layer. The specific process is as follows:
[0149] 1) Input diverse features into the squeeze module for channel dimensionality reduction, and input them into the average pooling layer to extract global information to obtain squeezed features.
[0150] 2) Input the squeezed features into the fully connected layer for feature reconstruction and dimensionality transformation to obtain fully connected output features.
[0151] 3) Weight the fully connected output features and input them into the excitation module to obtain weighted activation features.
[0152] 4) Input the weighted activation features into the convolutional layer for channel compression and recombination to obtain multi-scale aggregated features.
[0153] S3-2-2. Input the multi-scale key features into the neck network for feature fusion and enhancement to obtain the final output feature map.
[0154] As Figure 3 shown, the specific process is as follows:
[0155] (1) In the neck network, input the aggregated features output by the C2f module, the refined features output by the first dynamic dense feature extraction module, and the multi-scale key features output by the fourth MobileNetV3 module into the first network to obtain enhanced fusion features.
[0156] The specific process is as follows:
[0157] 1) Input the multi-scale key features obtained from the fourth MobileNetV3 module into the first SPPF module simultaneously to obtain the first pooled feature.
[0158] 2) Upsample the first pooled feature and concatenate it with the refined feature output by the first dynamic dense feature extraction module to obtain the first fused feature.
[0159] 3) Input the first fused feature into the second dynamic dense feature extraction module for deep feature extraction to obtain the prominent feature.
[0160] 4) Upsample the prominent feature and concatenate it with the aggregated feature output by the C2f module to obtain the second fused feature.
[0161] Input the second fused feature into the third dynamic dense feature extraction module for deep feature extraction to obtain the enhanced fused feature.
[0162] (2) Obtain the cross-layer fused feature through the second network processing based on the enhanced fused feature.
[0163] The specific process is as follows:
[0164] 1) Input the multi-scale key features output by the ninth MobileNetV3 module into the second SPPF module to obtain the second pooled feature.
[0165] 2) Upsample the second pooled feature and concatenate it with the key features output by the fourth MobileNetV3 module to obtain the third fused feature.
[0166] 3) Input the third fused feature into the fourth dynamic dense feature extraction module for feature enhancement to obtain the first enhanced feature.
[0167] 4) Upsample the first enhanced feature and concatenate it with the refined features output by the first dynamic dense feature extraction module and the second dynamic dense feature extraction module to obtain the fourth fused feature.
[0168] 5) Input the fourth fused feature into the fifth dynamic dense feature extraction module for feature extraction to obtain the integrated feature.
[0169] 6) Upsample the integrated feature and concatenate it with the aggregated feature output by the C2f module and the refined features output by the third dynamic dense feature extraction module to obtain the fifth fused feature.
[0170] 7) Input the fifth fused feature into the dense feature extraction module to obtain the cross-layer fused feature, that is, the large target.
[0171] Such as Figure 5As shown in the figure, the dense feature extraction module includes two convolutional layers and a segmentation module, namely multiple dense-bottleneck modules. The specific process is as follows:
[0172] 1) Input the fifth fused feature into the convolutional layer for channel expansion to obtain the upsampled feature.
[0173] 2) Input the upsampled feature into the segmentation module for segmentation to obtain two feature groups.
[0174] 3) Input the third feature group into multiple dynamic-bottleneck modules in sequence to obtain deeper features respectively.
[0175] In the dense-bottleneck module, it includes four convolutional layers. The output of each previous convolutional layer is concatenated as the input of the next convolutional layer.
[0176] 4) Concatenate the deeper features with the fourth feature group to obtain the enhanced fused feature.
[0177] 5) Input the enhanced fused feature into the convolutional layer to obtain the cross-layer fused feature.
[0178] (3) According to the cross-layer fused feature, obtain the final input feature through the processing of the third network.
[0179] Among them, in the third network, the specific process is as follows:
[0180] 1) Input the cross-layer fused feature output by the dense feature extraction module into the convolutional layer for downsampling to obtain the deep feature.
[0181] 2) Concatenate the deep feature with the refined feature output by the second dynamic dense feature extraction module and the refined feature output by the fifth dynamic dense feature extraction module to obtain the sixth fused feature.
[0182] 3) Input the sixth fused feature into the sixth dynamic dense feature extraction module to obtain the refined feature, that is, the medium target.
[0183] 4) Input the refined feature output by the sixth dynamic dense feature extraction module into the DynamicConv module for dynamic downsampling to obtain the dynamic fused feature.
[0184] In the DynamicConv module, input the second enhanced feature into the attention module to obtain the attention-weighted feature;
[0185] Input the attention-weighted feature into multiple convolutional layers to obtain branch features respectively;
[0186] Concatenate the branch features to obtain the comprehensive feature;
[0187] The comprehensive features and the refined features output by the sixth dynamic dense feature extraction module are input into a convolutional layer and normalized to obtain dynamic fusion features.
[0188] 5) Concatenate the dynamic comprehensive features, the first pooling features output by the first SPPF module, and the refined features output by the fourth dynamic dense feature extraction module to obtain the seventh fusion feature.
[0189] 6) Input the seventh fusion feature into the seventh dynamic dense feature extraction module for dynamic feature extraction to obtain refined features, i.e., small targets.
[0190] S3-2-3: Input the final output feature map into the head network for detection to obtain pedestrian target detection results.
[0191] In the head network, input the large targets output by the dense feature extraction module into the first detection module for detection.
[0192] Input the medium targets output by the sixth dynamic dense feature extraction module into the second detection module for detection.
[0193] Input the small targets output by the seventh dynamic dense feature extraction module into the third detection module for detection.
[0194] After the above steps, a trained pedestrian target detection model based on the SVDO-YOLO network is obtained, which can more accurately detect multi-scale and densely occluded pedestrian targets and ensure the accuracy of subsequent pedestrian target detection.
[0195] S3-3: Output the training results of SVDP-YOLO and test and evaluate the trained weights.
[0196] Use precision, recall, and mean average precision to evaluate the overall detection effect of the model.
[0197] Use precision to measure the accuracy of the model's prediction of positive classes. The formula is:
[0198]
[0199] Use recall to evaluate the model's ability to capture all actual positive classes. The formula is:
[0200]
[0201] Use mean average precision (mAP) to comprehensively quantify the overall performance of the model in multi-class detection. The formula is:
[0202]
[0203] Among them, TP means that the model correctly identifies a pedestrian object as a pedestrian, FP means that the model identifies a non-pedestrian object as a pedestrian, and FN means the actual pedestrians that the model fails to successfully identify.
[0204] As Figure 2 、 Figure 3 shown, in step S4, the trained pedestrian target detection model is used to detect the pedestrian target to be detected.
[0205] Specifically, a 640×640 street view image containing multi-scale and densely occluded pedestrians is input into the SVDO-YOLO model; multi-scale features are extracted layer by layer through convolutional layers and downsampling in the Backbone network; upsampling and feature fusion of different scales are performed in the Neck network; and feature maps of different scales are mapped to the prediction space in the Detect network.
[0206] This embodiment illustrates the effects achieved by the present invention in combination with the accompanying drawings and data. The detection effect of SVDO-YOLO is tested through a comparative experiment between SVDO-YOLO and a benchmark model, and the experimental results are shown in Table 1. The model proposed by the present invention is superior to the benchmark model in terms of precision, recall, and mean average precision, and the number of parameters and computational complexity are basically the same as those of the benchmark model.
[0207] Table 1 Experimental comparison between YOLOv8 and SVDO-YOLO
[0208]
[0209] To demonstrate the detection results of the present invention, in combination with the attached Figures 9 - 10 description, SVDO-YOLO and YOLOv8 are compared. The left side is the detection image of YOLOv8, and the right side is the detection image of SVDO-YOLO. As shown in the attached Figure 9 figure, in the case of a smaller pedestrian scale, SVDO-YOLO can detect more small-scale pedestrians than YOLOv8; as shown in the attached Figure 10 figure, in the case of densely occluded pedestrians, SVDO-YOLO can detect more occluded pedestrians than YOLOv8.
[0210] Embodiment 2
[0211] The purpose of this embodiment is to provide a pedestrian target detection system based on the SVDO-YOLO network, including:
[0212] A data acquisition module, configured to acquire a pedestrian target image;
[0213] A data preprocessing module, configured to preprocess the pedestrian target image to obtain a preprocessed pedestrian target image;
[0214] The model construction and training module is used to construct a pedestrian target detection model, input the preprocessed pedestrian target images into the pedestrian target detection model for training, and obtain the trained pedestrian target detection model;
[0215] Among them, when inputting the preprocessed pedestrian target images into the pedestrian target detection model for training, the pedestrian target detection model is based on the YOLOv8 network architecture, constructs a new bidirectional multi-layer fusion network based on the bidirectional feature pyramid network, constructs a dynamic dense feature extraction module to replace the second C2f module in the backbone network and the neck network C2f module, and constructs a dense feature extraction module to replace the C2f module in the head network. The specific process is as follows:
[0216] Input the target images into the improved backbone network for feature extraction and enhancement to obtain multi-scale key feature maps;
[0217] Input the multi-scale key features into the neck network for feature fusion and enhancement to obtain the final output feature maps;
[0218] Input the final output feature maps into the head network for detection to obtain the pedestrian target detection results;
[0219] The model detection module is used to detect the pedestrian targets to be detected by using the trained pedestrian target detection model.
[0220] The pedestrian target detection system based on the SVDO-YOLO network implements the method steps in Embodiment 1.
[0221] In the devices of the above embodiments, the steps involved correspond to those in Method Embodiment 1. For the specific implementation manners, reference may be made to the relevant description parts of Embodiment 1. The term "computer-readable storage medium" should be understood to include a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.
[0222] Those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computer device. Optionally, they can be implemented by program codes executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.
[0223] Although the specific implementation manners of the present invention have been described above in conjunction with the accompanying drawings, it is not a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications or deformations that can be made without creative efforts on the basis of the technical solution of the present invention are still within the protection scope of the present invention.
Claims
1. A pedestrian target detection method based on the SVDO-YOLO network, characterized in that, Including: Obtain a pedestrian target image; Preprocess the pedestrian target image to obtain a preprocessed pedestrian target image; Construct a pedestrian target detection model, and input the preprocessed pedestrian target image into the pedestrian target detection model for training to obtain a trained pedestrian target detection model; Use the trained pedestrian target detection model to detect the pedestrian target to be detected; Among them, input the preprocessed pedestrian target image into the pedestrian target detection model for training. Among them, the pedestrian target detection model is based on the YOLOv8 network architecture, and a new bidirectional multi-layer fusion network is constructed based on the bidirectional feature pyramid network. A dynamic dense feature extraction module is constructed to replace the second C2f module in the backbone network and the neck network C2f module, and a dense feature extraction module is constructed to replace the C2f module in the head network. The specific process is as follows: Input the target image into the improved backbone network for feature extraction and enhancement to obtain a multi-scale key feature map; Input the multi-scale key features into the neck network for feature fusion and enhancement to obtain a final output feature map; Input the final output feature map into the head network for detection to obtain the pedestrian target detection result.
2. The pedestrian target detection method based on the SVDO-YOLO network according to claim 1, wherein, Input the target image into the improved backbone network for feature extraction and enhancement to obtain a multi-scale key feature map. The specific process is as follows: In the backbone network, input the picture into two convolutional layers in sequence for downsampling to obtain a low-level feature map; Input the low-level features into the C2f module for feature aggregation and compression to obtain aggregated features; Input the aggregated features into a convolutional layer for further downsampling and dimensionality reduction to obtain a deep feature representation; Input the deep features into the first dynamic dense feature extraction module for dynamic feature extraction to obtain refined features; Input the refined features into 9 MobileNetV3 modules in sequence for feature enhancement to obtain multi-scale key features.
3. The pedestrian target detection method based on the SVDO-YOLO network according to claim 2, It is characterized in that Input the deep features into the first dynamic dense feature extraction module for dynamic feature extraction to obtain refined features. Each dynamic dense feature extraction module includes two convolutional layers, a segmentation module, and multiple dynamic-bottleneck modules. The specific process is as follows: Input the deep features into a convolutional layer for channel expansion to obtain expanded features; Input the expanded features into the segmentation module to obtain two feature groups; Input the first feature group into multiple dynamic-bottleneck modules in sequence to obtain deeper features respectively; Concatenate the deeper features with the second feature group to obtain fused features; Input the fused features into a convolutional layer to obtain refined features.
4. The pedestrian target detection method based on the SVDO-YOLO network according to claim 2, wherein, Input the refined features into 9 MobileNetV3 modules in sequence. The MobileNetV3 module includes three convolutional layers, a compression and excitation module; In the first MobileNetV3 module, input the refined features into two convolutional layers in sequence to obtain dimension-increased lightweight features; Input the dimension-increased lightweight features into the compression and excitation module for channel weighting to obtain weighted activation features; Input the weighted activation features into the third convolutional layer to obtain multi-scale key features.
5. The pedestrian target detection method based on the SVDO-YOLO network according to claim 1, characterized in that, Input the multi-scale key features into the neck network for feature fusion and enhancement to obtain the final output feature map. The specific process is as follows: In the neck network, input the aggregated features output by the C2f module, the refined features output by the first dynamic dense feature extraction module, and the multi-scale key features output by the fourth MobileNetV3 module into the first network to obtain enhanced fusion features; According to the enhanced fusion features, obtain cross-layer fusion features through the processing of the second network; According to the cross-layer fusion features, obtain the final input features through the processing of the third network.
6. The pedestrian target detection method based on the SVDO-YOLO network according to claim 5, wherein, Input the aggregated features output by the C2f module, the refined features output by the first dynamic dense feature extraction module, and the multi-scale key features output by the fourth MobileNetV3 module into the first network. The specific process is as follows: Input the multi-scale key features obtained by the fourth MobileNetV3 module into the first SPPF module at the same time to obtain the first pooling feature; Upsample the first pooling feature and concatenate it with the refined features output by the first dynamic dense feature extraction module to obtain the first fusion feature; Input the first fusion feature into the second dynamic dense feature extraction module for deep feature extraction to obtain prominent features; Upsample the prominent features and concatenate them with the aggregated features output by the C2f module to obtain the second fusion feature; Input the second fusion feature into the third dynamic dense feature extraction module for deep feature extraction to obtain enhanced fusion features.
7. The pedestrian target detection method based on the SVDO-YOLO network according to claim 5, characterized in that, According to the enhanced fusion features, obtain cross-layer fusion features through the processing of the second network. The specific process is as follows: Input the multi-scale key features output by the ninth MobileNetV3 module into the second SPPF module to obtain the second pooling feature; Upsample the second pooling feature and concatenate it with the key features output by the fourth MobileNetV3 module to obtain the third fusion feature; Input the third fusion feature into the fourth dynamic dense feature extraction module for feature enhancement to obtain the first enhanced feature; Upsample the first enhanced feature and concatenate it with the refined features of the first dynamic dense feature extraction module and the refined features output by the second dynamic dense feature extraction module to obtain the fourth fusion feature; Input the fourth fusion feature into the fifth dynamic dense feature extraction module for feature extraction to obtain integrated features; Upsample the integrated features and concatenate them with the aggregated features output by the C2f module and the refined features output by the third dynamic dense feature extraction module to obtain the fifth fusion feature; Input the fifth fusion feature into the dense feature extraction module to obtain cross-layer fusion features.
8. The pedestrian target detection method based on the SVDO-YOLO network according to claim 5, characterized in that, According to the cross-layer fusion features, obtain the final input features through the processing of the third network. The specific process is as follows: Input the cross-layer fusion features output by the dense feature extraction module into the convolutional layer for downsampling to obtain deep features; Concatenate the deep features with the refined features output by the second dynamic dense feature extraction module and the refined features output by the fourth dynamic dense feature extraction module to obtain the sixth fusion feature; Input the sixth fusion feature into the sixth dynamic dense feature extraction module for feature extraction to obtain the second enhanced feature; Input the second enhanced feature into the DynamicConv module for dynamic downsampling to obtain a dynamically fused feature; Input the dynamically fused feature into the seventh dynamic dense feature extraction module for dynamic feature extraction to obtain the final output feature.
9. The pedestrian target detection method based on the SVDO-YOLO network according to claim 8, wherein, Input the second enhanced feature into the DynamicConv module for dynamic downsampling to obtain a dynamically fused feature. The specific process is as follows: Input the second enhanced feature into the attention module to obtain an attention-weighted feature; Input the attention-weighted feature into multiple convolutional layers to obtain branch features respectively; Concatenate the branch features to obtain a comprehensive feature; Input the obtained comprehensive feature and the refined feature output by the sixth dynamic dense feature extraction module into a convolutional layer and perform normalization processing to obtain a dynamically fused feature.
10. A pedestrian target detection system based on the SVDO - YOLO network, characterized in that, Including: A data acquisition module for acquiring pedestrian target images; A data preprocessing module for preprocessing the pedestrian target image to obtain a preprocessed pedestrian target image; A model construction and training module for constructing a pedestrian target detection model and inputting the preprocessed pedestrian target image into the pedestrian target detection model for training to obtain a trained pedestrian target detection model; Among them, input the preprocessed pedestrian target image into the pedestrian target detection model for training. Among them, the pedestrian target detection model is based on the YOLOv8 network architecture, constructs a new bidirectional multi-layer fusion network based on the bidirectional feature pyramid network, constructs a dynamic dense feature extraction module to replace the second C2f module in the backbone network and the neck network C2f module, and constructs a dense feature extraction module to replace the C2f module in the head network. The specific process is as follows: Input the target image into the improved backbone network for feature extraction and enhancement to obtain a multi-scale key feature map; Input the multi-scale key feature into the neck network for feature fusion and enhancement to obtain the final output feature map; Input the final output feature map into the head network for detection to obtain the pedestrian target detection result; A model detection module for detecting the pedestrian target to be detected using the trained pedestrian target detection model.