Vehicle target detection method and device based on multi-scale feature extraction and fusion

Abstract

The invention relates to a vehicle target detection method based on multi-scale feature extraction and fusion. The method comprises the following steps: acquiring an original image data set; the yolov8 model is improved, and an improved yolov8 model, namely a vehicle target detection model, is obtained; performing iterative training on the vehicle target detection model; and inputting a to-be-detected image into the trained vehicle target detection model to obtain a vehicle target detection result. According to the invention, the MS-FFCN module effectively captures diversified image features from local details to a macrostructure by introducing a multi-scale convolutional layer, can extract features from detail and global information at the same time, enhances the processing capability of various targets, and improves the detection precision; according to the method, the feature extraction and representation capability is improved, the identification capability of the network to details is enhanced, the discriminability of the features is enhanced, the processing speed is remarkably improved, and the overall performance of the network in target detection is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle target detection, and in particular to a vehicle target detection method and device based on multi-scale feature extraction and fusion. Background Art

[0002] With the rapid development of intelligent transportation, autonomous driving, and intelligent monitoring, the importance of vehicle detection algorithms in practical applications has become increasingly prominent. By efficiently and accurately identifying the location and type of vehicles in images or videos, a variety of intelligent applications such as traffic flow control, road anomaly monitoring, and parking management can be realized. Current vehicle detection algorithms need to maintain high performance in a variety of complex scenarios, such as vehicle occlusion, lighting changes, and dense traffic environments, which places higher requirements on the robustness and accuracy of the algorithm.

[0003] With the continuous advancement of deep learning and computer vision technology, many application scenarios have put forward stricter standards for the accuracy and missed detection rate of vehicle detection, making urban vehicle target detection a hot area of ​​academic research. At present, target detection algorithms based on deep learning are mainly divided into two categories: two-stage detection methods (such as RCNN, etc.) and one-stage detection methods (such as SSD and YOLO, etc.). Although the two-stage detection method has outstanding performance in accuracy, it has a slow processing speed and is not suitable for urban vehicle target detection tasks. The one-stage detection method can complete the detection through a single feature extraction, and the processing speed is faster, but it may be accompanied by a certain loss of accuracy.

[0004] Although existing methods have improved the performance of vehicle detection models to a certain extent, due to the complexity and diversity of vehicles in urban environments, the detection accuracy is often low and the missed detection rate is high. Therefore, it is urgent to research and develop optimized detection algorithms to improve the model while ensuring accuracy to meet the needs of practical applications. Summary of the invention

[0005] In order to solve the defects of low vehicle detection accuracy and high missed detection rate, the primary purpose of the present invention is to provide a vehicle target detection method based on multi-scale feature extraction and fusion, which can extract features from details and global information at the same time, enhance the processing ability of various targets, improve the detection accuracy, and improve the overall performance of the network in target detection.

[0006] To achieve the above object, the present invention adopts the following technical solution: a vehicle target detection method based on multi-scale feature extraction and fusion, the method comprising the following steps in order:

[0007] (1) Obtain the original image dataset and divide it into training set, validation set and test set in proportion;

[0008] (2) Improve the yolov8 model to obtain the improved yolov8 model, i.e., the vehicle target detection model;

[0009] (3) Set network training hyperparameters;

[0010] (4) Use the training set to iteratively train the vehicle target detection model;

[0011] (5) Input the image to be detected into the trained vehicle target detection model to obtain the vehicle target detection result.

[0012] In step (1), the original image dataset adopts the Venom dataset obtained from the roboflow platform; the original image dataset is divided into a training set, a validation set, and a test set in a ratio of 7:2:1.

[0013] Step (2) specifically refers to: replacing the SPPF module in the yolov8 model with the MS-FFCN module, and adding the OKSF module and the spatial depth conversion convolution module, namely the SPDConv module, to the detection head of the yolov8 model.

[0014] Step (3) specifically refers to: using the SGD optimizer to optimize the vehicle target detection model, with an initial learning rate of 0.01, a final learning rate of 0.0001, a momentum of 0.937, a batch normalization size of 16, and 150 iterations.

[0015] The MS-FFCN module includes two 1×1 convolutions, one 3×3 convolution with a step size of 1, one 3×3 convolution with a step size of 3, one 3×3 convolution with a step size of 5, and a concatenation layer; the two 1×1 convolutions include a first 1×1 convolution and a second 1×1 convolution; first, the number of channels of the input feature map is adjusted by the first 1×1 convolution to obtain the first feature map; the 3×3 convolution with a step size of 1 is used to obtain the first feature map. The convolution performs a convolution operation on the first feature map to extract spatial features and obtain a second feature map; the input feature map and the second feature map are added element by element to obtain a third feature map; the 3×3 convolution with a step size of 3 extracts local features while reducing the spatial size of the feature map output by the 3×3 convolution with a step size of 3 by sliding the convolution kernel on the third feature map to obtain the fourth feature map; the 3×3 convolution with a step size of 5 extracts local features and greatly reduces the spatial size of the feature map output by the 3×3 convolution with a step size of 5 by sliding the convolution kernel on the fourth feature map with a step size of 5 pixels to obtain the fifth feature map; the first, second, fourth, and fifth feature maps are spliced ​​through a splicing layer to obtain the sixth feature map, and the second 1×1 convolution reduces the dimension of the sixth feature map to obtain the output feature map.

[0016] The OKSF module includes two 1×1 convolutions, an OmniKernel module and a splicing module, the two 1×1 convolutions include a third 1×1 convolution and a fourth 1×1 convolution, the feature map input to the OKSF module is first subjected to the third 1×1 convolution to adjust the number of channels to obtain the seventh feature map, and then the seventh feature map is divided into two parts, one part of which is input to the OmniKernel module for multi-scale feature extraction to obtain the eighth feature map, and the other part retains the original features to obtain the ninth feature map; the eighth feature map and the ninth feature map are combined by splicing operation through the splicing module, and then refined by the fourth 1×1 convolution to finally generate an optimized feature map.

[0017] Another object of the present invention is to provide an electronic device, comprising:

[0018] Processor; and

[0019] A memory, in which computer program instructions are stored, and when the computer program instructions are executed by the processor, the processor executes the vehicle target detection method based on multi-scale feature extraction and fusion as described above.

[0020] The present invention also provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the vehicle target detection method based on multi-scale feature extraction and fusion as described above.

[0021] It can be seen from the above technical scheme that the beneficial effects of the present invention are: First, the MS-FFCN module in the present invention effectively captures diversified image features from local details to macro structures by introducing a multi-scale convolution layer. Compared with the traditional single-scale convolution, multi-scale feature extraction can make the network more accurate and comprehensive when processing targets of different sizes and types, especially in target detection tasks. It can extract features from details and global information at the same time, enhance the processing ability of various targets, and improve detection accuracy; Second, the OKSF module and SPDConv module in the present invention improve the feature extraction and representation capabilities while reducing the amount of calculation, enhance the network's recognition of details, and make the features not only more discriminative, but also significantly improve the processing speed, thereby greatly improving the overall performance of the network in target detection; Third, the present invention greatly improves the detection accuracy, processing efficiency and adaptability of the network. Whether in target detection, image classification, or other computer vision tasks, it can provide higher accuracy and stronger robustness, and significantly optimize the overall performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of the method of the present invention;

[0023] Figure 2 It is a schematic diagram of the structure of the MS-FFCN module in the present invention;

[0024] Figure 3 This is a schematic diagram of the structure of the detection head of the yolov8 model after adding the OKSF module and the SPDConv module;

[0025] Figure 4 They are respectively schematic diagrams of the structure of the OKSF module in the present invention. DETAILED DESCRIPTION

[0026] like Figure 1 As shown, a vehicle target detection method based on multi-scale feature extraction and fusion includes the following steps in sequence:

[0027] (1) Obtain the original image dataset and divide it into training set, validation set and test set in proportion;

[0028] (2) Improve the yolov8 model to obtain the improved yolov8 model, i.e., the vehicle target detection model;

[0029] (3) Set network training hyperparameters;

[0030] (4) Use the training set to iteratively train the vehicle target detection model;

[0031] (5) Input the image to be detected into the trained vehicle target detection model to obtain the vehicle target detection result.

[0032] In step (1), the original image dataset uses the Venom dataset obtained from the roboflow platform; the original image dataset is divided into a training set, a validation set, and a test set in a ratio of 7:2:1. The main purpose of the training set is to allow the model to grasp the core features and patterns of the data; the validation set is used to fine-tune the model parameters and select the optimal model architecture during the training process, while monitoring whether the model is overfitting or underfitting; the role of the test set is to ultimately evaluate the generalization ability of the model and measure its performance on unseen data, which provides a fair benchmark for evaluating the effectiveness of the model in practical applications.

[0033] Step (2) specifically refers to: replacing the SPPF module in the yolov8 model with the MS-FFCN module, adding the OKSF module and the spatial depth conversion convolution module, namely the SPDConv module, to the detection head of the yolov8 model. The improved detection head shows higher accuracy and efficiency when processing target detection tasks of different scales, and significantly enhances the feature representation capability and adaptability to various scenarios, such as Figure 3 shown.

[0034] Step (3) specifically refers to: using the SGD optimizer to optimize the vehicle target detection model, with an initial learning rate of 0.01, a final learning rate of 0.0001, a momentum of 0.937, a batch normalization size of 16, and 150 iterations.

[0035] like Figure 2 As shown in the figure, the MS-FFCN module includes two 1×1 convolutions, a 3×3 convolution with a step size of 1, a 3×3 convolution with a step size of 3, a 3×3 convolution with a step size of 5, and a concatenation layer; the two 1×1 convolutions include a first 1×1 convolution and a second 1×1 convolution; first, the number of channels of the input feature map is adjusted by the first 1×1 convolution to obtain the first feature map; the 3×3 convolution with a step size of 1 is used to obtain the first feature map. The convolution performs a convolution operation on the first feature map to extract spatial features and obtain the second feature map; the input feature map and the second feature map are added element by element to obtain the third feature map; the 3×3 convolution with a step size of 3 extracts local features by sliding the convolution kernel on the third feature map while reducing the spatial size of the feature map output by the 3×3 convolution with a step size of 3 to obtain the fourth feature map; the 3×3 convolution with a step size of 5 extracts local features and greatly reduces the spatial size of the feature map output by the 3×3 convolution with a step size of 5 on the fourth feature map by sliding the convolution kernel with a step size of 5 pixels to obtain the fifth feature map; the first, second, fourth, and fifth feature maps are spliced ​​through the splicing layer to obtain the sixth feature map, and the second 1×1 convolution reduces the dimension of the sixth feature map to obtain the output feature map. The MS-FFCN module effectively improves the efficiency of feature extraction, significantly reduces the consumption of computing resources, and greatly enhances the performance of the model in feature expression and capture.

[0036] like Figure 2 As shown in the figure, the MS-FFCN module can effectively process image data and extract meaningful features through multi-scale feature extraction and fusion, thereby solving various computer vision tasks.

[0037] The first 1×1 convolution is mainly used to change the number of channels of the feature map. It generates new features by linearly combining each pixel of the input. This is very effective in adjusting the complexity of the network and fusing the features of different channels. It also has a relatively small amount of calculation and can play a role in optimization and adjustment in the network structure, preparing for subsequent more complex convolution operations.

[0038] Residual connection Add can effectively alleviate the problem of gradient vanishing or gradient exploding in deep neural networks. At the same time, residual connection is conducive to the network learning identity mapping, allowing simple features to be directly transmitted. As training progresses, the newly learned complex features can be combined with the original features. From the actual effect, it can significantly improve the performance of the network in visual tasks such as target detection, enabling the network to better utilize input information, extract more valuable features, and enhance the processing capabilities of features such as texture and shape in the image, thereby improving the accuracy of classification or detection.

[0039] The 3×3 convolution with a stride of 1 is mainly used to extract local features of the image. It slides on the input feature map with a small 3×3 window, sliding one pixel at a time. This process is similar to using a small "detector" to scan the image block by block. Because the convolution kernel size is small, it can well capture the relationship between adjacent pixels in the image, such as edges, textures and other detailed features. Through a large number of such convolution operations, the network can learn rich local feature patterns. The extraction of such local features is a very critical step in computer vision tasks, because much visual information is reflected in the local structure of the image.

[0040] The 3×3 convolution with a step size of 3 can effectively reduce the size of the feature map while extracting features. A larger step size allows the convolution kernel to skip more pixels when sliding, thereby reducing the resolution of the feature map. On the one hand, it reduces the complexity of subsequent calculations because the feature map becomes smaller and fewer pixels are involved in the calculation. On the other hand, it can extract more global features.

[0041] The 3×3 convolution with a stride of 5 further reduces the size of the feature map, can capture image features from a more macroscopic perspective, can compress image information to a lower dimension more quickly, and also helps to highlight the main objects or large structural patterns in the image. The 3×3 convolution with a stride of 5 combined with the previous convolutional layers with strides of 1 and 3 can form a multi-scale feature extraction system from fine-grained local features to coarse-grained global features, allowing the network to focus on both the details and the overall structure of the image.

[0042] The second 1×1 convolution is mainly used to further adjust the concatenated feature map. It can change the number of channels of the feature map and optimize the feature representation according to the needs of the specific task, so that the output features are more in line with the task requirements and provide a more accurate feature basis for the final output results.

[0043] The MS-FFCN module can effectively process image data and extract meaningful features through multi-scale feature extraction and fusion, thereby solving various computer vision tasks.

[0044] like Figure 3As shown in Figure 1, p2 is the bottom-level input feature map, which first passes through the SPDConv module. The SPDConv module consists of a space-to-depth SPD layer and a convolution-free stride Conv layer, which can be applied to most CNN architectures.

[0045] p3 is another input feature map, which is slightly higher in level than p2.

[0046] p4 is a higher-level input feature map.

[0047] p5 is the highest level input feature map, located at the top layer of the entire detection head.

[0048] like Figure 4 As shown, the OKSF module includes two 1×1 convolutions, an OmniKernel module and a splicing module, the two 1×1 convolutions include a third 1×1 convolution and a fourth 1×1 convolution, and the feature map input to the OKSF module is first subjected to the third 1×1 convolution to adjust the number of channels to obtain the seventh feature map, and then the seventh feature map is divided into two parts, one part of which is input into the OmniKernel module for multi-scale feature extraction to obtain the eighth feature map, and the other part retains the original features to obtain the ninth feature map; the eighth feature map and the ninth feature map are combined by splicing operation through the splicing module, and then refined by the fourth 1×1 convolution to finally generate the optimized feature map.

[0049] In summary, the MS-FFCN module in the present invention effectively captures diversified image features from local details to macro structures by introducing a multi-scale convolution layer. Compared with the traditional single-scale convolution, multi-scale feature extraction can make the network more accurate and comprehensive when processing targets of different sizes and types, especially in target detection tasks. It can extract features from details and global information at the same time, enhance the processing capabilities of various targets, and improve detection accuracy; the OKSF module and SPDConv module in the present invention improve the feature extraction and representation capabilities while reducing the amount of calculation, enhance the network's recognition of details, and make the features not only more discriminative, but also significantly improve the processing speed, thereby greatly improving the overall performance of the network in target detection; the present invention greatly improves the detection accuracy, processing efficiency and adaptability of the network, whether in target detection, image classification, or other computer vision tasks, it can provide higher accuracy and stronger robustness, and significantly optimize the overall performance of the model.

[0050] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A vehicle target detection method based on multi-scale feature extraction and fusion, characterized in that: The method comprises the following steps in order: (1) Obtain the original image dataset and divide it into training set, validation set and test set in proportion; (2) Improve the yolov8 model to obtain the improved yolov8 model, i.e., the vehicle target detection model; (3) Set network training hyperparameters; (4) Use the training set to iteratively train the vehicle target detection model; (5) Input the image to be detected into the trained vehicle target detection model to obtain the vehicle target detection result; Step (2) specifically refers to: replacing the SPPF module in the yolov8 model with the MS-FFCN module, and adding the OKSF module and the spatial depth conversion convolution module, namely the SPDConv module, to the detection head of the yolov8 model.

2. The vehicle target detection method based on multi-scale feature extraction and fusion according to claim 1, characterized in that: In step (1), the original image dataset adopts the Venom dataset obtained from the roboflow platform; the original image dataset is divided into a training set, a validation set, and a test set in a ratio of 7:2:

1.

3. The vehicle target detection method based on multi-scale feature extraction and fusion according to claim 1, characterized in that: Step (3) specifically refers to: using the SGD optimizer to optimize the vehicle target detection model, with an initial learning rate of 0.01, a final learning rate of 0.0001, a momentum of 0.937, a batch normalization size of 16, and 150 iterations.

4. The vehicle target detection method based on multi-scale feature extraction and fusion according to claim 1, characterized in that: The MS-FFCN module includes two 1×1 convolutions, a 3×3 convolution with a step size of 1, a 3×3 convolution with a step size of 3, a 3×3 convolution with a step size of 5, and a concatenation layer; the two 1×1 convolutions include a first 1×1 convolution and a second 1×1 convolution; first, the number of channels of the input feature map is adjusted by the first 1×1 convolution to obtain the first feature map; the 3×3 convolution with a step size of 1 is used to obtain the first feature map. The convolution performs a convolution operation on the first feature map to extract spatial features and obtain a second feature map; the input feature map and the second feature map are added element by element to obtain a third feature map; the 3×3 convolution with a step size of 3 extracts local features while reducing the spatial size of the feature map output by the 3×3 convolution with a step size of 3 by sliding the convolution kernel on the third feature map to obtain the fourth feature map; the 3×3 convolution with a step size of 5 extracts local features and greatly reduces the spatial size of the feature map output by the 3×3 convolution with a step size of 5 by sliding the convolution kernel on the fourth feature map with a step size of 5 pixels to obtain the fifth feature map; the first, second, fourth, and fifth feature maps are spliced ​​through a splicing layer to obtain the sixth feature map, and the second 1×1 convolution reduces the dimension of the sixth feature map to obtain the output feature map.

5. The vehicle target detection method based on multi-scale feature extraction and fusion according to claim 1, characterized in that: The OKSF module includes two 1×1 convolutions, an OmniKernel module and a splicing module, the two 1×1 convolutions include a third 1×1 convolution and a fourth 1×1 convolution, the feature map input to the OKSF module is first subjected to the third 1×1 convolution to adjust the number of channels to obtain the seventh feature map, and then the seventh feature map is divided into two parts, one part of which is input to the OmniKernel module for multi-scale feature extraction to obtain the eighth feature map, and the other part retains the original features to obtain the ninth feature map; the eighth feature map and the ninth feature map are combined by splicing operation through the splicing module, and then refined by the fourth 1×1 convolution to finally generate an optimized feature map.

6. An electronic device comprising: processor; and a memory, in which a computer program instruction is stored, and when the computer program instruction is executed by the processor, the processor executes the vehicle target detection method based on multi-scale feature extraction and fusion as claimed in any one of claims 1 to 5. A memory, in which a computer program instruction is stored, and when the computer program instruction is executed by the processor, the processor executes the vehicle target detection method based on multi-scale feature extraction and fusion as claimed in any one of claims 1 to 5.

7. A computer-readable storage medium having computer program instructions stored thereon, wherein when the computer program instructions are executed by a processor, the processor executes the vehicle target detection method based on multi-scale feature extraction and fusion as described in any one of claims 1 to 5.

Images