A vehicle detection and tracking method, apparatus, and computer-readable storage medium

Abstract

This invention belongs to the field of vehicle trajectory recognition technology, and relates to a vehicle detection and tracking method, device, and computer-readable storage medium. It involves acquiring a trained target detection model, which includes a feature extraction network, a multi-scale feature fusion network, and a target detection network. The feature extraction network includes a CBS module, four convolutional attention modules, and a spatial pyramid pooling module. The convolutional attention modules include a first phantom convolution sub-module and a C3GhostV2 sub-module. Each frame of the video to be detected is input into the feature extraction network to obtain a first feature map, a second feature map, and a third feature map, which are then input into the multi-scale feature fusion network to output a target feature map. The target feature map is input into the target detection network to output the vehicle bounding box and position information. The vehicle bounding boxes and position information from all frames of the video to be detected are input into the DeepSort multi-target tracking model to output the vehicle's trajectory.

Description

Technical Field

[0001] This invention relates to the field of vehicle trajectory recognition technology, and in particular to a vehicle detection and tracking method, device, and computer-readable storage medium. Background Technology

[0002] With the rapid development of industrial productivity and the continuous improvement of transportation infrastructure, people's demand for motor vehicle travel is also constantly increasing. However, the increase in road traffic vehicles has brought significant challenges to traffic safety, and the persistently high traffic accident rate has become a focus in the field of safe driving. Real-time and accurate vehicle identification in road traffic scenarios can help vehicles on the road avoid safety hazards caused by abnormalities of vehicles ahead, thereby effectively improving the safety of driving systems. In recent years, with the rapid development of artificial intelligence, computer vision technology can be used to analyze and identify vehicle images or videos, thereby achieving automatic vehicle detection, identification, and tracking.

[0003] Existing technologies combine real-time object detection algorithms (You Only Look Once, YOLO) with multi-object tracking algorithms (DeepSort) for vehicle detection and tracking. YOLO is used to detect multiple vehicle bounding boxes and their position information in the current frame of the video. DeepSort receives these bounding boxes and uses a deep learning model to extract features from each vehicle, generating unique feature vectors as vehicle identifiers. Then, a Kalman filter is used to predict the vehicle's position in the next frame based on its position information in the current frame. Finally, a Hungarian algorithm is used to perform concatenated matching and IoU matching between the predicted position and the bounding boxes and position information of multiple vehicles in the next frame, as output by YOLO in real time. If a matching bounding box is detected in the next frame, the vehicle's position information is updated, and its feature vector is re-extracted, thus forming the vehicle's continuous trajectory. This method can detect and track vehicles on the road in real time. However, the accuracy of detection and tracking often depends on the accuracy of target detection. With the continuous evolution of the YOLO series of algorithms, larger and more complex target detection models have been proposed. As the model size increases, the network structure becomes more and more complex, and the extraction and recognition results of vehicles in the image become more and more accurate. The accuracy of the output vehicle bounding boxes also becomes higher and higher. However, the increase in model size and structural complexity also leads to a larger number of model parameters and a larger model size. This not only requires more computing resources and memory, but also increases computational complexity, affecting the real-time performance of vehicle detection and tracking. As a result, it is impossible to adjust traffic flow density and provide early warning of traffic problems in a timely manner based on vehicle detection and tracking data, thus affecting road traffic safety.

[0004] In summary, while existing vehicle detection and tracking methods improve detection and tracking accuracy, they reduce the real-time performance of detection and tracking. They are unable to adjust traffic density and provide early warnings of traffic problems based on vehicle detection and tracking data in a timely manner, thus affecting road traffic safety. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problem that the existing vehicle detection and tracking methods, while improving the accuracy of detection and tracking, reduce the real-time performance of detection and tracking, and cannot adjust traffic flow density and provide early warning of traffic problems in a timely manner based on vehicle detection and tracking data, thereby affecting road traffic safety.

[0006] To solve the above-mentioned technical problems, the present invention provides a vehicle detection and tracking method, comprising:

[0007] Obtain a trained target detection model; the target detection model includes a feature extraction network, a multi-scale feature fusion network and a target detection network. The feature extraction network includes a CBS module, four convolutional attention modules and a spatial pyramid pooling module connected in series along the forward propagation direction. Each of the convolutional attention modules includes a first phantom convolutional submodule and a C3GhostV2 submodule connected in series.

[0008] Each frame of the video to be detected is input into the feature extraction network. Based on the outputs of the second convolutional attention module, the third convolutional attention module, and the spatial pyramid pooling module, a first feature map, a second feature map, and a third feature map are obtained. The C3GhostV2 submodule performs feature extraction on the feature maps, including the following steps:

[0009] The feature maps are input into the first CBS unit and the second CBS unit respectively, and the first intermediate feature and the second intermediate feature are output.

[0010] The first intermediate feature is input into the long-distance attention mechanism enhancement unit, and the third intermediate feature is output.

[0011] The second intermediate feature and the third intermediate feature are input into the feature fusion unit, and the fourth intermediate feature is output.

[0012] The fourth intermediate feature is input into the third CBS unit, and the output of the third CBS unit is used as the output of the C3GhostV2 submodule.

[0013] The first feature map, the second feature map, and the third feature map are input into the multi-scale feature fusion network to output the target feature map;

[0014] The target feature map is input into the target detection network, which outputs the vehicle bounding box and location information.

[0015] The vehicle bounding boxes and position information in all frames of the video to be detected are input into the trained DeepSort multi-object tracking model, which outputs the running trajectory of each vehicle in the video to be detected.

[0016] Preferably, the training process of the target detection model includes:

[0017] Obtain the training set of images to be detected;

[0018] The images in the training set of images to be detected are input into the target detection model, and the predicted vehicle bounding boxes in the images are output.

[0019] Based on the intersection-union ratio of the predicted vehicle bounding box and the real vehicle bounding box, the Euclidean distance between the center point of the predicted vehicle bounding box and the center point of the real vehicle bounding box, and the diagonal length, width and height of the minimum bounding rectangle covering the predicted vehicle bounding box and the real vehicle bounding box, a loss function that minimizes the width and height difference is constructed.

[0020] Based on the loss function that minimizes the width and height difference and the intersection-union ratio of the predicted vehicle bounding box and the real vehicle bounding box, a target detection loss function is constructed;

[0021] The target detection model is trained using images from the training set of images to be detected until the value of the target detection loss function is minimized, thus obtaining a trained target detection model.

[0022] Preferably, the formula for calculating the intersection-union ratio (IUR) of the predicted vehicle bounding box and the actual vehicle bounding box is:

[0023] ,

[0024] in, This represents the intersection-union ratio (IoU) between the predicted vehicle bounding box and the actual vehicle bounding box. This represents the predicted vehicle bounding box; Represents the actual vehicle bounding box;

[0025] The loss function for minimizing the width-to-height difference is expressed as:

[0026] ,

[0027] in, This represents the loss function that minimizes the difference between width and height. express loss; Indicates distance loss; Indicates the loss in side length; This represents the square of the Euclidean distance between the center point of the predicted vehicle bounding box and the center point of the actual vehicle bounding box; This represents the diagonal length of the smallest bounding rectangle that covers both the predicted vehicle bounding box and the actual vehicle bounding box. This represents the square of the difference between the width of the predicted vehicle bounding box and the width of the actual vehicle bounding box. This represents the width of the minimum bounding rectangle that covers both the predicted vehicle bounding box and the actual vehicle bounding box. This represents the square of the difference between the height of the predicted vehicle bounding box and the height of the actual vehicle bounding box. This represents the height of the smallest bounding rectangle that covers both the predicted vehicle bounding box and the actual vehicle bounding box.

[0028] The target detection loss function is expressed as follows:

[0029] ,

[0030] in, This represents the target detection loss function; This represents the hyperparameter used to control the curvature of the curve.

[0031] Preferably, the multi-scale feature fusion network includes a bottom-up feature fusion module, a top-down feature fusion module, and a convolution module;

[0032] The bottom-up feature fusion module includes a second phantom convolution submodule, a first upsampling submodule, a first feature fusion submodule, a first C3 submodule, a third phantom convolution submodule, a second upsampling submodule, and a second feature fusion submodule, which are sequentially connected in series along the forward propagation direction.

[0033] The top-down feature fusion module includes a second C3 submodule, a fourth phantom convolution submodule, a third feature fusion submodule, a third C3 submodule, a fifth phantom convolution submodule, a fourth feature fusion submodule, and a fourth C3 submodule, which are connected in series along the forward propagation direction.

[0034] The convolution module includes a sixth phantom convolution submodule, a seventh phantom convolution submodule, and an eighth phantom convolution submodule.

[0035] Preferably, the first feature map, the second feature map, and the third feature map are input into the multi-scale feature fusion network, and the output target feature map includes:

[0036] The first feature map is input into the second feature fusion submodule of the bottom-up feature fusion module; the second feature map is input into the second feature fusion submodule of the bottom-up feature fusion module; the third feature map is input into the second phantom convolution submodule of the bottom-up feature fusion module;

[0037] The feature map output by the second phantom convolution submodule is input into the fourth feature fusion submodule of the top-down feature fusion module; the feature map output by the third phantom convolution submodule is input into the third feature fusion submodule of the top-down feature fusion module; the feature map output by the second feature fusion submodule is input into the second C3 submodule of the top-down feature fusion module.

[0038] The feature map output by the second C3 submodule is input into the sixth phantom convolution submodule to output the first target feature map; the feature map output by the third C3 submodule is input into the seventh phantom convolution submodule to output the second target feature map; the feature map output by the fourth C3 submodule is input into the eighth phantom convolution submodule to output the third target feature map.

[0039] Preferably, the target detection network includes a first detection head, a second detection head, and a third detection head.

[0040] Preferably, inputting the target feature map into the target detection network and outputting the vehicle bounding box and location information includes:

[0041] The first target feature map is input into the first detection head, and the coordinates of the first predicted bounding box, the confidence that the first predicted bounding box is a vehicle bounding box, and the probability distribution of the first predicted bounding box belonging to each category of bounding box are output.

[0042] The second target feature map is input into the second detection head, and the coordinates of the second predicted bounding box, the confidence that the second predicted bounding box is a vehicle bounding box, and the probability distribution of the second predicted bounding box belonging to each category of bounding box are output.

[0043] The third target feature map is input into the third detection head, and the coordinates of the third predicted bounding box, the confidence that the third predicted bounding box is a vehicle bounding box, and the probability distribution of the third predicted bounding box belonging to each category of bounding box are output.

[0044] Preferably, the feature extraction network in the DeepSort multi-target tracking model is ShuffleNetV2.

[0045] The present invention also provides a vehicle detection and tracking device, comprising:

[0046] The target detection model acquisition module is used to acquire a trained target detection model. The target detection model includes a feature extraction network, a multi-scale feature fusion network, and a target detection network. The feature extraction network includes a CBS module, four convolutional attention modules, and a spatial pyramid pooling module connected in series along the forward propagation direction. Each of the convolutional attention modules includes a first phantom convolutional submodule and a C3GhostV2 submodule connected in series.

[0047] The feature extraction module is used to input each frame of the video to be detected into the feature extraction network, and obtain a first feature map, a second feature map, and a third feature map based on the outputs of the second convolutional attention module, the third convolutional attention module, and the spatial pyramid pooling module; wherein, the C3GhostV2 submodule performs feature extraction on the feature maps in the following steps:

[0048] The feature maps are input into the first CBS unit and the second CBS unit respectively, and the first intermediate feature and the second intermediate feature are output.

[0049] The first intermediate feature is input into the long-distance attention mechanism enhancement unit, and the third intermediate feature is output.

[0050] The second intermediate feature and the third intermediate feature are input into the feature fusion unit, and the fourth intermediate feature is output.

[0051] The fourth intermediate feature is input into the third CBS unit, and the output of the third CBS unit is used as the output of the C3GhostV2 submodule.

[0052] The feature fusion module is used to input the first feature map, the second feature map, and the third feature map into the multi-scale feature fusion network and output the target feature map.

[0053] The target detection module is used to input the target feature map into the target detection network and output the vehicle bounding box and position information;

[0054] The target tracking module is used to input the vehicle bounding boxes and position information in all frames of the video to be detected into the trained DeepSort multi-target tracking model, and output the running trajectory of each vehicle in the video to be detected.

[0055] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the vehicle detection and tracking method described above.

[0056] The vehicle detection and tracking method provided in this application has the following beneficial effects:

[0057] 1. A high-precision and lightweight object detection model is constructed by designing a convolutional attention module in the feature extraction network using phantom convolution and C3GhostV2. Phantom convolution generates feature maps through a small number of non-linear convolution operations, and then a linear operation on these feature maps generates even more. By concatenating all feature maps to obtain all feature information and eliminate redundancy between them, the number of parameters and computational cost of the feature extraction network can be reduced, improving the efficiency of feature extraction. Simultaneously, two parallel CBS units in the C3GhostV2 submodule are used to extract features from the input feature maps at different scales, enhancing the ability to capture local features. The output of one of the CBS units is input to a long-range attention mechanism enhancement unit, effectively fusing global information and compensating for the limitations of pure convolutional structures. By addressing the shortcomings of long-distance feature dependence, a feature fusion unit is used to fuse extracted local and global features, enhancing feature representation capabilities. Finally, a CBS unit is used to optimize the fused features, generating more feature maps without significantly increasing computational load and parameters. This not only improves feature extraction efficiency but also ensures feature extraction quality. Combining low computational complexity with powerful feature representation capabilities, an efficient and high-precision convolutional attention mechanism is formed. Target detection is performed using feature maps extracted by the feature extraction network based on a multi-scale feature fusion network and a detection network. This ensures the detection accuracy of the target detection model while achieving model lightweighting. Consequently, it can be combined with the DeepSort multi-target tracking model for real-time and high-precision detection and tracking of road vehicles, providing a data foundation for road traffic safety.

[0058] 2. By concatenating four convolutional attention modules, feature information is extracted from different scales and angles, enabling the feature extraction network to capture features from fine-grained to coarse-grained, thus improving the performance of the object detection model. Finally, the extracted features are aggregated in the spatial dimension using a spatial pyramid pooling module, compressing the spatial information into a smaller representation and reducing the spatial resolution of the feature map. This reduces the number of parameters while maintaining the effectiveness of feature extraction, further achieving the lightweighting of the object detection model and reducing the redundancy of feature maps generated during object detection, thereby improving the real-time performance of object detection and vehicle tracking. Attached Figure Description

[0059] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0060] Figure 1 The flowchart of the vehicle detection and tracking method provided in this application;

[0061] Figure 2 This is a schematic diagram of the phantom convolutional submodule structure provided in this application;

[0062] Figure 3 This is a schematic diagram of the C3GhostV2 submodule structure provided in this application;

[0063] Figure 4 This is a schematic diagram of the target detection model structure provided in this application;

[0064] Figure 5 A schematic diagram illustrating the intersection of the predicted vehicle bounding box and the actual vehicle bounding box provided in this application;

[0065] Figure 6 This is a schematic diagram of the ShuffleNetV2 architecture provided in this application;

[0066] Figure 7 This is a schematic diagram of the vehicle inspection and tracking process provided in this application;

[0067] Figure 8 A comparative diagram showing the tracking performance of the YOLOv5-Deepsort model and the object detection model + DeepSort multi-object tracking model provided in this application on vehicles on an elevated bridge under cloudy conditions;

[0068] Figure 9 A schematic diagram comparing the tracking performance of the YOLOv5-Deepsort model and the object detection model + DeepSort multi-object tracking model provided in this application under vehicle occlusion conditions;

[0069] Figure 10 The results of YOLOv5-Deepsort model and the target detection model + DeepSort multi-target tracking model provided in this application were tested on highway vehicles under clear skies.

[0070] Figure 11 Tracking results of YOLOv5-Deepsort model and the target detection model + DeepSort multi-target tracking model provided in this application for highway vehicles under clear skies from frame 1626 to frame 1721;

[0071] Figure 12 The results of tracking vehicles traveling in tunnels using the YOLOv5-Deepsort model and the target detection model + DeepSort multi-target tracking model provided in this application are presented.

[0072] Figure 13 Tracking results of a vehicle traveling in a tunnel from frame 1626 to frame 1721 for the YOLOv5-Deepsort model and the target detection model + DeepSort multi-target tracking model provided in this application;

[0073] Figure 14This is a schematic diagram of the vehicle detection and tracking device provided in this application. Detailed Implementation

[0074] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0075] Please see Figure 1 , Figure 1 The diagram shown is a flowchart of the vehicle detection and tracking method provided in this application. The method specifically includes:

[0076] S10: Obtain the trained object detection model; the object detection model includes a feature extraction network, a multi-scale feature fusion network and an object detection network. The feature extraction network includes a CBS module, four convolutional attention modules and a spatial pyramid pooling module connected in series along the forward propagation direction. Each convolutional attention module includes a first phantom convolutional submodule and a C3GhostV2 submodule connected in series.

[0077] S20: Input each frame of the video to be detected into the feature extraction network, and obtain the first feature map, the second feature map, and the third feature map based on the outputs of the second convolutional attention module, the third convolutional attention module, and the spatial pyramid pooling module; wherein, the C3GhostV2 submodule performs feature extraction on the feature maps in the following steps:

[0078] The feature maps are input into the first CBS unit and the second CBS unit respectively, and the first intermediate feature and the second intermediate feature are output.

[0079] The first intermediate feature is input into the long-range attention mechanism enhancement unit, and the third intermediate feature is output.

[0080] The second and third intermediate features are input into the feature fusion unit, and the fourth intermediate feature is output.

[0081] The fourth intermediate feature is input into the third CBS unit, and the output of the third CBS unit is used as the output of the C3GhostV2 submodule.

[0082] S30: Input the first feature map, the second feature map, and the third feature map into the multi-scale feature fusion network and output the target feature map;

[0083] S40: Input the target feature map into the target detection network and output the vehicle bounding box and location information;

[0084] S50: Input the vehicle bounding boxes and position information from all frames of the video to be detected into the trained DeepSort multi-object tracking model, and output the running trajectory of each vehicle in the video to be detected.

[0085] In existing object detection models, feature extraction networks mostly use multiple traditional Conv convolutional blocks to convolve the input image, thereby generating a large number of feature maps. These feature maps often contain a lot of redundant information, resulting in high computational complexity, low detection efficiency, and increased storage requirements for object detection models. Therefore, this application adopts convolution operations based on grouped convolution and channel rearrangement to handle the redundancy between feature maps.

[0086] Specifically, such as Figure 2 The diagram shows the structure of the GhostConv submodule provided in this application. An initial feature map is generated through a small number of non-linear convolution operations, followed by a linear operation to generate more new feature maps. These new feature maps are called the ghosts of the initial feature maps. Finally, the initial feature maps and the new feature maps are concatenated to obtain all the feature information of the image, eliminating redundancy between feature maps. This application uses GhostConv instead of the traditional Conv, which can effectively reduce feature redundancy generated when the object detection model extracts features from the image, saving computational resources, reducing the number of model parameters and computational load, achieving model lightweighting while improving the efficiency of object detection, thereby improving the real-time performance of vehicle detection and tracking.

[0087] Furthermore, in addition to using the phantom convolution submodule, this application also designs a C3GhostV2 submodule. Specifically, GhostNetV2, as a lightweight network module, achieves a good balance in terms of long-range modeling, computational efficiency, global receptive field, feature representation, accuracy, and inference speed. Therefore, this application embeds GhostNetV2 into the C3 module to obtain the C3GhostV2 submodule. The structure of the C3GhostV2 submodule is as follows: Figure 3 As shown.

[0088] The CBS unit consists of convolution, batch normalization, and SiLu activation layers, which are used to enhance the nonlinearity of the model while extracting image features.

[0089] This application concatenates the phantom convolution submodule and the C3GhostV2 submodule into a single convolutional attention module. The phantom convolution submodule enhances the expressive power of the convolutional layer and reduces computation and parameter count by decomposing the original convolutional operation. It transmits information across different convolutional kernels, capturing fine-grained features while improving feature extraction efficiency. Then, the C3GhostV2 submodule generates more feature maps without significantly increasing computation and parameters. This not only improves feature extraction efficiency but also ensures feature extraction quality. By combining low computational complexity with powerful feature representation capabilities, a highly efficient and accurate convolutional attention mechanism is formed. In addition, by concatenating four convolutional attention modules, feature information is extracted from different scales and angles, enabling the feature extraction network to capture features from fine-grained to coarse-grained, thus improving the performance of the object detection model. Finally, the extracted features are aggregated in the spatial dimension using a spatial pyramid pooling module, compressing the spatial information into a smaller representation and reducing the spatial resolution of the feature map. The feature extraction network designed in this application reduces the number of parameters while maintaining the effectiveness of feature extraction, achieving a lightweight object detection model, reducing the redundancy of feature maps generated during object detection, and ensuring object detection accuracy. This improves the timeliness and accuracy of vehicle detection and tracking, enabling real-time and high-precision detection and tracking of road vehicles, providing a data foundation for road traffic safety.

[0090] Specifically, in some embodiments of this application, the multi-scale feature fusion network includes a bottom-up feature fusion module, a top-down feature fusion module, and a convolution module;

[0091] The bottom-up feature fusion module includes a second phantom convolution submodule, a first upsampling submodule, a first feature fusion submodule, a first C3 submodule, a third phantom convolution submodule, a second upsampling submodule, and a second feature fusion submodule, which are sequentially connected in the forward propagation direction.

[0092] The top-down feature fusion module includes the second C3 sub-module, the fourth phantom convolution sub-module, the third feature fusion sub-module, the third C3 sub-module, the fifth phantom convolution sub-module, the fourth feature fusion sub-module, and the fourth C3 sub-module, which are sequentially connected in the forward propagation direction.

[0093] The convolution module includes a sixth phantom convolution submodule, a seventh phantom convolution submodule, and an eighth phantom convolution submodule.

[0094] The multi-scale feature fusion network adopts the PANet structure, which integrates feature maps from different levels in the feature extraction network, improving the detection performance of the target detection model for targets at different scales. This application also uses the phantom convolution submodule to replace the traditional convolutional block in the multi-scale feature fusion network, reducing the number of parameters and computation in the multi-scale feature fusion network, further realizing the lightweighting of the target detection model, and improving the detection efficiency of the target detection model and the real-time performance of vehicle detection and tracking.

[0095] Based on the specific structure of the multi-scale feature fusion network described above, step S30 specifically includes:

[0096] The first feature map is input into the second feature fusion submodule of the bottom-up feature fusion module; the second feature map is input into the second feature fusion submodule of the bottom-up feature fusion module; the third feature map is input into the second phantom convolution submodule of the bottom-up feature fusion module;

[0097] The feature map output by the second phantom convolution submodule is input into the fourth feature fusion submodule in the top-down feature fusion module; the feature map output by the third phantom convolution submodule is input into the third feature fusion submodule in the top-down feature fusion module; the feature map output by the second feature fusion submodule is input into the second C3 submodule in the top-down feature fusion module.

[0098] The feature map output from the second C3 submodule is input into the sixth phantom convolution submodule to output the first target feature map; the feature map output from the third C3 submodule is input into the seventh phantom convolution submodule to output the second target feature map; the feature map output from the fourth C3 submodule is input into the eighth phantom convolution submodule to output the third target feature map.

[0099] Furthermore, the target detection network includes a first detection head, a second detection head, and a third detection head, such as... Figure 4 The diagram shown is a schematic diagram of the target detection model structure provided in this application.

[0100] Step S40 inputs the target feature map into the target detection network and outputs the vehicle bounding box and location information, specifically including:

[0101] The first target feature map is input into the first detection head, and the coordinates of the first predicted bounding box, the confidence that the first predicted bounding box is a vehicle bounding box, and the probability distribution of the first predicted bounding box belonging to each category of bounding box are output.

[0102] The second target feature map is input into the second detection head, and the coordinates of the second predicted bounding box, the confidence that the second predicted bounding box is a vehicle bounding box, and the probability distribution of the second predicted bounding box belonging to each category of bounding box are output.

[0103] The third target feature map is input into the third detection head, and the coordinates of the third predicted bounding box, the confidence that the third predicted bounding box is a vehicle bounding box, and the probability distribution of the third predicted bounding box belonging to each category of bounding box are output.

[0104] During the training of object detection models, the Intersection over Union (IOU) ratio is typically used as a metric to measure the degree of overlap between the predicted and ground truth bounding boxes. Figure 5 As shown, a higher IOU indicates a more accurate detection result from the object detection model. The calculation formula is as follows:

[0105] ,

[0106] in, This represents the intersection-union ratio (IoU) between the predicted vehicle bounding box and the actual vehicle bounding box. This represents the predicted vehicle bounding box; Represents the actual vehicle bounding box;

[0107] The loss function of existing object detection models is based on the intersection-union ratio (IU) of the predicted vehicle bounding box and the ground truth vehicle bounding box, the Euclidean distance between the center points of the predicted and ground truth vehicle bounding boxes, and the diagonal length of the minimum bounding rectangle covering both the predicted and ground truth vehicle bounding boxes.

[0108] ,

[0109] ,

[0110] in, This is the loss function for existing target detection models; Indicates the weighting parameter; Used to measure similarity in aspect ratio; This indicates the width of the predicted vehicle bounding box; This indicates the predicted height of the vehicle's bounding box. This represents the width of the actual vehicle's bounding box; This represents the height of the actual vehicle's bounding box.

[0111] While existing loss functions consider factors such as center point distance and bounding box aspect ratio, they lack adaptability to loss and gradient, and suffer from insufficient accuracy in detecting small objects. Therefore, this application provides a training method for an object detection model. Specifically, the training process of the object detection model includes:

[0112] Obtain the training set of images to be detected;

[0113] The images in the training set of the images to be detected are input into the target detection model, and the predicted vehicle bounding boxes in the images are output.

[0114] Based on the intersection-union ratio of the predicted vehicle bounding box and the real vehicle bounding box, the Euclidean distance between the center point of the predicted vehicle bounding box and the center point of the real vehicle bounding box, and the diagonal length, width and height of the minimum bounding rectangle covering the predicted vehicle bounding box and the real vehicle bounding box, a loss function that minimizes the width and height difference is constructed.

[0115] Specifically, the loss function for minimizing the width-to-height difference is expressed as:

[0116] ,

[0117] in, This represents the loss function that minimizes the difference between width and height. express loss; Indicates distance loss; Indicates the loss in side length; This represents the square of the Euclidean distance between the center point of the predicted vehicle bounding box and the center point of the actual vehicle bounding box; This represents the diagonal length of the smallest bounding rectangle that covers both the predicted vehicle bounding box and the actual vehicle bounding box. This represents the square of the difference between the width of the predicted vehicle bounding box and the width of the actual vehicle bounding box. This represents the width of the minimum bounding rectangle that covers both the predicted vehicle bounding box and the actual vehicle bounding box. This represents the square of the difference between the height of the predicted vehicle bounding box and the height of the actual vehicle bounding box. This represents the height of the smallest bounding rectangle that covers both the predicted vehicle bounding box and the actual vehicle bounding box.

[0118] A target detection loss function is constructed based on the loss function that minimizes the difference between width and height and the intersection-union ratio of the predicted vehicle bounding box and the real vehicle bounding box.

[0119] Specifically, the target detection loss function is expressed as:

[0120] ,

[0121] in, This represents the target detection loss function; This represents the hyperparameter used to control the curvature of the curve.

[0122] The target detection model is trained using images from the training set of images to be detected until the target detection loss function is minimized, thus obtaining a trained target detection model.

[0123] This application utilizes a width-to-height difference minimization loss function to minimize the difference in height and width between the predicted bounding box and the ground truth bounding box, thereby accelerating the convergence speed of the model and obtaining more accurate target detection results. Furthermore, it combines the intersection-union ratio of the predicted bounding box and the ground truth bounding box to obtain the target detection loss function, making the bounding box regression more focused on high-quality bounding boxes, enhancing the model's ability to identify target vehicles that are small in distant imaging, and further improving the recognition accuracy of the target detection model.

[0124] Specifically, after receiving the target detection results, the DeepSort multi-target tracking model first initializes the target trajectory and then predicts the target trajectory using Kalman filtering. The prediction expression is as follows:

[0125] ,

[0126] ,

[0127] according to State estimates and covariance matrix at time t, for prediction The state estimate and covariance matrix at time 1 are used by the Kalman filter to predict the vehicle's state at time 2. The state at any given moment, The state matrix is ​​(covariance). For external control matrix, Here is the state transition matrix. External interference matrix;

[0128] right The confirmed trajectory at time step 1 is concatenated with the current vehicle target bounding box for matching. The matching result is divided into three cases: successful match, trajectory mismatch, and detection mismatch. If the match is successful, the target bounding box predicted by the Kalman filter is updated, and then the target trajectory at the current time step is predicted. The update expression is as follows:

[0129] ,

[0130] ,

[0131] ,

[0132] The prior estimate is corrected and adjusted based on the current observations to obtain the posterior estimate. The observation matrix represents the mapping of the state vector to the observation space; for The optimal estimate matrix of (covariance) at time t. The measurement matrix is ​​(covariance).

[0133] If a match fails, the failed trajectory is matched with the failed detection using IOU. If a match is successful, an update is performed and a new round of tracking is initiated.

[0134] If the target detection fails to match again, it may be due to the appearance of a new target or the target being occluded for a long time. In this case, the system will establish a new trajectory but temporarily mark it as undetermined. After three checks and confirmations, if the target is confirmed to be real, the status will be changed to confirmed and the tracking will be updated.

[0135] If the trajectory fails to match again, it may be because the target was not detected accurately. In this case, the trajectory status is processed. If the status is uncertain, it is deleted directly. If the status is confirmed and has not changed within the set lifespan, it is considered that the target has moved out of the camera and the trajectory is deleted. Otherwise, three checks are performed and the tracking is updated again after confirmation.

[0136]

[0137] ,

[0138] Used to measure the distance between the detected target and the trajectory, where This represents the detected target feature vector. The feature vector representing the trajectory, It is the inverse of the covariance matrix; Used to associate detected targets with trajectories, where Represents the set of trajectory feature vectors. Represents a known trajectory The eigenvector of the first Each component.

[0139] The feature extraction network in the DeepSort multi-object tracking model extracts relevant feature information from the input target image to describe the appearance and structural features of the target, thereby realizing the recognition and tracking of the target. In the existing technology, the feature extraction network in the DeepSort multi-object tracking model mainly consists of two convolutional layers, one max pooling layer, six residual networks and one fully connected layer. Its specific network structure is shown in Table 1.

[0140] Table 1

[0141]

[0142] To further improve the real-time performance of vehicle detection and tracking, this application also lightweights the DeepSort multi-object tracking model. Specifically, in some embodiments of this application, the feature extraction network in the DeepSort multi-object tracking model is ShuffleNetV2.

[0143] like Figure 6 The diagram shows the ShuffleNetV2 architecture provided in this application. ShuffleNetV2 achieves lightweighting through channel shuffle (CS) and depthwise separable convolution operations. Channel shuffle involves grouping the input feature channels and then interleaving and recombinating them after convolution, enabling information exchange and fusion between different channels. Depthwise separable convolution decomposes the convolution operation into depthwise convolution and pointwise convolution, effectively reducing the number of parameters and computational cost of the model, making the network more lightweight while maintaining effective feature extraction capabilities. Therefore, replacing the feature extraction network in the DeepSort multi-object tracking model with ShuffleNetV2 ensures that the model's tracking accuracy remains within an acceptable range while maintaining its expressive power, thus ensuring the model's lightweighting.

[0144] Specifically, in some embodiments, when using the DeepSort multi-object tracking model for vehicle tracking, the size of the rectangular bounding box of the input image needs to be transformed to fit the size of the vehicle image. In a specific example of this application, the size of the rectangular bounding box of the 128×64 image with a height of 128 and a width of 64 is adjusted to 64×128. Table 2 shows the specific parameters of ShuffleNetV2 as the feature extraction network after adjusting the bounding box size.

[0145] Table 2

[0146]

[0147] like Figure 7 The diagram illustrates the vehicle detection and tracking process provided in this application. The video stream is used as input data and preprocessed. A pre-trained target detection model is loaded and initialized. The pre-processed video is then input into the target detection model to detect target objects, and the detection results are extracted. The detected target objects are input into the DeepSort multi-target tracking model. The DeepSort multi-target tracking model is initialized with input feature vectors and detection boxes. The detection results of the current frame are input into the DeepSort multi-target tracking model to update the tracking state and generate a new tracking ID. The above steps are repeated for each frame in the video stream, continuously performing target detection and tracking until all input data is processed. The running trajectories of each vehicle are obtained, completing the vehicle detection and tracking process.

[0148] This application uses the BDD100K dataset to train the benchmark YOLOv5 model and the object detection model provided in this application. It covers different weather conditions, including sunny, cloudy and rainy days, as well as different times of day and night. This dataset is one of the largest and most diverse autonomous driving datasets to date. The dataset contains 10,000 images, covering vehicle images of different models in various driving scenarios and different weather conditions. It is then divided into training and validation sets in an 8:2 ratio, and the images are preprocessed with noise addition, flipping, brightness adjustment and other methods. Since the focus of this application is on identifying vehicles in front rather than their categories, the dataset label is uniformly 'car'.

[0149] Veri-776 is a large vehicle re-identification dataset containing over 40,000 images of 776 vehicles. Each vehicle contains images taken from different angles by 2-18 cameras. The dataset is divided into training and testing sets in a 7:3 ratio. The original feature extraction network model of the Deepsort multi-object tracking model and the lightweight feature extraction network model of ShuffleNetv2 are trained for 120 rounds.

[0150] The object detection model provided in this application was compared with the YOLOv5s model and mainstream lightweight backbone network models (ShuffleNetV2, MobileNetV3, and EfficientNetV2). The experimental results are shown in Table 3.

[0151] Table 3

[0152]

[0153] As shown in Table 3, ShuffleNetv2 has a GFLOPS of only 1.8 and 842,358 parameters, but its mAP@0.5 drops to 94.60%. EfficientNetV2 and MobileNetV3 achieve a better balance between GFLOPS and parameter count, reaching 5.6 GFLOPS and 2.5 GFLOPS respectively, but their mAP@0.5 also decreases to 96.60% and 95.20% respectively. All of these lightweight models suffer varying degrees of loss in detection accuracy. While YOLOv8s offers a slight improvement in object detection accuracy compared to the model provided in this application, its GFLOPS and parameter count are nearly three times that of our optimized model. Based on the latest end-to-end real-time object detection model YOLOv10 published by Tsinghua University, although the mAP value is improved by 0.20% compared to the original model, it is still lower than the optimized model, and its GFLOPS and parameter count are nearly twice that of the optimized model.

[0154] Experiments show that by optimizing the model structure and algorithm, detection accuracy can be maintained or even improved to a certain extent while significantly reducing computational complexity and the number of parameters. The target detection model provided in this application has a GFLOPS of 10.0, which is 37% lower than the YOLOv5s model, and the number of parameters is 4,601,086, a decrease of 34.50%. Furthermore, the mAP@0.5 reaches 97.10%, higher than the YOLOv5s model. This indicates that the target detection model provided in this application successfully reduces computational complexity and the number of parameters while maintaining high accuracy, achieving a lightweight model suitable for vehicle detection tasks in practical applications.

[0155] This application also investigates the impact of different lightweight modules on model performance through ablation experiments. The GhostConv and C3GhostV2 modules in the feature extraction network are selected to replace the Conv convolutional layer in the multi-scale feature fusion network, along with the object detection loss function. As independent variables, the impact of each component on the overall performance of the model was analyzed, and the experimental results are shown in Table 4:

[0156] Table 4

[0157]

[0158] As shown in Table 4, after replacing the Conv and C3 modules in the feature extraction network with GhostConv and C3GhostV2 respectively, the model's mAP@0.5 increased from 96.80% to 96.90%, GFLOPS decreased from 15.9 to 10.6, and the number of parameters decreased from 7,022,326 to 5,042,046. This indicates that the lightweight convolutional module used in this application can effectively reduce computational complexity and the number of parameters without significantly affecting accuracy. Furthermore, replacing the Conv module in the multi-scale feature fusion network with GhostConv slightly improved the accuracy to 97%, further reduced GFLOPS to 10.0, and decreased the number of parameters to 4,601,086. This shows that applying the lightweight convolutional module in the multi-scale feature extraction network can not only further reduce computational complexity and the number of parameters, but also slightly improve the model's accuracy. Finally, after adding the object detection loss function provided in this application, the model's mAP@0.5 was further improved to 97.10%, while GFLOPS and the number of parameters remained unchanged. This fully demonstrates the advantages of the lightweight convolution module in further reducing computation and parameters while improving the original accuracy.

[0159] Weight size is one of the important indicators for measuring the lightweightness of a model, and it directly affects the model's performance on embedded devices or in resource-constrained environments. To evaluate the lightweighting effect of the object detection model and the DeepSort multi-object tracking model, this experiment will analyze and compare the model weight size of the two algorithms. The experimental results are shown in Table 5.

[0160] Table 5

[0161]

[0162] In existing technologies, the YOLOv5s+DeepSORT combination model has a weight of 24.7MB. After using the object detection model provided in this application as the detector in the tracking model, the weight is reduced to 20.28MB, a reduction of approximately 17.9%. Furthermore, by using ShuffleNetV2 as the DeepSort feature extraction network, the combined model weight is reduced to 13.62MB, a reduction of approximately 44.9% compared to the original model. The results show that by lightweighting the object detection model and the DeepSort multi-object tracking model, the model weight is significantly reduced.

[0163] The effectiveness of the vehicle detection and tracking method provided in the above embodiments is verified through specific examples below:

[0164] The vehicle tracking videos captured by the onboard camera of the Tesla Model Y were used as a validation set to evaluate the effectiveness of vehicle tracking. The dataset included three scenarios: overcast elevated roads, sunny highways, and tunnels, with each video lasting 59 seconds.

[0165] like Figure 8 The image shows a comparison of the tracking performance of the YOLOv5-Deepsort model and the object detection model + DeepSort multi-object tracking model provided in this application on vehicles on an elevated bridge under cloudy conditions. Figure 8 (a) and Figure 8 (b) shows the tracking results of the YOLOv5-Deepsort model from frames 628 to 632, indicating that it failed to identify the bus in front. Figure 8 (c) and Figure 8 (d) in the image represents the tracking result of the target detection model + DeepSort multi-target tracking model provided in this application in the corresponding frame. It can accurately identify the bus that the original algorithm failed to track and is identified as ID 16.

[0166] Figure 9The image shows the ID changes after the vehicle was re-identified following occlusion from frames 861 to 924. The two sets of images above and below also show the verification results of the YOLOv5-Deepsort model and the object detection model + DeepSort multi-object tracking model provided in this application, respectively. Figure 9 In (c), the vehicle with ID 2 was accurately identified, while Figure 9 It was not accurately identified in (a) of the data; in frames 861 to 924, Figure 9 In (a) of the diagram, vehicle ID 21 was briefly obscured by vehicle ID 19. Figure 9 In (b) of the image, after the occluded vehicle was re-identified in frame 924, the ID of the vehicle that previously had an ID of 21 was changed to 24. However, the corresponding frame below shows that the ID of the vehicle with an ID of 17 remained 17 after it was re-identified after being occluded, and no change occurred.

[0167] Figure 10 The image shows the inspection results of two methods on highway vehicles under clear skies, frames 1599 to 1606. Figure 10 (a) and Figure 10 In (b), the vehicle in front with ID 16 is partially obscured and was not recognized by the YOLOv5-Deepsort model. Figure 10 (c) and Figure 10 As shown in (d) in this application, the target detection model + DeepSort multi-target tracking model can still accurately identify it, with an ID of 4.

[0168] like Figure 11 The image shows the tracking results of two methods for highway vehicles on a sunny day from frames 1626 to 1721. Figure 11 In (a), the vehicle with ID 5 was not recognized by the YOLOv5-Deepsort model between frames 1626 and 1721 due to occlusion. However, by frame 1721... Figure 11 After (b) is detected, the ID switches to 24. Figure 11 The vehicle with ID 4 in (c) can be accurately tracked by the target detection model + DeepSort multi-target tracking model provided in this application from frame 1644 to frame 1705. Figure 11 (d) in the middle was not accurately tracked due to occlusion, while in frame 1705 ( Figure 11 When (e) is blocked, the vehicle is identified again, and ID 4 does not switch.

[0169] Figure 12 The results of two methods for tracking and verifying vehicles traveling in tunnels are shown in frames 1125 to 1128. Figure 12(a) and Figure 12 In (b), the vehicle in front of vehicle ID 16 was obscured and could not be accurately identified by the YOLOv5-Deepsort model. Figure 12 (c) and Figure 12 (d) in the figure shows that the corresponding vehicle can be identified by the target detection model + DeepSort multi-target tracking model provided in this application, and its ID is 3.

[0170] Figure 13 The image shows the tracking results of two methods for vehicles traveling in the tunnel from frame 1626 to frame 1721. Figure 13 (a) and Figure 13 In (b), there is a black sedan in front, and because it is too far away, the YOLOv5-Deepsort model is unable to track the small vehicle. Figure 13 (c) and Figure 13 (d) in the figure shows that the corresponding vehicle can be accurately tracked by the target detection model + DeepSort multi-target tracking model provided in this application, and the vehicle ID is 35.

[0171] In summary, the improved model provided in this application performs excellently in various environments, including cloudy days, highways, and tunnels, demonstrating stable and accurate vehicle identification and tracking. This indicates that the improved model has strong robustness under different conditions, while the existing YOLOv5-Deepsort model is more prone to missing vehicle detections. Furthermore, compared to the YOLOv5-Deepsort model, the improved model can more accurately identify occluded vehicles and maintains vehicle ID consistency better, avoiding ID switching issues caused by occlusion and thus improving tracking continuity and accuracy. In addition, the improved model can accurately identify and track vehicles at a greater distance, while the YOLOv5-Deepsort model often fails to identify them successfully in similar situations. Experiments show that the improved model not only achieves a lighter size to fit mobile devices but also tracks vehicles ahead more accurately and stably than the YOLOv5-Deepsort model, achieving a good balance between detection accuracy, processing efficiency, and resource consumption, demonstrating significant practical value.

[0172] Visual analysis of examples of multi-target tracking algorithms shows that the improved target detection model + DeepSort multi-target tracking model provided in this application significantly outperforms the YOLOv5-Deepsort model in accurately tracking vehicles.

[0173] This application also provides a vehicle detection and tracking device, such as... Figure 14As shown, the device specifically includes:

[0174] The object detection model acquisition module 10 is used to acquire the trained object detection model. The object detection model includes a feature extraction network, a multi-scale feature fusion network and an object detection network. The feature extraction network includes a CBS module, four convolutional attention modules and a spatial pyramid pooling module connected in series along the forward propagation direction. Each convolutional attention module includes a first phantom convolutional submodule and a C3GhostV2 submodule connected in series.

[0175] Feature extraction module 20 is used to input each frame of the video to be detected into the feature extraction network, and obtain a first feature map, a second feature map, and a third feature map based on the outputs of the second convolutional attention module, the third convolutional attention module, and the spatial pyramid pooling module; wherein, the C3GhostV2 submodule performs feature extraction on the feature maps in the following steps:

[0176] The feature maps are input into the first CBS unit and the second CBS unit respectively, and the first intermediate feature and the second intermediate feature are output.

[0177] The first intermediate feature is input into the long-range attention mechanism enhancement unit, and the third intermediate feature is output.

[0178] The second intermediate feature and the third intermediate feature are input into the feature fusion unit, and the fourth intermediate feature is output.

[0179] The fourth intermediate feature is input into the third CBS unit, and the output of the third CBS unit is used as the output of the C3GhostV2 submodule.

[0180] Feature fusion module 30 is used to input the first feature map, the second feature map and the third feature map into the multi-scale feature fusion network and output the target feature map;

[0181] The target detection module 40 is used to input the target feature map into the target detection network and output the vehicle bounding box and position information;

[0182] The target tracking module 50 is used to input the vehicle bounding boxes and position information in all frames of the video to be detected into the trained DeepSort multi-target tracking model, and output the running trajectory of each vehicle in the video to be detected.

[0183] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the vehicle detection and tracking method described above.

[0184] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A vehicle detection and tracking method, characterized in that, include: Obtain a trained target detection model; the target detection model includes a feature extraction network, a multi-scale feature fusion network and a target detection network. The feature extraction network includes a CBS module, four convolutional attention modules and a spatial pyramid pooling module connected in series along the forward propagation direction. Each of the convolutional attention modules includes a first phantom convolutional submodule and a C3GhostV2 submodule connected in series. Each frame of the video to be detected is input into the feature extraction network. Based on the outputs of the second convolutional attention module, the third convolutional attention module, and the spatial pyramid pooling module, a first feature map, a second feature map, and a third feature map are obtained. The C3GhostV2 submodule performs feature extraction on the feature maps, including the following steps: The feature maps are input into the first CBS unit and the second CBS unit respectively, and the first intermediate feature and the second intermediate feature are output. The first intermediate feature is input into the long-distance attention mechanism enhancement unit, and the third intermediate feature is output. The second intermediate feature and the third intermediate feature are input into the feature fusion unit, and the fourth intermediate feature is output. The fourth intermediate feature is input into the third CBS unit, and the output of the third CBS unit is used as the output of the C3GhostV2 submodule. The first feature map, the second feature map, and the third feature map are input into the multi-scale feature fusion network to output the target feature map; The target feature map is input into the target detection network, which outputs the vehicle bounding box and location information. The vehicle bounding boxes and position information in all frames of the video to be detected are input into the trained DeepSort multi-object tracking model, which outputs the running trajectory of each vehicle in the video to be detected.

2. The vehicle detection and tracking method according to claim 1, characterized in that, The training process of the target detection model includes: Obtain the training set of images to be detected; The images in the training set of images to be detected are input into the target detection model, and the predicted vehicle bounding boxes in the images are output. Based on the intersection-union ratio of the predicted vehicle bounding box and the real vehicle bounding box, the Euclidean distance between the center point of the predicted vehicle bounding box and the center point of the real vehicle bounding box, and the diagonal length, width and height of the minimum bounding rectangle covering the predicted vehicle bounding box and the real vehicle bounding box, a loss function that minimizes the width and height difference is constructed. Based on the loss function that minimizes the width and height difference and the intersection-union ratio of the predicted vehicle bounding box and the real vehicle bounding box, a target detection loss function is constructed; The target detection model is trained using images from the training set of images to be detected until the value of the target detection loss function is minimized, thus obtaining a trained target detection model.

3. The vehicle detection and tracking method according to claim 2, characterized in that, The formula for calculating the intersection-union ratio (IUR) of the predicted vehicle bounding box and the actual vehicle bounding box is as follows: ， in, This represents the intersection-union ratio (IoU) between the predicted vehicle bounding box and the actual vehicle bounding box. This represents the predicted vehicle bounding box; Represents the actual vehicle bounding box; The loss function for minimizing the width-to-height difference is expressed as: ， in, This represents the loss function that minimizes the difference between width and height. express loss; Indicates distance loss; Indicates the loss in side length; This represents the square of the Euclidean distance between the center point of the predicted vehicle bounding box and the center point of the actual vehicle bounding box; This represents the diagonal length of the smallest bounding rectangle that covers both the predicted and actual vehicle bounding boxes. This represents the square of the difference between the width of the predicted vehicle bounding box and the width of the actual vehicle bounding box. This represents the width of the minimum bounding rectangle that covers both the predicted vehicle bounding box and the actual vehicle bounding box. This represents the square of the difference between the height of the predicted vehicle bounding box and the height of the actual vehicle bounding box. This represents the height of the minimum bounding rectangle that covers both the predicted vehicle bounding box and the actual vehicle bounding box. The target detection loss function is expressed as follows: ， in, This represents the target detection loss function; This represents the hyperparameter used to control the curvature of the curve.

4. The vehicle detection and tracking method according to claim 1, characterized in that, The multi-scale feature fusion network includes a bottom-up feature fusion module, a top-down feature fusion module, and a convolutional module; The bottom-up feature fusion module includes a second phantom convolution submodule, a first upsampling submodule, a first feature fusion submodule, a first C3 submodule, a third phantom convolution submodule, a second upsampling submodule, and a second feature fusion submodule, which are sequentially connected in series along the forward propagation direction. The top-down feature fusion module includes a second C3 submodule, a fourth phantom convolution submodule, a third feature fusion submodule, a third C3 submodule, a fifth phantom convolution submodule, a fourth feature fusion submodule, and a fourth C3 submodule, which are connected in series along the forward propagation direction. The convolution module includes a sixth phantom convolution submodule, a seventh phantom convolution submodule, and an eighth phantom convolution submodule.

5. The vehicle detection and tracking method according to claim 4, characterized in that, The first feature map, the second feature map, and the third feature map are input into the multi-scale feature fusion network, and the output target feature map includes: The first feature map is input into the second feature fusion submodule of the bottom-up feature fusion module; the second feature map is input into the second feature fusion submodule of the bottom-up feature fusion module; the third feature map is input into the second phantom convolution submodule of the bottom-up feature fusion module; The feature map output by the second phantom convolution submodule is input into the fourth feature fusion submodule of the top-down feature fusion module; the feature map output by the third phantom convolution submodule is input into the third feature fusion submodule of the top-down feature fusion module; the feature map output by the second feature fusion submodule is input into the second C3 submodule of the top-down feature fusion module. The feature map output by the second C3 submodule is input into the sixth phantom convolution submodule to output the first target feature map; the feature map output by the third C3 submodule is input into the seventh phantom convolution submodule to output the second target feature map; the feature map output by the fourth C3 submodule is input into the eighth phantom convolution submodule to output the third target feature map.

6. The vehicle detection and tracking method according to claim 5, characterized in that, The target detection network includes a first detection head, a second detection head, and a third detection head.

7. The vehicle detection and tracking method according to claim 6, characterized in that, The target feature map is input into the target detection network, and the output vehicle bounding box and location information include: The first target feature map is input into the first detection head, and the coordinates of the first predicted bounding box, the confidence that the first predicted bounding box is a vehicle bounding box, and the probability distribution of the first predicted bounding box belonging to each category of bounding box are output. The second target feature map is input into the second detection head, and the coordinates of the second predicted bounding box, the confidence that the second predicted bounding box is a vehicle bounding box, and the probability distribution of the second predicted bounding box belonging to each category of bounding box are output. The third target feature map is input into the third detection head, and the coordinates of the third predicted bounding box, the confidence that the third predicted bounding box is a vehicle bounding box, and the probability distribution of the third predicted bounding box belonging to each category of bounding box are output.

8. The vehicle detection and tracking method according to claim 1, characterized in that, The feature extraction network in the DeepSort multi-object tracking model is ShuffleNetV2.

9. A vehicle detection and tracking device, characterized in that, include: The target detection model acquisition module is used to acquire a trained target detection model. The target detection model includes a feature extraction network, a multi-scale feature fusion network, and a target detection network. The feature extraction network includes a CBS module, four convolutional attention modules, and a spatial pyramid pooling module connected in series along the forward propagation direction. Each of the convolutional attention modules includes a first phantom convolutional submodule and a C3GhostV2 submodule connected in series. The feature extraction module is used to input each frame of the video to be detected into the feature extraction network, and obtain a first feature map, a second feature map, and a third feature map based on the outputs of the second convolutional attention module, the third convolutional attention module, and the spatial pyramid pooling module; wherein, the C3GhostV2 submodule performs feature extraction on the feature maps in the following steps: The feature maps are input into the first CBS unit and the second CBS unit respectively, and the first intermediate feature and the second intermediate feature are output. The first intermediate feature is input into the long-distance attention mechanism enhancement unit, and the third intermediate feature is output. The second intermediate feature and the third intermediate feature are input into the feature fusion unit, and the fourth intermediate feature is output. The fourth intermediate feature is input into the third CBS unit, and the output of the third CBS unit is used as the output of the C3GhostV2 submodule. The feature fusion module is used to input the first feature map, the second feature map, and the third feature map into the multi-scale feature fusion network and output the target feature map. The target detection module is used to input the target feature map into the target detection network and output the vehicle bounding box and position information; The target tracking module is used to input the vehicle bounding boxes and position information in all frames of the video to be detected into the trained DeepSort multi-target tracking model, and output the running trajectory of each vehicle in the video to be detected.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the vehicle detection and tracking method according to any one of claims 1-8.

Images