City road target detection and tracking identification method based on deepsort algorithm

Abstract

The application provides a kind of city road target detection and tracking identification method based on Deepsort algorithm, comprising: obtaining city road target image, the image obtained is preprocessed, city road target detection dataset is constructed, city road target detection model and city road target tracking model are constructed, city road target detection model is trained based on city road target detection dataset, city road target tracking model is trained based on the preset city road target tracking dataset, obtain model, input the video to be detected into final city road target detection model and city road target tracking model, city road target detection is carried out through city road target detection model, based on the detection result, the detection result is tracked through city road target tracking model, and the tracking result is output.The application optimizes the target detection and tracking process, and realizes the precision improvement of target tracking.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, and in particular to a method for urban road target detection and tracking based on the Deepsort algorithm. Background Technology

[0002] In Intelligent Transportation Systems (ITS), accurate identification and tracking of vehicles, pedestrians, and non-motorized vehicles on the road are key technologies for achieving road safety and traffic management.

[0003] Currently, existing urban road target detection and tracking methods suffer from poor detection and tracking performance due to factors such as complex road environments, varying lighting conditions, and mutual occlusion of targets. These methods also suffer from high computational load and low accuracy, making them inconvenient to use. Therefore, it is essential to design an urban road target detection and tracking method based on the Deepsort algorithm. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a method for urban road target detection and tracking based on the Deepsort algorithm.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A method for urban road target detection and tracking based on the Deepsort algorithm includes:

[0007] Acquire target images of urban roads;

[0008] The acquired images are preprocessed to construct an urban road target detection dataset;

[0009] Construct urban road target detection model and urban road target tracking model;

[0010] The urban road target detection model is trained based on the urban road target detection dataset, and the urban road target detection and tracking model is trained based on the preset urban road target detection and tracking dataset to obtain the final urban road target detection model and urban road target detection and tracking model.

[0011] The video to be detected is input into the final urban road target detection model and urban road target tracking model. The urban road target detection model is used to detect urban road targets. Based on the detection results, the urban road target tracking model is used to track the detection results and output the tracking results.

[0012] Preferably, acquiring an image of the urban road target involves:

[0013] Images were acquired through photography, online downloads, and publicly available datasets. Target images of urban roads were filtered according to six categories: car, person, bus, bicycle, train, and motorcycle, covering multiple weather scenarios including sunny, cloudy, rainy, and foggy days.

[0014] Preferably, the acquired images are preprocessed to construct an urban road target detection dataset, specifically as follows:

[0015] Acquire low-light images from urban road target images and perform image enhancement processing on them;

[0016] Randomly select a portion of the target images of urban roads and perform image noise addition and image blurring on them;

[0017] The LabelImg annotation tool is used to annotate the original urban road target images, the low-light images after image enhancement, and some images after image noise addition and blurring to construct an urban road target detection dataset.

[0018] Preferably, a low-light image is acquired from the target image of the urban road, and image enhancement processing is performed on it, specifically as follows:

[0019] Build an image-to-patch augmentation model and pre-train it based on a pre-defined dataset;

[0020] Low-light images of urban road targets are acquired and input into a pre-trained Image-to-patch enhancement model to obtain processed low-light images.

[0021] Preferably, the urban road target detection model is an improved YOLOv4 model, and the specific improvements include:

[0022] Replace the standard convolutions in the traditional YOLOv4 model with depthwise separable convolutions;

[0023] In addition to the three output layers of the traditional YOLOv4 model, an output layer with a grid of 104×104 is added;

[0024] Introduce some Inceptionv3 structures into the Head part of the traditional YOLOv4 model.

[0025] Preferably, the urban road target detection and tracking model is the DeepSort model.

[0026] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0027] This invention provides a method for urban road target detection and tracking based on the Deepsort algorithm. The method includes: acquiring urban road target images; preprocessing the acquired images; constructing an urban road target detection dataset; constructing an urban road target detection and tracking model; training the urban road target detection and tracking model based on the urban road target detection dataset to obtain the final urban road target detection and tracking model; inputting the video to be detected into the final urban road target detection and tracking model to obtain the detection result. This invention improves the accuracy of target tracking by optimizing the target detection and tracking process, making it easier to use. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A flowchart of the method provided in an embodiment of the present invention;

[0030] Figure 2 A diagram illustrating the training strategy for Image-to-patch augmented models;

[0031] Figure 3 This is a schematic diagram of the Patch-net network structure;

[0032] Figure 4 A comparative illustration of the Image-to-patch enhancement model;

[0033] Figure 5 Illustration of adding noise to an image (a is the original image, b is the image after adding salt and pepper noise);

[0034] Figure 6 This is a schematic diagram of image blurring (a is the original image, b is the image after Gaussian blur is applied);

[0035] Figure 7 This is a schematic diagram of the standard convolution process;

[0036] Figure 8 This is a schematic diagram of the depthwise separable convolution process;

[0037] Figure 9 This is a schematic diagram of the YOLO Head structure;

[0038] Figure 10 This is a schematic diagram of the Inceptionv3 Head structure;

[0039] Figure 11 A schematic diagram of the improved YOLOv4 network framework;

[0040] Figure 12 shows the effect of the improved YOLOv4 model (a is the original image, b is the effect of the YOLOv4 model, and c is the effect of the improved YOLOv4 model).

[0041] Figure 13 This is a schematic diagram of a target tracking framework based on improved YOLOv4 and DeepSort;

[0042] Figure 14 This is a schematic diagram of the tracking process for scenario one and scenario two;

[0043] Figure 15 This is a schematic diagram of the tracking process for scenario three;

[0044] Figure 16 This is a schematic diagram of the tracking process for scenarios four and five. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] The purpose of this invention is to provide a method for urban road target detection and tracking based on the Deepsort algorithm. By optimizing the target detection and tracking process, the accuracy of target tracking is improved, making it easier to use.

[0047] Figure 1 The method flowchart provided in the embodiments of the present invention is as follows: Figure 1 As shown, this invention provides a method for urban road target detection and tracking based on the Deepsort algorithm, including:

[0048] Step 100: Acquire target images of urban roads;

[0049] Step 200: Preprocess the acquired images to construct an urban road target detection dataset;

[0050] Step 300: Construct an urban road target detection model and an urban road target tracking model;

[0051] Step 400: Train the urban road target detection model based on the urban road target detection dataset, and train the urban road target detection and tracking model based on the preset urban road target detection and tracking dataset to obtain the final urban road target detection model and urban road target detection and tracking model.

[0052] Step 500: Input the video to be detected into the final urban road target detection model and urban road target tracking model. Perform urban road target detection through the urban road target detection model. Based on the detection results, track the detection results through the urban road target tracking model and output the tracking results.

[0053] In step 100, the target image of the urban road is obtained, specifically as follows:

[0054] Images were acquired through photography, online downloads, and publicly available datasets. Target images of urban roads were filtered according to six categories: car, person, bus, bicycle, train, and motorcycle, covering multiple weather scenarios including sunny, cloudy, rainy, and foggy days.

[0055] In step 200, the acquired images are preprocessed to construct an urban road target detection dataset, specifically as follows:

[0056] Step 201: Obtain the low-light image from the urban road target image and perform image enhancement processing on it;

[0057] Step 202: Randomly select a portion of the city road target images and perform image noise addition and image blurring processing on them;

[0058] Step 203: Based on the LabelImg annotation tool, the original urban road target images, the low-light images after image enhancement, and some images after image noise addition and image blurring are annotated to construct an urban road target detection dataset.

[0059] In step 201, a low-light image is acquired from the urban road target image, and image enhancement processing is performed on it, specifically as follows:

[0060] Build an image-to-patch augmentation model and pre-train it based on a pre-defined dataset;

[0061] Low-light images of urban road targets are acquired and input into a pre-trained Image-to-patch enhancement model to obtain processed low-light images.

[0062] A detailed introduction to the Image-to-patch enhancement model:

[0063] The mathematical representation of the image-to-patch augmentation model is:

[0064] H(x,y)=L(x,y)+I×P(x,y)

[0065] Where L(x, y) is the input low-light image, and P(x, y) is the inpainted image, which is an illumination estimation image. In the dark area of ​​this intermediate image, it mainly contains the incremental information of the low-light image to be restored to the normal image, while in the normal area, it is mainly to maintain the harmony between the dark area and the normal area. Therefore, the pixel values ​​of the corresponding normal area in the inpainted image are mainly close to 0 or directly equal to 0, which can well protect the highlight information in the original image.

[0066] By fusing with the original image, image details in darkness are restored. The enhancement level (I) constrains the strength of the patch image, primarily limiting the degree of enhancement of dark information to prevent excessive enhancement of dark areas from causing overall image inconsistency. Furthermore, the introduced enhancement level (I) is an adjustable parameter, effectively improving the generalization ability of the enhancement network. Existing end-to-end enhancement methods often only handle low-light images with brightness similar to the training set. The image-to-patch model, by adjusting the overall strength of the patch image, makes brightness enhancement more flexible, meeting the image enhancement needs under different exposure conditions.

[0067] In the image-to-patch enhancement model, the convolutional neural network needs to learn the mapping from the low-light image to the patched image. The mathematical description of the network's optimization objective is as follows:

[0068] argmin[Y(x,y)-L(x,y)-I×P(x,y)]

[0069] Where Y(x, y) is the reference image, L(x, y) is the input low-light image, P(x, y) is the patched image output by the convolutional neural network, and I is the enhancement level, which is set to 1 during the training phase and has a maximum value of 1 during the testing phase. This value can be quantized into 10 levels, and the enhancement level can be constrained by setting this value. The training strategy for the network in the image-to-patch enhancement model is as follows: Figure 2 As shown.

[0070] The strategy in the network training phase can be understood as residual learning. By learning the residuals of normal images and low-light images, the network can accelerate its convergence speed and reduce its focus on normal regions. Therefore, a more lightweight network structure can be designed based on this.

[0071] To reduce resource consumption, this invention proposes that the Patch-net network consists of only 6 convolutional layers, with the following structure: Figure 3 As shown;

[0072] The network mainly consists of three parts: feature extraction, feature mapping, and feature reconstruction. The specific network parameters are shown in Table 1. First, a feature extraction layer composed of three 3×3 convolutions + PReLU is used to extract features from the input image, increasing the dimension of the feature map to 64. Then, a 1×1 convolution is used for spatial transformation, converting the extracted 64 feature maps into 32, transforming the image feature domain into the residual domain. Next, two 3×3 convolutions are used to reconstruct the inpainted image to predict the generated inpainted image. For the generation of the inpainted image, this invention does not limit the range of pixel values; by using PReLU instead of the ReLU activation function, negative values ​​are also possible in the inpainted image. The final enhanced image is obtained by fusing the inpainted image with the input image.

[0073] Table 1 Patch-net Network Parameter Settings

[0074]

[0075]

[0076] To better constrain the network-generated patch image and fuse it with the input image to generate a high-quality, properly exposed image, L1 loss function and SSIM loss function were used during the training phase. The L1 loss function is expressed as:

[0077]

[0078] Wherein, F(X) org X(θ) is the predicted output graph of the network. org For low-light conditions, Y ref For reference, ||·||1 represents the first norm. The use of L1 loss instead of L2 loss is primarily because the network focuses on details in dark areas. After fusing with the dark areas of the input image, these details need to be recovered. Using L2 loss would result in overly smooth details in the dark areas, causing blurring. In addition to L1 loss, this invention also uses the SSIM loss function to constrain the overall structural information of the image. SSIM can be expressed as:

[0079]

[0080] Where z is the enhanced image after fusing the patched image and the input image, y is the reference image, and μ z It enhances the average pixel value of the image, μ y The average pixel value of the reference image. and These are the variances of the enhanced image and the reference image, respectively, σ z,yThis represents the covariance between the enhanced image and the reference image. C1 and C2 are two constants to avoid division by zero. For the SSIM loss function, it is expressed as:

[0081]

[0082] Among them, z i For the i-th output image, y i Using the i-th reference image, after multiple experiments, this invention achieves the best results by setting the weight of the SSIM loss function to 0.1. The loss function of the entire network is expressed as:

[0083] Loss = L1 + 0.1 × L SSIM

[0084] This invention provides an embodiment that compares the algorithm with existing high-performing algorithms, including Dong, LIME, SRIE, GLADNet, NPE, and BIMEF, as illustrated in the diagram below. Figure 4 As shown, the method described in this invention achieves better results.

[0085] In step 202, a portion of the city road target image is randomly selected and subjected to image noise addition and image blurring processing, specifically as follows:

[0086] A portion of images from randomly selected urban road target images is processed by adding noise to half of them, for example, salt-and-pepper noise. Salt-and-pepper noise is a mixed noise containing both pepper and salt noise. Pepper noise appears as black spots in the image, while salt noise appears as white spots. Adding salt-and-pepper noise is achieved by randomly selecting image pixels and setting them as high-brightness and low-grayscale points. Images with added salt-and-pepper noise have a certain similarity to images affected by noise, effectively simulating the effect of noisy images. This enhances the robustness of deep learning models to noisy and blurred images. The results of adding salt-and-pepper noise to images in the dataset are shown below. Figure 5 As shown;

[0087] The other half of the image is blurred, for example, using Gaussian blur. Gaussian blur is a commonly used data smoothing method in image processing. It makes the image smoother and more blurred. The specific steps are: first, scan each pixel in the image; then, apply a weighted average of the surrounding pixels using a Gaussian curve; finally, remove details from the image to blur it. The image obtained after Gaussian blur has a certain similarity to images obtained due to vehicle vibration or excessive speed, effectively simulating blurred images and thus enhancing the robustness of deep learning models to blurred images. The results of Gaussian blurring of images in the dataset are shown below. Figure 6 As shown;

[0088] In step 203, the original urban road target image, the low-light image after image enhancement, and some images after image noise reduction and blurring are labeled using the LabelImg annotation tool to construct an urban road target detection dataset, specifically:

[0089] All original urban road target images (including low-light images before processing and images without image noise addition and blurring processing), as well as low-light images after image enhancement processing and some images after image noise addition and blurring processing, are obtained and labeled. The labels are then labeled using the LabelImg labeling tool to generate an urban road target detection dataset.

[0090] The urban road target detection model is an improved YOLOv4 model;

[0091] First, we will introduce the improved YOLOv4 model, including the following improvements:

[0092] (a) Replace the standard convolutions in the traditional YOLOv4 model with depthwise separable convolutions, specifically:

[0093] The YOLOv4 model uses many standard 3×3 convolutions, resulting in a large number of parameters and computational cost, making it unsuitable for detection on devices with limited computing power and memory resources. Therefore, depthwise separable convolutions are introduced to replace standard convolutions, which significantly reduces the number of parameters and computational cost of the model and improves the detection speed.

[0094] The process of standard convolution is as follows Figure 7 As shown. For example, when its input feature map is D x ×D y When the number of channels is ×3, the feature map is convolved with N 3×3 convolution kernels respectively, and finally a new feature map with N channels is obtained.

[0095] The process of depthwise separable convolution is as follows Figure 8 As shown. When its input is also D x ×D y When dealing with a 3×3 feature map, depthwise separable convolution first convolves the input feature map layer by layer with three 3×3 convolution kernels to obtain a feature map with the same number of channels as the input. Then, this feature map is convolved point by point with N 1×1 convolution kernels to obtain a new feature map with N channels.

[0096] The formula for calculating the number of parameters in a convolution when considering bias is shown below:

[0097] P conv = (W×H×S+1)×N

[0098] In the formula, the parameters are as follows:

[0099] P conv The number of parameters in the convolution;

[0100] The size of the W×H convolution kernel;

[0101] S is the number of input channels for the convolution;

[0102] N is the number of output channels of the convolution.

[0103] When the input is D x ×D y The feature map of size S is convolved with N 3×3 convolution kernels, and the final output is D. h ×D w When the number of parameters P in standard convolution is ×N, conv The calculation formula is as follows:

[0104] P conv1 = (3 × 3 × S + 1) × N = (9 S + 1) × N

[0105] Depthwise separable convolution first convolves the input feature map layer by layer with three 3×3 convolution kernels, and then convolves the resulting new feature map point by point with N 1×1 convolution kernels. The number of parameters P in depthwise separable convolution is... conv2 As shown below:

[0106] P conv2 =P1+P2=9S+(S+1)×N

[0107] In the formula, the parameters are as follows:

[0108] P1 represents the number of parameters during layer-by-layer convolution;

[0109] P2 represents the number of parameters used in pointwise convolution.

[0110] Comparing the above formulas, we obtain the following formula:

[0111]

[0112] Where k1 is the ratio of the number of parameters used in depthwise separable convolution to the number of parameters used in standard convolution. Since the number of input channels S and the number of output channels N are both large in YOLOv4, k1 is much less than 1. The computational cost of convolution is calculated as follows:

[0113] C conv =W×H×S×N×D h ×D w

[0114] In the formula, the parameters are as follows:

[0115] C conv The computational cost of convolution;

[0116] D h ×D w This specifies the size of the output feature map.

[0117] When the input is D x ×D y The feature map of size S is convolved with N 3×3 convolution kernels, and the final output is D. h ×D w When the value is ×N, the computational cost of standard convolution, Ccony1, is as follows:

[0118] C conv1 =3×3×S×N×D h ×D w =9×S×N×D h ×D

[0119] Depthwise separable convolution first performs layer-by-layer convolution, then point-by-point convolution. The computational cost of depthwise separable convolution is C. conv2 As shown below:

[0120] C conv2 =C1+C2=9S×D h ×D w +S×N×D h ×D w

[0121] In the formula, the parameters are as follows:

[0122] C1 represents the computational cost of layer-by-layer convolution;

[0123] C2 represents the computational cost of pointwise convolution.

[0124] Comparing the above formulas, we obtain the following formula:

[0125]

[0126] Where k2 is the ratio of the computational cost of depthwise separable convolution to that of standard convolution. Since the number of output channels N in YOLOv4 is relatively large, k2 is much less than 1.

[0127] In summary, using depthwise separable convolution can achieve similar results to standard convolution while significantly reducing the number of parameters and computational cost. Therefore, depthwise separable convolution is introduced to replace standard convolution.

[0128] (II) In addition to the three output layers of the traditional YOLOv4 model, an additional output layer with a mesh size of 104×104 is added, specifically as follows:

[0129] In the YOLOv4 network architecture, the YOLO Head only outputs feature maps with grid sizes of 13×13, 26×26, and 52×52. In deep learning networks, the receptive field (RF) varies for feature maps of different scales. The receptive field refers to the size of the region corresponding to a pixel in the output feature map obtained by the convolutional neural network on the input image. Therefore, the extracted features differ depending on the receptive field. The formula for calculating the receptive field is as follows:

[0130] F i =S i (F i-1 -1)+K i

[0131] In the formula, the parameters are as follows:

[0132] F i The receptive field on the feature map of the i-th layer;

[0133] S i Let be the stride of the i-th convolutional layer;

[0134] F i-1 The receptive field is the feature map of the (i-1)th layer;

[0135] K i is the kernel size of the i-th convolutional layer.

[0136] As the formula shows, as the model network becomes deeper, the receptive field corresponding to its feature map also increases. However, in urban traffic, the acquired images often show that targets closer to the camera occupy a larger number of pixels, while targets farther away occupy a relatively smaller area. Therefore, in deeper network models, more information about small distant vehicles is lost, which can easily lead to missed detections and reduce the model's detection performance.

[0137] The detection head portion of the YOLOv4 network only utilizes features at three scales, resulting in a limited number of grid cells and insufficient utilization of shallow information within the network structure. Furthermore, the YOLOv4 model has a relatively deep network structure, leading to significant information loss regarding small, distant targets during detection, increasing the likelihood of missed detections and reducing the model's detection performance. Therefore, a 104×104 grid cell is added to the existing three output layers. The 52×52 grid cell feature layer from the path aggregation network PANet is upsampled to obtain a 104×104 grid cell feature layer. This 104×104 grid cell feature layer is then stacked with a shallow 104×104 grid cell feature layer and subjected to five convolutions to create a new feature layer. This new feature layer is used as input to the detection head portion for detection, resulting in a 104×104 grid cell output layer. Simultaneously, it undergoes downsampling and is stacked with the deeper feature layer through convolution operations, providing more feature information to the detection network and improving the model's target detection performance.

[0138] (III) Introducing a portion of the Inceptionv3 structure into the Head section of the traditional YOLOv4 model, specifically:

[0139] In the head part of the YOLOv4 network, the detection accuracy is often related to the feature information obtained after the feature fusion network. The more refined the input features, the greater the probability of the target being detected. Generally, in order to improve the detection accuracy of the model, the depth of the network model is usually increased to obtain more refined features, thereby achieving the purpose of improving the detection accuracy of the network model. However, increasing the depth of the network model will also lead to a significant increase in the number of network parameters and computational cost, resulting in an increase in model size and a decrease in detection speed.

[0140] The Inceptionv3 network features a sparse structure while possessing the high performance of a dense matrix. The Inceptionv3 network is mainly composed of different modules, the most important of which is to perform multiple convolutions on the input using kernel sizes of 1×1, 3×3, and 5×5. Finally, the outputs containing feature information at different scales are stacked and fused. This structure enables the model to improve its detection performance without significantly increasing the computational load.

[0141] When performing detection in the Head section, the YOLOv4 network first performs a 3×3 convolution on the feature map input to the Head section. Then, it normalizes the result and applies the Leaky ReLU activation function. Finally, it performs feature integration using a 1×1 convolution. The YOLO Head structure is as follows: Figure 9 As shown.

[0142] To further enhance the target detection performance of the YOLOv4 network, the head part of the YOLOv4 network model is improved by introducing a portion of the Inceptionv3 structure. This allows the head part to perform convolution operations on the input using kernels of different sizes during detection, thereby obtaining receptive fields of varying sizes. The newly obtained head part, denoted as the Inceptionv3 Head, has the following structure: Figure 10 As shown.

[0143] Depend on Figure 10 As can be seen, the Inceptionv3 Head structure has four parallel paths. The first path performs a single convolution operation on the input using a 1×1 kernel, followed by normalization and Leaky ReLU activation on the result. The second path first performs a single convolution operation, normalization, and Leaky ReLU activation on the input using a 1×1 kernel, then performs a single convolution operation, normalization, and Leaky ReLU activation on the result using a 5×5 kernel. The third path first performs a single convolution operation, normalization, and Leaky ReLU activation on the input using a 1×1 kernel, then performs two consecutive convolution operations, normalization, and Leaky ReLU activation on the result using a 3×3 kernel. The fourth path first performs a 3×3 average pooling operation on the input, then performs a single convolution operation, normalization, and Leaky ReLU activation on the pooled result using a 1×1 kernel. The ReLU activation function is applied; finally, the outputs of these four paths are stacked, and a 1×1 convolutional kernel is used for feature integration. By introducing the Inceptionv3 structure into the Head part, the Head part can obtain receptive fields of different sizes when performing vehicle detection, and the model's target detection performance can be further improved without significantly increasing the computational cost.

[0144] After the above three improvements, the final improved network framework of the YOLOv4 model is as follows: Figure 11 As shown. The introduction of depthwise separable convolution is as follows: Figure 11 As shown in the improved figure 1 (with dashed box), depthwise separable convolution is represented by convolution_DW, and an output layer with a grid of 104×104 is added as follows. Figure 11 As shown in the improved dashed box 2, the Inceptionv3 structure is introduced into the Head section as follows: Figure 11 The improved dashed box is shown in Figure 3.

[0145] During the training process of large convolutional neural network models, due to the complexity of the network and the sheer volume of data samples (e.g., the ImageNet dataset contains over ten million images), training the model from scratch would be extremely time-consuming. Conversely, if the training dataset is too small, overfitting can easily occur, reducing the model's training effectiveness. Therefore, this invention uses the weights of the YOLOv4 model as pre-training weights for the improved YOLOv4 model. This effectively improves the training efficiency of the improved YOLOv4 model while reducing training time, resulting in better training performance.

[0146] The training environment for the improved YOLOv4 model was configured. This invention uses PyCharm on a Windows 10 computer to train and test the improved YOLOv4 model. The deep learning framework used is PyTorch. Other computer configurations and required training environment are shown in Table 2.

[0147] Table 2 Experimental Environment Configuration Table

[0148]

[0149]

[0150] The present invention provides an embodiment as shown in Figure 12, wherein, Figure 12a This is the original image. Figure 12b This is a schematic diagram illustrating the detection performance of the YOLOv4 model. Figure 12c This is a schematic diagram illustrating the detection performance of the improved YOLOv4 model. The improved YOLOv4 model demonstrates better detection performance.

[0151] The urban road target detection and tracking model is the DeepSort model, which will be described in detail below:

[0152] This invention primarily involves using an improved YOLOv4 model to detect objects in an input video, then employing the DeepSort model to track the objects within the video, and finally outputting the tracking results. The object tracking framework based on the improved YOLOv4 and DeepSort is as follows: Figure 13 As shown;

[0153] Taking vehicle detection and tracking as an example, from Figure 13 It can be seen that the tracking algorithm based on the improved YOLOv4 and DeepSort has the following five situations when performing target tracking:

[0154] (1) Scenario 1: Kalman filter prediction result is certain and cascade matching is successful. For the input vehicle video, the improved YOLOv4 model is first used for vehicle detection. At the same time, the Kalman filter algorithm is used to predict the target vehicle tracking box in the next frame of the input video. There are two states: certain and uncertain. Predicting the target vehicle is a certain state, and otherwise it is an uncertain state. After the Kalman filter prediction result is certain, the detection box obtained by the improved YOLOv4 model is associated with the tracking box predicted by the Kalman filter. The data association method is cascade matching. When the detection box and the tracking box are successfully matched, the Kalman filter algorithm is used to update the predicted tracking box to reduce the error. Finally, the prediction, matching and update operations are continuously repeated until the input video tracking ends.

[0155] (2) Case 2: The Kalman filter prediction result is determined, but the cascade matching result fails, while the IOU matching is successful.

[0156] During the prediction phase, the Kalman filter algorithm's prediction result remains certain. However, during the concatenation matching of detection boxes and tracking boxes, matching fails. There are two reasons for this failure: tracking matching failure and detection matching failure. Tracking matching failure might be due to missed detections during the detection phase, resulting in the predicted tracking box not having a matching detection box. Detection matching failure might be due to a new target vehicle entering the scene, preventing a tracking box from being concatenated with the detection box. When concatenation matching fails, IOU matching is performed. If the detection box and tracking box match successfully, the Kalman filter algorithm is used to update the tracking box to reduce errors. Finally, this prediction, matching, and update operation is continuously repeated until the input video tracking is complete.

[0157] The tracking processes for scenarios one and two are as follows: Figure 14 As shown.

[0158] (3) Case 3: The Kalman filter prediction result is certain, but the cascade matching result fails, and the IOU matching result also fails. Similar to Case 2, in the prediction stage, the Kalman filter algorithm predicts a certain result; in the cascade matching stage, the matching fails, and then the IOU matching stage begins. If the IOU matching also fails, it is determined whether the tracking matching failed or the detection matching failed. If the detection matching failed, a new tracking box is created and set to an uncertain state, and then the observation stage begins. If three consecutive frames are hit, the uncertain state is changed to a certain state, and prediction, matching, and update operations are performed. If the tracking matching failed, it is determined whether the tracking box is in a certain state or an uncertain state. If it is in an uncertain state, the tracking box is deleted. If it is in a certain state, it is checked whether the set time threshold is exceeded. If the time threshold is exceeded, the tracking box is also deleted. If the time threshold is not exceeded, the three observation stages begin. If the observation is successful, prediction, matching, and update operations are performed. The tracking process for Case 3 is as follows: Figure 15 As shown.

[0159] (4) Scenario 4: Kalman filter prediction result is uncertain, but IOU matching is successful. In the prediction stage, if the prediction result of the Kalman filter algorithm is uncertain, it directly enters the IOU matching stage. When the detection box and the tracking box are successfully matched, the Kalman filter algorithm is used to update the predicted tracking box to reduce the error. Similarly, the prediction, matching and update operations are continuously looped until the input video tracking ends.

[0160] (5) Case 5: The Kalman filter prediction result is uncertain and the IOU matching result is unsuccessful. During the prediction stage, the prediction result of the Kalman filter algorithm is uncertain, and the detection box and the tracking box fail to match during IOU matching. In this case, as in Case 3, it is necessary to determine whether the tracking matching failed or the detection matching failed.

[0161] The tracking process for scenarios four and five is as follows: Figure 16 As shown.

[0162] This invention provides an embodiment for comparing the tracking performance of different models, and the specific comparison results are shown in Table 3.

[0163] Table 3 Comparison of Tracking Performance of Different Models

[0164]

[0165] As shown in Table 3, the vehicle tracking performance of the improved YOLOv4+DeepSort model proposed in this invention is superior to other models.

[0166] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0167] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for urban road target detection and tracking based on the Deepsort algorithm, characterized in that, include: Acquire target images of urban roads; The acquired images are preprocessed to construct an urban road target detection dataset; Construct urban road target detection model and urban road target tracking model; The urban road target detection model is trained based on the urban road target detection dataset, and the urban road target detection and tracking model is trained based on the preset urban road target detection and tracking dataset to obtain the final urban road target detection model and urban road target detection and tracking model. The video to be detected is input into the final urban road target detection model and urban road target tracking model. The urban road target detection model is used to detect urban road targets. Based on the detection results, the urban road target tracking model is used to track the detection results and output the tracking results. The low-light image of the target image in the urban road is acquired, and image enhancement processing is performed on it, specifically as follows: Build an image-to-patch augmentation model and pre-train it based on a pre-defined dataset; Low-light images of urban road targets are acquired and input into a pre-trained Image-to-patch enhancement model to obtain processed low-light images. The mathematical representation of the image-to-patch augmentation model is: Where L(x,y) is the input low-light image, P(x,y) is the patched image output by the convolutional neural network, the dark areas of the patched image contain incremental information for restoring the low-light image to a normal image, and the pixel values ​​of the corresponding normal areas in the patched image are close to 0; I is the enhancement level, which is used to constrain the intensity of the patched image and the degree of enhancement of dark information; the enhancement level I is an adjustable parameter. In the image-to-patch enhancement model, the convolutional neural network learns the mapping from a low-light image to a patch image. The mathematical description of the optimization objective of the convolutional neural network is as follows: Where Y(x, y) is the reference figure; A convolutional neural network consists of three parts: feature extraction, feature mapping, and feature reconstruction. The convolutional neural network first uses a feature extraction layer composed of three 3×3 convolutions + PReLU to extract the features of the input image, increasing the dimension of the feature map to 64. Then, a 1×1 convolution is used to transform the space, converting the extracted 64 feature maps into 32, and transforming the image feature domain into the residual domain. Finally, two 3×3 convolutions are used to reconstruct the patched image to predict the generated patched image.

2. The urban road target detection and tracking method based on the Deepsort algorithm according to claim 1, characterized in that, Obtaining target images of urban roads, specifically: Images were acquired through photography, online downloads, and publicly available datasets. Target images of urban roads were filtered according to six categories: car, person, bus, bicycle, train, and motorcycle, covering multiple weather scenarios including sunny, cloudy, rainy, and foggy days.

3. The urban road target detection and tracking method based on the Deepsort algorithm according to claim 1, characterized in that, The acquired images are preprocessed to construct an urban road target detection dataset, specifically as follows: Acquire low-light images from urban road target images and perform image enhancement processing on them; Randomly select a portion of the target images of urban roads and perform image noise addition and image blurring on them; The LabelImg annotation tool is used to annotate the original urban road target images, the low-light images after image enhancement, and some images after image noise addition and blurring to construct an urban road target detection dataset.

4. The urban road target detection and tracking method based on the Deepsort algorithm according to claim 1, characterized in that, The urban road target detection model is an improved YOLOv4 model, with specific improvements including: Replace the standard convolutions in the traditional YOLOv4 model with depthwise separable convolutions; In addition to the three output layers of the traditional YOLOv4 model, an output layer with a grid of 104×104 is added; Introduce some Inceptionv3 structures into the Head part of the traditional YOLOv4 model.

5. The urban road target detection and tracking method based on the Deepsort algorithm according to claim 1, characterized in that, The target detection and tracking model for urban roads is the DeepSort model.

Images