A multi-stage vehicle re-identification method based on spatiotemporal criteria and hybrid network architecture

Abstract

The present invention discloses a multi-stage vehicle re-identification method based on spatiotemporal criteria and a hybrid network architecture. A new sampling strategy based on mixup of multi-view images is proposed for the re-identification dataset to enhance the robustness of the model to vehicle perspective changes. In order to address the problem that previous deep learning algorithms have difficulty distinguishing between vehicle categories with similar appearances, the spatiotemporal criteria are designed to reduce false detections and improve the accuracy of model retrieval; through the hybrid network architecture, fine-grained global information is extracted to overcome the shortcomings of the CNN network structure in losing detailed information and enhance the robustness of the model's feature extraction. While ensuring the retrieval speed, the present invention greatly improves the accuracy of the model. The model can operate stably and efficiently in actual urban traffic scenarios and has broad application prospects.

Description

Technical Field

[0001] The present invention belongs to the technical field of pattern recognition, and in particular relates to a multi-stage vehicle re-identification method. Background Art

[0002] Re-identification algorithms, also known as re-identification or cross-camera tracking algorithms, aim to retrieve the same given target in videos or images captured by multiple cameras. Currently, the main research targets for re-identification algorithms, both domestically and internationally, are pedestrians and vehicles. Pedestrians are the primary objects in road surveillance videos, so pedestrian re-identification algorithms have been widely studied. However, vehicles, another major object in road traffic, still face significant research challenges in the field of re-identification algorithms. Vehicle re-identification involves retrieving target vehicles of interest based on images, videos, or even text descriptions, in scenarios where multiple cameras capture non-overlapping content within a certain area.

[0003] With the development of road traffic, the increasing number of vehicles on the road has complicated traffic conditions and posed significant challenges to traffic management. Due to the urgent need for public safety and the increasing use of surveillance cameras, research on vehicle re-identification (VRE) has become of great practical significance. As a key complement to license plate recognition (LPR), VRE algorithms can retrieve and track the same target vehicle across multiple cameras, even when the camera cannot capture a clear license plate, providing retrieval results with temporal and spatial information.

[0004] In recent years, with the development of deep learning algorithms, vehicle re-identification (RID) algorithms have made rapid progress. However, their application has not been widely adopted. The main reasons are insufficient model accuracy and difficulty in effective application in urban areas, especially on busy roads with a large number of vehicles and a wide variety of types. Common categories such as taxis and buses have low intra-class differentiation, high mobility, and frequent appearances across cameras in different areas, constituting a large portion of the search dataset. However, buses and taxis present difficult examples in RID because their appearance is similar within the same class, making them difficult to distinguish. Only detailed information such as annual inspection labels can be utilized, but this is often difficult to effectively utilize due to camera angles. While some studies have utilized fine-grained networks to enhance the utilization of local information, these methods overlook a crucial clue in the retrieval task: the spatiotemporal criterion of vehicle movement under surveillance. Furthermore, CNN-based network structures can only process a portion of local information at a time, and downsampling and pooling operations can lose detailed information. Summary of the Invention

[0005] In order to overcome the shortcomings of the existing technology, the present invention provides a multi-level vehicle re-identification method based on spatiotemporal criteria and a hybrid network architecture. A new sampling strategy based on mixup of multi-view images is proposed for the re-identification dataset to enhance the robustness of the model to vehicle perspective changes. In order to address the problem that previous deep learning algorithms have difficulty distinguishing between vehicle categories with similar appearances, the spatiotemporal criteria are designed to reduce false detections and improve the accuracy of model retrieval; through the hybrid network architecture, fine-grained global information is extracted to overcome the shortcomings of the CNN network structure in losing detailed information and enhance the robustness of the model's feature extraction. While ensuring the retrieval speed, the present invention greatly improves the accuracy of the model. The model can operate stably and efficiently in actual urban traffic scenarios and has broad application prospects.

[0006] The technical solution adopted by the present invention to solve the technical problem includes the following steps:

[0007] Step 1: Read the traffic video stream containing moving vehicles to obtain video frame information, and sample the video frames to obtain key frames;

[0008] The key frames are images of the same vehicle from three angles: the front, the side, and the rear. The vehicle's passing time is determined based on the vehicle's speed at the same camera's perspective and the known distance between the cameras, and the key frames are selected based on the vehicle's passing time.

[0009] Step 2: Extract vehicle information through the object recognition detector;

[0010] Detect the key frames, extract the vehicle target and location information, crop the vehicle image in the key frames according to the location information and save the vehicle image as the query set;

[0011] Step 3: Use the public vehicle re-ID dataset as the training set to train the hybrid re-ID network based on the transformer and CNN structure;

[0012] The hybrid network uses CNN as the baseline network DenseNet121 and introduces a transformer structure in the middle layer to enhance the model's global information acquisition ability. The transformer structure is inserted into three different levels of middle modules, namely the 5th module, the 8th module, and the 11th module.

[0013] First, the vehicle ID information in the training set is obtained. Then, images of each vehicle at different angles are obtained based on the vehicle ID information. The images of the vehicle at different angles are combined into a mixup image pool. Then, all images in the same mixup image pool are mixed and fused. The fused images are then fed into the hybrid re-ID network for training.

[0014] Step 4: Use the trained hybrid re-ID network to extract the features of the target vehicle and all vehicles in the query set. Use Euclidean distance to calculate the similarity between the features of the target vehicle and all vehicles in the query set. Then sort the vehicle images in the query set according to the similarity.

[0015] Step 5: Use the spatiotemporal criteria to re-rank the ranking results of step 4, and use the re-ranked results as the final retrieval results;

[0016] Based on the camera annotations and time information of the video frames in the vehicle re-ID dataset, we use spatiotemporal criteria to reorder the images and eliminate false positives due to similar appearance features. The details are as follows:

[0017] (1) According to the camera distance s and the maximum road speed v max Calculate the minimum vehicle travel time t min :

[0018] t min =s / v max

[0019] The time interval between the next camera continuously shooting the same vehicle should be greater than t min If the camera information of the vehicle to be queried in the retrieval list is the next camera of the target vehicle, further judgment is required; if the time marked by the timestamp of the vehicle to be queried is t gallery The time t marked by the target vehicle timestamp target , the calculated elapsed time t gallery -t target <t min , then the vehicle passing the camera at this time is not the target vehicle, that is, the search is wrong, and the vehicle to be queried is placed at the end;

[0020] (2) If there are multiple images of the target vehicle, all the cameras that captured these images will form the vehicle's travel path. Based on the timestamp information and camera information of the target vehicle image, the complete vehicle travel path is obtained. At this time, by comparing the timestamp information and camera information of the vehicles in the query set, the vehicles that do not meet the conditions are excluded. That is, under the same timestamp conditions, if the camera information of the vehicle in the query set is different from that of the target vehicle, that is, the shooting location is different, the retrieval is wrong, and the vehicle in the query set is sorted to the end.

[0021] (3) According to the vehicle's driving path, if in a certain time period t x , a vehicle travels between two cameras and has the following timestamp t q Vehicles to be checked:

[0022] t max=s / v min

[0023] t x +t min <t q <t x +t max

[0024] Among them, t min Indicates the minimum time difference between a vehicle passing two cameras, t max Indicates the maximum time difference between a vehicle passing two cameras;

[0025] If the camera number information is not the same as the information of these two cameras, the retrieval is wrong and the vehicle to be queried is placed at the end.

[0026] Preferably, the key frame selection method is as follows:

[0027] Based on the straight-line distance L between the two surveillance cameras and the vehicle's speed V, calculate the time t the vehicle takes to pass a camera. drive :

[0028]

[0029] Calculate t drive After that, three frames per second, namely the first frame, the middle frame and the last frame, are taken as key frames.

[0030] Preferably, the vehicle information is extracted by the target recognition detector as follows:

[0031] Yolov3 is used as the target detector. Yolov3 is a single-stage target detection algorithm. Yolov3 is divided into three parts: Darknet53, FPN, and Head. Darknet53 is used as the backbone to extract image features. In the backbone stage, three feature layers are extracted, namely 8, 16, and 32 times downsampling features. The Darknet53 network uses 1x1 convolution. The convolutional network adopts the CNN+BN+LeakyReLU structure.

[0032] The purpose of the FPN structure is to combine feature information at different scales, obtain three effective feature layers from the backbone, and then further extract features and perform feature fusion;

[0033] Head is the classification layer of the algorithm, which uses the three-layer features passed through FPN to predict the coordinate position, length, width and category of the target box;

[0034] The prediction results of the Head layer are used to obtain the vehicle's position information, thereby cropping the vehicle target from the video frame to obtain a single vehicle image.

[0035] Preferably, the vehicle re-identification dataset is VehicleID.

[0036] The beneficial effects of the present invention are as follows:

[0037] The present invention optimizes and improves the benchmark of the re-identification network and proposes a hybrid network architecture that combines the transformer and CNN network structures to give full play to the advantages of each and obtain more accurate vehicle feature information. During training, a sampling strategy based on mixup of multi-perspective images is introduced, so that the model can fully learn the perspective information of the vehicle, thereby greatly improving the matching success rate of the same vehicle under different perspectives. Finally, the spatiotemporal criteria are used to re-sort the retrieval results to eliminate the influence of vehicles with similar appearances but different IDs. While ensuring the retrieval speed, the present invention greatly improves the accuracy of the model. The model can operate stably and efficiently in actual urban traffic scenarios and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 2 is a schematic diagram of the overall process of the vehicle re-identification method of the present invention.

[0039] Figure 2 It is a schematic diagram of the hybrid network structure of the present invention.

[0040] Figure 3 Schematic diagram of the transformer module of the present invention.

[0041] Figure 4 It is a schematic diagram of the convolution module (ConvBlock) of the present invention. DETAILED DESCRIPTION

[0042] The present invention will be further described below with reference to the accompanying drawings and examples.

[0043] To overcome the shortcomings of existing technologies, this paper provides a multi-stage vehicle re-identification method based on spatiotemporal criteria and a hybrid network architecture. It also proposes a new sampling strategy for re-identification datasets, using a mixup of multi-view images, to enhance the model's robustness to vehicle viewpoint changes. This approach addresses the difficulty of previous deep learning algorithms in distinguishing between similar-looking vehicles. By designing a spatiotemporal criterion, the method reduces false positives and improves model retrieval accuracy. Furthermore, the hybrid network architecture extracts fine-grained global information, overcoming the loss of detailed information inherent in CNN architectures and enhancing the robustness of the model's feature extraction.

[0044] like Figure 1As shown, the technical solution adopted by the present invention to solve its technical problem includes the following steps:

[0045] Step 1: Use a real traffic road camera to shoot a traffic video stream containing moving vehicles, read the video stream to obtain video frame information, and sample the video frames to obtain key video frames.

[0046] Traffic video streams often contain large amounts of vehicle data, but this data is often repetitive. Not all frames require feature extraction from the re-ID network. By selecting keyframes, we can avoid repeated feature extraction for the same vehicle, improve retrieval efficiency, and reduce the time required to retrieve the target vehicle. Keyframes are images of the same vehicle from three angles: the front, side, and rear. The vehicle's speed from the camera's perspective and the known distance between the cameras are used to determine the vehicle's passing time. Keyframes are selected based on the vehicle's passing time, thus filtering out most of the repetitive data.

[0047] Step 2: Extract vehicle information using an object recognition detector. Detect the sampled key video, extract the vehicle target and location information, crop the vehicle image from the video frame based on the location information, and save the vehicle image as the query set.

[0048] Step 3, such as Figure 2 As shown, the hybrid network is trained. The hybrid neural network in this invention is trained using the public vehicle re-identification dataset, VehicleID. In the re-identification dataset, images captured by different cameras are annotated with the camera ID information. Since the re-identification task aims to perform cross-camera retrieval in non-overlapping areas, different cameras represent different shooting angles. Previous studies often ignore this angle information in the dataset, and instead construct a minibatch as the model input by simply randomly shuffling and then sampling.

[0049] To fully utilize the perspective information in the re-ID dataset, this paper designs a mixup sampling strategy based on multi-view images. First, the vehicle ID information in the training set is obtained. Then, images corresponding to the vehicle IDs at different angles are acquired. These images form a mixup image pool. All images in the same pool are then subjected to a mixup fusion process. This fusion of images from different perspectives enhances the model's robustness to vehicle perspective shifts while mitigating overfitting.

[0050] In step 4, the target vehicle is input and the hybrid re-ID network is used for feature extraction. The features of the target vehicle and all vehicles in the query set are extracted and stored respectively. Then, through feature matching, the Euclidean distance is calculated and all images with the same ID are retrieved based on the similarity.

[0051] Using a JSON file format, store the target vehicle's features and the features of all vehicles to be compared in two different JSON files. Use Euclidean distance to calculate the distance between features, and use this distance to generate search results. A smaller Euclidean distance indicates a closer distance between features, and a more similar vehicle to the target vehicle.

[0052] Step 5: Use the spatiotemporal criteria to re-rank the retrieval results to improve the accuracy of the retrieval results and overcome the problem of high false positive rate of re-identification of similar vehicles.

[0053] According to the spatiotemporal criteria formulated by the present invention, the sorting results obtained in step 4 are re-sorted, and the re-sorted results are used as the final retrieval results. Specific embodiment:

[0055] To address the existing problems of vehicle re-identification algorithms, this paper proposes a multi-stage vehicle re-identification method based on spatiotemporal criteria and a hybrid network architecture. The method includes the following steps:

[0056] Step 1: Obtain video keyframes. These keyframes are used to construct a vehicle re-ID query dataset. In re-ID tasks, the query dataset, also known as the Gallery set, is the collection of all vehicle images to be retrieved.

[0057] OpenCV is used to read the video stream and obtain real-time video data. In the actual retrieval task, the source of the video stream is the video information stored by the camera that may be present in the target vehicle. In order to perform the retrieval more efficiently, the present invention selects the key frames of the video to construct the data set to be searched. According to the distance L of the high-speed monitoring camera and the driving speed V of the vehicle, the time t when the vehicle passes a camera is calculated. drive .

[0058]

[0059] Calculate t drive After that, three frames (the first frame, the middle frame, and the last frame) are taken as key frames every second. Generally, the video has a frame rate of 25 frames per second, so the first, 13th, and 25th frames are selected as key frames.

[0060] Step 2: Extract vehicle information using an object recognition detector. Use the YOLOv3 algorithm as the object detector to extract vehicle targets and their location information. The YOLOv3 algorithm has good real-time performance and accuracy, enabling effective detection in live video streams.

[0061] YOLOV3 is a single-stage target detection algorithm. Yolov3 can be divided into three parts: Darknet53, FPN and Head. Darknet53 is used as the backbone to extract image features. This new network structure combines the residual block characteristics of Resnet to ensure super-strong expression of features while avoiding the gradient problem caused by the network being too deep. In the Backbone stage, a total of three feature layers were extracted, which are 8, 16, and 32 times downsampling features. The Darknet53 network part uses a large number of 1x1 convolutions, which are mainly used for channel expansion and reduction. The overall convolutional network adopts the CNN+BN+LeakyReLU structure. Figure 4 shown.

[0062] The purpose of the FPN structure is to combine feature information at different scales. After obtaining three valid feature layers from the backbone, further features are extracted and fused.

[0063] Head is the classification layer of the algorithm, which uses the three-layer features passed through FPN to predict the coordinate position, length, width and category of the target box.

[0064] This method uses the prediction results of the Head layer to obtain vehicle location information, thereby cropping the vehicle target from the video frame to obtain a single vehicle image. All key video frames are subjected to the vehicle target extraction operation using the YOLOV3 network. All acquired vehicle target images constitute the query set. The vehicles in the query set are then searched to see if they contain the target vehicle.

[0065] Step 3: Train the hybrid network. Figure 3 As shown, a hybrid network based on transformer and CNN structure is trained. In this stage, the hybrid neural network of the present invention is trained using the public vehicle re-identification dataset VehicleID.

[0066] The VehicleID dataset, also known as PKU VehicleID, was released by Peking University and contains images of cars in a small Chinese city. The data was captured by multiple real-world surveillance cameras distributed throughout the city during the day. The dataset contains 221,763 images of 26,267 vehicles. Each image is annotated with an ID corresponding to its real-world identity. In addition, the dataset includes 90,196 images of 10,319 vehicles, manually annotated with vehicle model information (such as "MINI-Cooper," "Audi A6L," and "BWM 1 Series").

[0067] During training, a novel sampling strategy based on multi-view image mixup, proposed in this paper, is used. Vehicle ID information is first obtained from the training set. Images corresponding to the vehicle IDs at different angles are then acquired. These images form a pool of mixed images, and all images in the pool are then subjected to a mixup fusion process. This fusion of images from different angles enhances feature robustness while mitigating model overfitting. The model's memory of noisy samples is reduced, thereby minimizing the impact of multiple models of noisy samples.

[0068] Mixup obtains new training samples by linear interpolation of samples. Fusion calculation formula: x i and x j They represent different angle images of the same ID vehicle, λ is the fusion parameter, which is set to 0.3 in the present invention. is the new sampled image after fusion.

[0069]

[0070] The Transformer is essentially an encoder-decoder structure based on the self-attention mechanism. Both the encoder and decoder contain six blocks. The encoder block includes one multi-head attention block, while the decoder block contains two multi-head attention blocks (one of which uses a masked block). Above the multi-head attention block is an Add&Norm layer. Add represents a residual connection, which is used to prevent network degradation, and Norm represents Layer Normalization, which is used to normalize the activation values ​​of each layer. In recent years, the Transformer model has become one of the main highlights of the progress of deep learning and deep neural networks, and has been successfully applied from natural language processing to computer vision.

[0071] The self-attention mechanism function of Transformer is a function that maps a query and some key and value pairs into an output, where all queries (q), keys (k), values ​​(v) and outputs are vectors.

[0072] q: query vector, k: vector representing the correlation between the queried information and other information, v: vector representing the queried information, and i represents the i-th self-attention head.

[0073] q i =W q a i

[0074] k i =W k a i

[0075] v i =W v a i

[0076]

[0077] In addition to image embeddings, the Transformer architecture also requires positional embeddings to represent the locations of individual image patches within the image. Because the Transformer doesn't employ an RNN structure and instead relies on global information, it can't utilize image positional information, which is crucial for accurate recognition. Therefore, positional embeddings are used in the Transformer to store the relative or absolute position of words within the sequence. Therefore, the input representation X of an image in the Transformer is the sum of the word embeddings and the positional embeddings (positional encoding).

[0078] In theory, transformers can achieve better model performance than CNNs. However, because the transformer's own structure consumes a lot of computation, it leads to huge computational losses when calculating global attention. Especially in shallow networks, the larger the feature map, the higher the computational complexity. Therefore, this paper proposes to insert the transformer into the CNN backbone network to form a new hybrid neural network architecture.

[0079] Step 4: Use the hybrid re-ID network to extract features and obtain a preliminary ranking. The target vehicle and the vehicle images in the query set are fed into the hybrid network to extract features. The similarity between the features is calculated using Euclidean distance, and the vehicle images in the query set are ranked based on this similarity.

[0080] The Euclidean distance is calculated as follows:

[0081]

[0082] Step 5: Re-rank the search results using spatiotemporal criteria. This paper proposes specific spatiotemporal criteria to further refine the sorting of the initial search results. Based on the camera annotations and time information of the video frames in the re-identification dataset, the spatiotemporal criteria are used to re-rank the search results, eliminating false positives due to similar appearances.

[0083] Space-time principles:

[0084] (1) According to the camera distance s and the maximum road speed v max Calculate vehicle travel time t min ,

[0085] t min =s / v max

[0086] The time interval between the next camera continuously shooting the same vehicle should be greater than t min If the camera information of the vehicle to be queried in the search list is the next camera of the target vehicle, further judgment is required. If the time marked by the timestamp of the vehicle to be queried is t gallery The time t marked by the target vehicle timestamp target , the calculated elapsed time t gallery -t target <t min , then the vehicle passing the camera at this time is not the target vehicle, that is, the retrieval is wrong, and the retrieval result is ranked last.

[0087] (2) If there are multiple images of the target vehicle, then the cameras that all the images pass through will constitute the vehicle's travel path. Based on the timestamp information and camera information of the target vehicle image, the complete vehicle travel path is obtained. At this time, the unqualified vehicles in the query set can be excluded by comparing the timestamp information and camera information of the vehicles in the query set. That is, under the same timestamp conditions, if the camera information of the query set vehicle and the target vehicle is different, that is, the shooting location is different, the search is wrong, and the search result is sorted to the end.

[0088] (3) According to the vehicle's driving path, if in a certain time period t x , the vehicle is traveling between two specific cameras, if it has the following timestamp t q Vehicles to be checked:

[0089] t max =s / v min

[0090] t x +t min <t q <t x +t max

[0091] If the camera information is not the same as the two camera information, the search is wrong and the search result is sorted to the end.

Claims

1. A multi-stage vehicle re-identification method based on spatiotemporal criteria and hybrid network architecture, characterized in that: The following steps are involved: Step 1: Read the traffic video stream containing moving vehicles to obtain video frame information, and sample the video frames to obtain key frames; The key frames are images of the same vehicle from three angles: the front, the side, and the rear. The vehicle's passing time is determined based on the vehicle's speed at the same camera's perspective and the known distance between the cameras, and the key frames are selected based on the vehicle's passing time. Step 2: Extract vehicle information through the object recognition detector; Detect the key frames, extract the vehicle target and location information, crop the vehicle image in the key frames according to the location information and save the vehicle image as the query set; Step 3: Use the public vehicle re-ID dataset as the training set to train the hybrid re-ID network based on the transformer and CNN structure; The hybrid network uses CNN as the baseline network DenseNet121 and introduces a transformer structure in the middle layer to enhance the model's global information acquisition ability. The transformer structure is inserted into three different levels of middle modules, namely the 5th module, the 8th module, and the 11th module. First, the vehicle ID information in the training set is obtained. Then, images of each vehicle at different angles are obtained based on the vehicle ID information. The images of the vehicle at different angles are combined into a mixup image pool. Then, all images in the same mixup image pool are mixed and fused. The fused images are then fed into the hybrid re-ID network for training. Step 4: Use the trained hybrid re-ID network to extract the features of the target vehicle and all vehicles in the query set. Use Euclidean distance to calculate the similarity between the features of the target vehicle and all vehicles in the query set. Then sort the vehicle images in the query set according to the similarity. Step 5: Use the spatiotemporal criteria to re-rank the ranking results of step 4, and use the re-ranked results as the final retrieval results; Based on the camera annotations and time information of the video frames in the vehicle re-ID dataset, we use spatiotemporal criteria to reorder the images and eliminate false positives due to similar appearance features. The details are as follows: (1) According to the camera distance s and the maximum road speed v max Calculate the minimum vehicle travel time t min : t min =s / v max The time interval between the next camera continuously shooting the same vehicle should be greater than t min ; If the camera information of the vehicle to be queried in the retrieval list is the next camera of the target vehicle, further judgment is required; if the time marked by the timestamp of the vehicle to be queried is t gallery The time t marked by the target vehicle timestamp target , the calculated elapsed time t gallery -t target <t min , then the vehicle passing the camera at this time is not the target vehicle, that is, the search is wrong, and the vehicle to be queried is placed at the end; (2) If there are multiple images of the target vehicle, then all the cameras that captured these images will constitute the path of the vehicle; Based on the timestamp information and camera information of the target vehicle image, the complete vehicle driving path is obtained. At this time, by comparing the timestamp information and camera information of the vehicles in the query set, the vehicles that do not meet the conditions are excluded. That is, under the same timestamp conditions, if the camera information of the query set vehicle and the target vehicle is different, that is, the shooting location is different, the search is wrong, and the query vehicle is sorted to the end; (3) According to the vehicle's driving path, if in a certain time period t x , a vehicle travels between two cameras and has the following timestamp t q Vehicles to be checked: t max =s / v min t x +t min <t q <t x +t max Among them, t min Indicates the minimum time difference between a vehicle passing two cameras, t max Indicates the maximum time difference between a vehicle passing two cameras; If the camera number information is not the same as the information of these two cameras, the retrieval is wrong and the vehicle to be queried is placed at the end.

2. The multi-stage vehicle re-identification method based on spatiotemporal criteria and hybrid network architecture according to claim 1, characterized in that: The key frame selection method is as follows: Based on the straight-line distance L between the two surveillance cameras and the vehicle's speed V, calculate the time t the vehicle takes to pass a camera. drive : Calculate t drive After that, three frames per second, namely the first frame, the middle frame and the last frame, are taken as key frames.

3. The multi-stage vehicle re-identification method based on spatiotemporal criteria and hybrid network architecture according to claim 1 is characterized in that: The vehicle information is extracted by the target recognition detector as follows: The Yolov3 algorithm is used as the target detector. Yolov3 is a single-stage target detection algorithm. Yolov3 consists of three parts: Darknet53, FPN, and Head. Darknet53 is used as the backbone to extract image features. In the Backbone stage, three feature layers were extracted, which were 8-, 16-, and 32-fold downsampling features. The Darknet53 network used 1x1 convolution. The convolutional network used the CNN+BN+LeakyReLU structure. The purpose of the FPN structure is to combine feature information at different scales, obtain three effective feature layers from the backbone, and then further extract features and perform feature fusion; Head is the classification layer of the algorithm, which uses the three-layer features passed through FPN to predict the coordinate position, length, width and category of the target box; The prediction results of the Head layer are used to obtain the vehicle's position information, thereby cropping the vehicle target from the video frame to obtain a single vehicle image.

4. The multi-stage vehicle re-identification method based on spatiotemporal criteria and hybrid network architecture according to claim 1, characterized in that: The vehicle re-identification dataset is VehicleID.

Images