Cross-camera vehicle tracking system and method based on license plate historical sequence matching

Abstract

The invention relates to a cross-camera vehicle tracking system and method, in particular to a cross-camera vehicle tracking system and method based on license plate historical sequence matching. The invention aims to solve the problems that license plate matching among different cameras is difficult in a large traffic network, and particularly, license plate matching and vehicle tracking cannot be quickly and accurately carried out under the condition that coverage areas of a high-flow road section or a plurality of cameras are overlapped. According to the invention, license plate recognition is carried out on the video stream captured by each camera, all license plate recognition results of vehicles tracked by the same camera are used as a sequence, and Hungary matching is carried out on the historical license plate recognition sequences of the plurality of cameras; and if yes, the two vehicles are considered to be the same tracking target. And if the matching is successful and the vehicles are confirmed to be the same vehicle, the system distributes the same unique ID to the vehicles, so that continuous tracking across cameras is realized. The invention belongs to the technical field of computer vision.

Description

Technical Field

[0001] The invention relates to a cross-camera vehicle tracking system and method, and belongs to the technical field of computer vision. Background Art

[0002] With the acceleration of urbanization and the continuous growth of the number of motor vehicles, urban traffic management is facing increasing challenges. In order to ease traffic pressure, reduce traffic violations and improve traffic safety, Intelligent Transportation System (ITS) is widely used in urban traffic management. ITS systems rely on a large number of cameras deployed on urban roads to monitor and analyze traffic flow, vehicle information, traffic accidents, etc. in real time. However, how to accurately track the movement of the same vehicle between different cameras in a wide area covered by multiple cameras is a key and challenging problem in intelligent transportation systems.

[0003] Traditional vehicle tracking technologies mainly rely on the appearance features of vehicles, such as color, shape, logo, body size, etc. These technologies usually use the target detection and tracking algorithms in computer vision to analyze the vehicle images in the video stream, extract features and match them to achieve vehicle tracking. However, the appearance of vehicles has a high similarity, and the matching based on appearance features is not accurate, especially the appearance features are easily affected by many factors, such as lighting changes, vehicle occlusion, camera angle differences, etc., all of which will lead to a decrease in tracking accuracy. In addition, in actual applications, vehicles may appear at large time and space intervals in the perspectives of different cameras. In this case, the matching algorithm based on appearance features is prone to mismatching or loss of tracking.

[0004] In order to overcome these problems, tracking methods based on vehicle license plate recognition have gradually attracted attention in recent years. As the only identification of a vehicle, the license plate has significant stability and uniqueness. Therefore, cross-camera tracking of vehicles through license plate recognition can, to a certain extent, avoid the problems caused by appearance changes or occlusions in traditional methods. License plate recognition technology (Automatic License Plate Recognition, referred to as ALPR) uses image processing and deep learning technology to detect and identify the vehicle's license plate number from video frames. With the improvement of computing power and algorithm accuracy, modern ALPR systems have been able to achieve a high recognition rate in complex environments.

[0005] However, there are also certain challenges in relying on license plate recognition for tracking. This is because it is impossible for the camera to always clearly capture the license plate information. In the case of motion blur, license plate angle error, light reflection, etc., the license plate recognition algorithm will continue to give incorrect recognition results. In addition, in large-scale transportation networks, it is also difficult to match license plates between different cameras, especially in high-traffic sections or when multiple cameras cover overlapping areas. How to quickly and accurately match license plates and track vehicles is an urgent problem that needs to be solved.

[0006] Based on the above background, the present invention proposes an innovative cross-camera license plate recognition and vehicle tracking system and method. The system performs license plate recognition on the video stream captured by each camera, takes all license plate recognition results of the vehicle tracked by the same camera as a sequence, and performs Hungarian matching on the license plate recognition history sequences of multiple cameras. If the historical license plate recognition sequences of two vehicles from different cameras can be matched, the two vehicles are considered to be the same tracking target. If the match is successful and it is confirmed to be the same vehicle, the system will assign it the same unique ID, thereby achieving continuous tracking across cameras. This method not only overcomes the limitations of traditional vehicle appearance feature matching, but also can significantly improve the accuracy and reliability of cross-camera vehicle tracking. Summary of the invention

[0007] The present invention aims to solve the problem that license plate matching between different cameras is difficult in large-scale transportation networks, especially in high-traffic sections or when the coverage areas of multiple cameras overlap, and license plate matching and vehicle tracking cannot be performed quickly and accurately. A cross-camera vehicle tracking system and method based on license plate historical sequence matching is proposed.

[0008] The technical solution adopted by the present invention to solve the above problems is: a cross-camera vehicle tracking system based on license plate historical sequence matching of the present invention comprises a license plate recognition module, a camera synchronization module, a license plate matching module, an ID allocation module and a data storage and management module;

[0009] License plate recognition module: used to extract vehicle images from video streams of multiple cameras and recognize the vehicle's license plate number;

[0010] Camera synchronization module: used to synchronize the video streams of multiple cameras and mark the time and location information of the vehicle. ;

[0011] License plate matching module: matches the license plate currently recognized by the camera with the previously recognized license plate in the database to determine whether they are the same vehicle;

[0012] ID allocation module: When the matching result shows that the license plates are the same, a unique ID that is consistent with the license plate number is allocated to the vehicle; otherwise, a new ID is allocated;

[0013] Data storage and management module: records the license plate, time, location, and ID information recognized by all cameras for subsequent vehicle tracking and analysis.

[0014] The steps of a cross-camera vehicle tracking method based on license plate historical sequence matching described in the present invention include:

[0015] Step 1: Obtain the video to be detected through multiple cameras, record the internal parameters of the camera, perform target detection on the video to be detected, and assign IDs and track the trajectories of the detected vehicles in the cameras;

[0016] Step 2: Capture the video stream of vehicles entering the parking lot through the entrance camera, and extract the license plate number using the license plate recognition module;

[0017] Step 3: Transmit the license plate recognition results to the central processing system and synchronize the data to ensure the consistency of time and location;

[0018] Step 4: The vehicle ID allocation central processing system receives the identified license plate number and queries the database;

[0019] Step 5: According to the result of the database, a result based on the license plate ID matching is regenerated and written into the file frame by frame;

[0020] Step 6: Visually track the vehicle based on its location, ID, and license plate information. Integrate the multi-camera vehicle information into a map under the same coordinate system through a unified coordinate system, draw the movement trajectory, and draw other parking spaces, lanes and other environmental information to finally generate a video.

[0021] Furthermore, step 2 specifically includes: using the trained convolutional neural network to perform vehicle detection and license plate recognition, and writing the position, ID, and vehicle of the vehicle recognized in each frame of each scene into a corresponding file.

[0022] Furthermore, in step 3, the timestamp and geographic location information are used to synchronize the data, and finally the same frame of each scene is integrated into the same file.

[0023] Furthermore, in step 4, a dictionary is created, the keys of which are ID and license plate number, and the corresponding values ​​are a list consisting of scene number and ID number under the scene. Each license plate number corresponds to an ID. If the license plate of the vehicle in this scene is consistent with the license plate number of a registered vehicle, the scene number and ID number under the scene corresponding to the vehicle at this time are added to the list of values ​​corresponding to the key corresponding to the license plate number. If the scene number and ID number under the scene are detected in subsequent frames, the ID in the key corresponding to the list where the scene number and ID number under the scene are assigned is used by default. If the vehicle is not registered, a unique ID is assigned to it, a new key ID and license plate number are created in the dictionary, an empty list is assigned to its value, and it is recorded in the database.

[0024] The beneficial effects of the present invention are as follows: the present invention provides a cross-camera license plate recognition and vehicle tracking system and method, the system performs license plate recognition on the video streams of multiple cameras, and if a license plate is recognized, it is matched with the license plate detected by the previous camera; if the matching result shows that the license plate numbers are the same, a unique ID consistent with the license plate number is assigned to the vehicle; through this method, continuous tracking of vehicles across cameras can be achieved, thereby improving the efficiency of traffic monitoring and management; the present invention not only overcomes the limitations of traditional vehicle appearance feature matching, but also can significantly improve the accuracy and reliability of cross-camera vehicle tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is the algorithm flow of the present invention;

[0026] Figure 2 This is a diagram of the tracking effect of two cameras. DETAILED DESCRIPTION

[0027] Specific implementation method 1: Figure 1 As shown, a cross-camera vehicle tracking system based on license plate historical sequence matching includes a license plate recognition module, a camera synchronization module, a license plate matching module, an ID allocation module and a data storage and management module;

[0028] License plate recognition module: used to extract vehicle images from video streams of multiple cameras and recognize the vehicle's license plate number;

[0029] Camera synchronization module: used to synchronize the video streams of multiple cameras and mark the time and location information of the vehicle. ;

[0030] License plate matching module: matches the license plate currently recognized by the camera with the previously recognized license plate in the database to determine whether they are the same vehicle;

[0031] ID allocation module: When the matching result shows that the license plates are the same, a unique ID that is consistent with the license plate number is allocated to the vehicle; otherwise, a new ID is allocated;

[0032] Data storage and management module: records the license plate, time, location, and ID information recognized by all cameras for subsequent vehicle tracking and analysis.

[0033] The system process is as follows:

[0034] Video capture: Multiple cameras capture traffic flow videos and transmit them to the central processing system in real time. N different video streams are denoted as S i , i=1,…,N.

[0035] License plate recognition: The license plate recognition module extracts vehicle images from the video and identifies the license plate number. When a vehicle enters the i-th video stream, a temporary ID is assigned to the vehicle: x i , the license plate detected in the video stream is recorded as M indicates that there are M license plates detected before time t.

[0036] License Plate Matching:

[0037] (Direct Matching) The license plate matching module compares the currently recognized license plate with the license plate previously recorded in the database to determine whether they are the same vehicle. At the same time, if a vehicle is also detected in another video stream j and a temporary ID is assigned to the vehicle: x j , for the new license plate detected and Compare all the license plates in the table. If there are identical license plate numbers, let x j ←x i , if With any The license plates in are all different, so construct Store license plates for comparison with other vehicles in other video streams.

[0038] (Weighted matching) The license plate matching module compares the currently recognized license plate with the license plate previously recorded in the database to determine whether they are the same vehicle. At the same time, if a vehicle is also detected in another video stream j and a temporary ID is assigned to the vehicle: x j , for the detected new license plate, construct Store the license plate until the vehicle disappears from the video stream j. All license plates and Compare all license plates in the table and set a weight of 0 at the beginning. If the qth license plate matches the license plate in other video streams successfully, then Final Order Equal to the license plate with the highest weight, x jThe license plate ID of the successfully matched license plate with the highest weight is also assigned.

[0039] ID allocation: If the license plate matching result shows that the license plates are the same, the ID allocation module will assign a unique ID to the vehicle that matches its license plate number (x j ←x i ); if they do not match, a new ID is generated (x j ).

[0040] Information storage and management: All identified license plate numbers, timestamps, locations, and assigned IDs are stored for subsequent query and analysis.

[0041] Specific implementation method 2: Figure 1 As shown, a cross-camera vehicle tracking method based on license plate history sequence matching includes the following specific steps:

[0042] Step 1: Obtain the video to be detected through multiple cameras, record the internal parameters of the camera, perform target detection on the video to be detected, and assign IDs and track the trajectories of the detected vehicles in the cameras;

[0043] Step 2: Capture the video stream of vehicles entering the parking lot through the entrance camera, and use the license plate recognition module to extract the license plate number; use the trained convolutional neural network to perform vehicle detection and license plate recognition, and write the position, ID, and vehicle of each frame of each scene into the corresponding file;

[0044] Step 3: Transmit the license plate recognition results to the central processing system and synchronize the data to ensure the consistency of time and location; synchronize the data using timestamp and geographic location information, and finally integrate the same frame of each scene into the same file;

[0045] Step 4. Vehicle ID assignment The central processing system receives the identified license plate number and queries the database; creates a dictionary whose keys are ID and license plate number, and whose corresponding values ​​are a list consisting of scene number and ID number under the scene. Each license plate number corresponds to an ID. If the license plate of the vehicle in this scene is consistent with the license plate number of a registered vehicle, then the scene number and ID number under the scene corresponding to the vehicle at this time are added to the list of values ​​corresponding to the key corresponding to the license plate number; if the scene number and ID number under the scene are detected in subsequent frames, the ID in the key corresponding to the list where the scene number and ID number under the scene are assigned is used by default; if the vehicle is not registered, assign it a unique ID, create a new key ID and license plate number in the dictionary, assign an empty list to its value, and record it in the database;

[0046] Step 5: According to the result of the database, a result based on the license plate ID matching is regenerated and written into the file frame by frame;

[0047] Step 6: Visually track the vehicle based on its location, ID, and license plate information. Integrate the multi-camera vehicle information into a map under the same coordinate system through a unified coordinate system, draw the movement trajectory, and draw other parking spaces, lanes and other environmental information to finally generate a video.

[0048] How it works

[0049] Since the license plate is detected based on video, the license plate detected in the video is wrong most of the time due to changes in lighting, vehicle occlusion, camera angle differences, etc., and it is difficult to determine which license plate is correct. The present invention is based on a basic mathematical assumption: and The probability of a license plate detection being mistakenly detected as the same wrong license plate is extremely small. Therefore, if two license plates are detected as the same in two video streams, the two vehicles are considered to be the same vehicle, and both license plate detections are correct. For direct matching, the probability of the same wrong detection is considered negligible; for weighted matching, the matching rate is slightly slower, but even if there are the same wrong detections, the probability of occurrence will be lower than the same correct detection. The same wrong detections can be excluded through weights, providing a higher accuracy rate. In actual situations, license plate matching can also be used in conjunction with spatial position matching to improve accuracy.

[0050] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A cross-camera vehicle tracking system based on license plate historical sequence matching, characterized in that: It includes license plate recognition module, camera synchronization module, license plate matching module, ID allocation module and data storage and management module; License plate recognition module: used to extract vehicle images from video streams of multiple cameras and recognize the vehicle's license plate number; Camera synchronization module: used to synchronize the video streams of multiple cameras and mark the time and location information of the vehicle. ; License plate matching module: matches the license plate currently recognized by the camera with the previously recognized license plate in the database to determine whether they are the same vehicle; ID allocation module: When the matching result shows that the license plate is the same, a unique ID that is consistent with the license plate number is allocated to the vehicle; Otherwise assign a new ID; Data storage and management module: records the license plate, time, location, and ID information recognized by all cameras for subsequent vehicle tracking and analysis.

2. A cross-camera vehicle tracking method based on license plate history sequence matching, characterized in that: The specific steps include: Step 1: Obtain the video to be detected through multiple cameras, record the internal parameters of the camera, perform target detection on the video to be detected, and assign IDs and track the trajectories of the detected vehicles in the cameras; Step 2: Capture the video stream of vehicles entering the parking lot through the entrance camera, and extract the license plate number using the license plate recognition module; Step 3: Transmit the license plate recognition results to the central processing system and synchronize the data to ensure the consistency of time and location; Step 4: The vehicle ID allocation central processing system receives the identified license plate number and queries the database; Step 5: According to the result of the database, a result based on the license plate ID matching is regenerated and written into the file frame by frame; Step 6: Visually track the vehicle based on its location, ID, and license plate information. Integrate the multi-camera vehicle information into a map under the same coordinate system through a unified coordinate system, draw the movement trajectory, and draw other parking spaces, lanes and other environmental information to finally generate a video.

3. The cross-camera vehicle tracking method based on license plate historical sequence matching according to claim 2 is characterized in that: Step 2 specifically includes: using the trained convolutional neural network to perform vehicle detection and license plate recognition, and writing the position, ID, and vehicle of the vehicle recognized in each frame of each scene into the corresponding file.

4. The cross-camera vehicle tracking method based on license plate historical sequence matching according to claim 2 is characterized in that: In step 3, the data is synchronized using timestamp and geographic location information, and finally the same frame of each scene is integrated into the same file.

5. The cross-camera vehicle tracking method based on license plate historical sequence matching according to claim 2 is characterized in that: In step 4, create a dictionary whose keys are ID and license plate number, and whose corresponding values ​​are a list consisting of scene number and ID number under the scene. Each license plate number corresponds to an ID. If the license plate number of the vehicle in this scene is the same as the license plate number of the registered vehicle, then add the scene number and ID number corresponding to the vehicle at this time to the list of values ​​corresponding to the key corresponding to the license plate number. If the scene number and the ID number under the scene are detected in the subsequent frames, you will be assigned the ID in the key corresponding to the list where the scene number and the ID number under the scene are located by default; If the vehicle is not registered, assign it a unique ID, create new keys ID and license plate number in the dictionary, assign an empty list as its value, and record it in the database.

Images