A cross-camera vehicle tracking system and method based on license plate historical sequence matching

Abstract

The application relates to a cross-camera vehicle tracking system and method based on license plate history sequence matching, and relates to a cross-camera vehicle tracking system and method. In order to solve the problem that license plate matching between different cameras is difficult in a large traffic network, especially in the case that a high-flow road section or multiple camera coverage areas overlap, license plate matching and vehicle tracking cannot be quickly and accurately performed, the application performs license plate recognition on each camera-captured video stream, takes all license plate recognition results of the same camera tracking vehicle as a sequence, performs Hungarian matching on license plate recognition history sequences of multiple cameras, and if the history license plate recognition sequences of two vehicles of different cameras can be matched, the two vehicles are considered to be the same tracking target. If the matching is successful and it is confirmed that the two vehicles are the same, the system will assign the same unique ID to the two vehicles, so as to realize continuous tracking across cameras. The application belongs to the technical field of computer vision.

Description

Technical Field

[0001] The present invention relates to a cross-camera vehicle tracking system and method, and belongs to the technical field of computer vision. Background Art

[0002] With the acceleration of urbanization and the continuous growth of the number of motor vehicles, urban traffic management faces increasing challenges. To alleviate traffic pressure, reduce traffic violations, and improve traffic safety, Intelligent Transportation Systems (ITS) are widely used in urban traffic management. ITS systems rely on a large number of cameras deployed on urban roads to monitor and analyze traffic flow, vehicle information, traffic accidents, etc. in real time. However, how to accurately track the movement of the same vehicle between different cameras in a wide area covered by multiple cameras is a key and challenging problem in intelligent transportation systems.

[0003] Traditional vehicle tracking technologies primarily rely on the appearance features of a vehicle, such as color, shape, logo, and body size. These technologies typically employ object detection and tracking algorithms from computer vision to analyze vehicle images in video streams, extract features, and perform matching to track the vehicle. However, vehicles have a high degree of similarity in appearance, and matching based on appearance features is not accurate. Appearance features are particularly susceptible to various factors, such as lighting changes, vehicle occlusions, and differences in camera angles, all of which can reduce tracking accuracy. Furthermore, in practical applications, vehicles may appear separated in time and space by significant time and space between different camera perspectives. Matching algorithms based on appearance features are prone to mismatches or loss of tracking in such situations.

[0004] To overcome these issues, tracking methods based on vehicle license plate recognition have gained increasing attention in recent years. As a vehicle's unique identifier, the license plate is remarkably stable and unique. Therefore, using license plate recognition to track a vehicle across cameras can, to a certain extent, avoid the issues associated with appearance changes or occlusions in traditional methods. Automatic License Plate Recognition (ALPR) technology utilizes image processing and deep learning techniques to detect and identify vehicle license plates from video frames. With advances in computing power and algorithm accuracy, modern ALPR systems are now able to achieve high recognition rates even in complex environments.

[0005] However, relying on license plate recognition for tracking also presents certain challenges. This is because cameras may not always clearly capture license plate information. In situations such as motion blur, misaligned license plate angles, and light reflections, the license plate recognition algorithm will consistently produce incorrect results. Furthermore, matching license plates between different cameras in large traffic networks is also difficult, especially on high-traffic roads or where multiple cameras have overlapping coverage areas. Therefore, how to quickly and accurately match license plates and track vehicles is a pressing issue that needs to be addressed.

[0006] Based on the above background, the present invention proposes an innovative cross-camera license plate recognition and vehicle tracking system and method. This system performs license plate recognition on the video stream captured by each camera, treating all license plate recognition results for vehicles tracked by the same camera as a sequence. Hungarian matching is then performed on the historical license plate recognition sequences of multiple cameras. If the historical license plate recognition sequences of two vehicles from different cameras match, the two vehicles are considered to be the same tracking target. If the match is successful and the vehicle is confirmed to be the same, the system assigns the same unique ID to them, thus achieving continuous tracking across cameras. This method not only overcomes the limitations of traditional vehicle appearance feature matching but also significantly improves the accuracy and reliability of cross-camera vehicle tracking. Summary of the Invention

[0007] The present invention aims to solve the problem of difficulty in matching license plates between different cameras in large-scale transportation networks, especially in high-traffic sections or when the coverage areas of multiple cameras overlap, which makes it impossible to quickly and accurately perform license plate matching and vehicle tracking. A cross-camera vehicle tracking system and method based on license plate historical sequence matching is proposed.

[0008] The technical solution adopted by the present invention to solve the above problems is as follows: a cross-camera vehicle tracking system based on license plate historical sequence matching includes a license plate recognition module, a camera synchronization module, a license plate matching module, an ID allocation module and a data storage and management module;

[0009] License plate recognition module: used to extract vehicle images from video streams of multiple cameras and identify the vehicle's license plate number;

[0010] Camera synchronization module: used to synchronize the video streams of multiple cameras and mark the time and location information of vehicle appearance.

[0011] License plate matching module: matches the license plate currently recognized by the camera with the previously recognized license plates in the database to determine whether they are the same vehicle;

[0012] ID assignment module: If the matching result shows that the license plates are the same, a unique ID that matches the license plate number will be assigned to the vehicle; otherwise, a new ID will be assigned.

[0013] Data storage and management module: records the license plate, time, location, and ID information of all vehicles recognized by the camera for subsequent vehicle tracking and analysis.

[0014] The steps of the cross-camera vehicle tracking method based on license plate history sequence matching described in the present invention include:

[0015] Step 1: Use multiple cameras to obtain the video to be detected, record the internal parameters of the camera, perform target detection on the video to be detected, and assign an ID and track the trajectory of the detected vehicle in the camera;

[0016] Step 2: Capture the video stream of vehicles entering the parking lot through the entrance camera and use the license plate recognition module to extract the license plate number;

[0017] Step 3: Transmit the license plate recognition results to the central processing system and synchronize the data to ensure consistency in time and location;

[0018] Step 4: The vehicle ID allocation central processing system receives the identified license plate number and queries the database;

[0019] Step 5: Based on the results of the database, regenerate a result based on the license plate ID matching and write it into the file frame by frame;

[0020] Step 6: Visually track the vehicle based on its location, ID, and license plate information. Integrate the multi-camera vehicle information into a map in the same coordinate system through a unified coordinate system, draw the movement trajectory, and draw other parking spaces, lanes and other environmental information to finally generate a video.

[0021] Furthermore, step 2 specifically includes: using the trained convolutional neural network to perform vehicle detection and license plate recognition, and writing the position, ID, and vehicle of the vehicle recognized in each frame of each scene into the corresponding file.

[0022] Furthermore, in step 3, the timestamp and geographic location information are used to synchronize the data, and finally the same frame of each scene is integrated into the same file.

[0023] Furthermore, in step 4, a dictionary is created. The keys of the dictionary are ID and license plate number, and the corresponding values ​​are lists consisting of scene numbers and ID numbers under the scene. Each license plate number corresponds to an ID. If the license plate of the vehicle in this scene is consistent with the license plate number of a registered vehicle, the scene number corresponding to the vehicle at this time and the ID number under the scene are added to the list of values ​​corresponding to the key corresponding to the license plate number; if the scene number and the ID number under the scene are detected in subsequent frames, the ID in the key corresponding to the list where the scene number and the ID number under the scene are located is assigned by default; if the vehicle is not registered, a unique ID is assigned to it, a new key ID and license plate number is created in the dictionary, an empty list is assigned to its value, and it is recorded in the database.

[0024] The beneficial effects of the present invention are as follows: the present invention provides a cross-camera license plate recognition and vehicle tracking system and method, which performs license plate recognition on the video streams of multiple cameras. If a license plate is recognized, it is matched with the license plate detected by the previous camera; if the matching result shows that the license plate numbers are the same, a unique ID consistent with its license plate number is assigned to the vehicle; through this method, continuous tracking of vehicles across cameras can be achieved, thereby improving the efficiency of traffic monitoring and management; the present invention not only overcomes the limitations of traditional vehicle appearance feature matching, but also can significantly improve the accuracy and reliability of cross-camera vehicle tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is the algorithm flow of the present invention;

[0026] Figure 2 This is a diagram of the tracking effect of two cameras. DETAILED DESCRIPTION

[0027] Specific implementation method 1: Figure 1 As shown, a cross-camera vehicle tracking system based on license plate historical sequence matching includes a license plate recognition module, a camera synchronization module, a license plate matching module, an ID allocation module, and a data storage and management module;

[0028] License plate recognition module: used to extract vehicle images from video streams of multiple cameras and identify the vehicle's license plate number;

[0029] Camera synchronization module: used to synchronize the video streams of multiple cameras and mark the time and location information of vehicle appearance.

[0030] License plate matching module: matches the license plate currently recognized by the camera with the previously recognized license plates in the database to determine whether they are the same vehicle;

[0031] ID assignment module: If the matching result shows that the license plates are the same, a unique ID that matches the license plate number will be assigned to the vehicle; otherwise, a new ID will be assigned.

[0032] Data storage and management module: records the license plate, time, location, and ID information of all vehicles recognized by the camera for subsequent vehicle tracking and analysis.

[0033] The system process is as follows:

[0034] Video capture: Multiple cameras capture traffic flow videos and transmit them to the central processing system in real time. i , i=1,…,N.

[0035] License plate recognition: The license plate recognition module extracts vehicle images from the video and identifies the license plate number. When a vehicle enters the i-th video stream, a temporary ID is assigned to the vehicle: x i , the license plate detected in the video stream is recorded as M indicates that there are M license plates detected before time t.

[0036] License Plate Matching:

[0037] (Direct Matching) The license plate matching module compares the currently recognized license plate with the license plate previously recorded in the database to determine whether it is the same vehicle. At the same time, if a vehicle is also detected in another video stream j and is assigned a temporary ID: x j , for the new license plate detected and Compare all the license plates in , if there are the same license plate numbers, let x j ←x i , if With any The license plates in are all different, so construct Store license plates for comparison with vehicles in other video streams.

[0038] (Weighted matching) The license plate matching module compares the currently recognized license plate with the license plate previously recorded in the database to determine whether it is the same vehicle. At the same time, if another vehicle is also detected in another video stream j and is assigned a temporary ID: x j , for the new license plate detected, construct Store the license plate until the vehicle disappears from the video stream j. All license plates and Compare all the license plates in the table and set an initial weight of 0. If the qth license plate matches the license plates in other video streams successfully, then Final Order Equal to the license plate with the highest weight, x jAlso give the highest weight plate matching success of the license plate ID.

[0039] ID allocation: if the license plate matching result shows that the license plate is the same, the ID allocation module will allocate a unique ID (x j ←x i ) consistent with its license plate number for the vehicle; if not, generate a new ID (x j ).

[0040] Information storage and management: all identified license plate numbers, timestamps, locations and allocated IDs, etc. Information is stored for subsequent query and analysis.

[0041] Specific implementation method two: as shown in Figure 1 , a cross-camera vehicle tracking method based on license plate historical sequence matching, the specific steps include:

[0042] Step 1, obtain the video to be detected by multiple cameras, and record the camera's intrinsic parameters, detect the target in the video to be detected, and allocate ID and track the trajectory of the detected vehicle in the camera;

[0043] Step 2, capture the vehicle video stream entering the parking lot through the entrance camera, and extract the license plate number using the license plate recognition module; use the trained convolutional neural network for vehicle detection and license plate recognition, and write the position, ID and vehicle of each vehicle recognized in each frame of each scene into the corresponding file;

[0044] Step 3, transmit the license plate recognition result to the central processing system, and at the same time, synchronize the data to ensure the consistency of time and location; use the timestamp and geographic location information to synchronize the data, and finally integrate the same frame of each scene into the same file;

[0045] Step 4, the vehicle ID allocation central processing system receives the recognized license plate number, and queries the database; create a dictionary, the key of the dictionary is ID and license plate number, and the corresponding value is a list composed of scene number and ID number under the scene, each license plate number corresponds to an ID, if the license plate of the vehicle in this scene is consistent with the license plate number of the registered vehicle, then the scene number and the ID number under the scene corresponding to the vehicle at this time are added to the list corresponding to the value of the key corresponding to the license plate number; If detected in the subsequent frame, the scene number and the ID number under the scene are defaulted to be the ID in the list corresponding to the key; if the vehicle is not registered, allocate a unique ID, create a new key ID and license plate number in the dictionary, and assign an empty list to its value, and record it in the database;

[0046] Step 5, according to the result of the database, regenerate a result based on license plate ID matching and write it into the file frame by frame;

[0047] Step 6: Visually track the vehicle based on its location, ID, and license plate information. Integrate the multi-camera vehicle information into a map in the same coordinate system through a unified coordinate system, draw the movement trajectory, and draw other parking spaces, lanes and other environmental information to finally generate a video.

[0048] How it works

[0049] Since license plates are detected based on video, the license plates detected are often wrong due to changes in lighting, vehicle occlusion, camera angle differences, etc., and it is difficult to determine which license plate is correct. This invention is based on a basic mathematical assumption: and The probability of falsely detecting the same license plate as an incorrect one is extremely small. Therefore, if two license plates are detected as identical in two video streams, the two vehicles are considered the same, and both license plate detections are considered correct. With direct matching, the probability of identical false detections is negligible. With weighted matching, the matching rate is slightly slower, but even if identical false detections occur, the probability is lower than that of identical correct detections. Weighting can eliminate identical false detections, providing higher accuracy. In practical applications, license plate matching can also be combined with spatial position matching to improve accuracy.

[0050] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any technician familiar with the present profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical content disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement of the above embodiments made according to the technical essence of the present invention, within the spirit and principles of the present invention, without departing from the content of the technical solution of the present invention, shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A cross-camera vehicle tracking system based on license plate history sequence matching, characterized in that: It includes license plate recognition module, camera synchronization module, license plate matching module, ID allocation module and data storage and management module; License plate recognition module: used to extract vehicle images from video streams of multiple cameras and identify the vehicle's license plate number; Camera synchronization module: used to synchronize the video streams of multiple cameras and mark the time and location information of vehicle appearance; License plate matching module: matches the license plate currently recognized by the camera with the previously recognized license plates in the database to determine whether they are the same vehicle; Specifically: When a car enters A video stream is used to assign a temporary ID to the vehicle: , the license plate detected in the video stream is recorded as , , Indicates at time The number of license plates detected previously is At the same time, if another video stream A car is also detected and given a temporary ID: , for the new license plate detected, construct Store the license plate until the vehicle is removed from the video stream will disappear in All license plates and Compare all the license plates in the table and set an initial weight of 0. , , Ruodi If the license plate matches the license plate in other video streams successfully, then ; Final order Equal to the license plate with the highest weight, The license plate ID of the license plate with the highest weight is also assigned; ID allocation module: When the matching result shows that the license plates are the same, a unique ID that matches the license plate number is assigned to the vehicle; Otherwise assign a new ID; Data storage and management module: records the license plate, time, location, and ID information of all vehicles recognized by the camera for subsequent vehicle tracking and analysis.

2. A vehicle tracking method based on the cross-camera vehicle tracking system according to claim 1, characterized in that: The steps of the method include: Step 1: Acquire the video to be detected through multiple cameras, record the internal parameters of the cameras, perform target detection on the video to be detected, and assign IDs and track the detection vehicles in the cameras; Step 2: Capture the video stream of vehicles entering the parking lot through the entrance camera and use the license plate recognition module to extract the license plate number; Step 3: Transmit the license plate recognition results to the central processing system and synchronize the data to ensure consistency in time and location; Step 4: The vehicle ID allocation central processing system receives the identified license plate number and queries the database; Step 5: Based on the results of the database, regenerate a result based on the license plate ID matching and write it into the file frame by frame; Step 6: Visually track the vehicle based on its location, ID, and license plate information. Integrate the multi-camera vehicle information into a map in the same coordinate system through a unified coordinate system, draw the movement trajectory, and draw other parking spaces, lanes and other environmental information to finally generate a video.

3. The cross-camera vehicle tracking method based on license plate history sequence matching according to claim 2 is characterized in that: Step 2 specifically includes: using the trained convolutional neural network to perform vehicle detection and license plate recognition, and writing the position, ID, and vehicle of the vehicle recognized in each frame of each scene into the corresponding file.

4. The cross-camera vehicle tracking method based on license plate history sequence matching according to claim 2 is characterized in that: In step 3, the data is synchronized using timestamps and geographic location information, and finally the same frame of each scene is integrated into the same file.

5. The cross-camera vehicle tracking method based on license plate history sequence matching according to claim 2 is characterized in that: In step 4, create a dictionary whose keys are the ID and license plate number, and whose values ​​are a list consisting of the scene number and the ID number under that scene. Each license plate number corresponds to an ID. If the license plate number of the vehicle in this scene is the same as the license plate number of a registered vehicle, then add the scene number and the ID number corresponding to the vehicle at that time to the list of values ​​corresponding to the key corresponding to the license plate number. If a scene number and an ID number under that scene are detected in subsequent frames, the ID in the key corresponding to the list where the scene number and the ID number under that scene are located will be assigned by default; If the vehicle is not registered, assign it a unique ID, create new keys ID and license plate number in the dictionary, assign an empty list as its value, and record it in the database.

Images