Method for identifying and comparing license plates in ticketless car parks

Abstract

The invention relates to a method for identifying vehicle license plates in ticketless car parks, in which, as a digital fingerprint of the license plate, a raw evaluation matrix is used, which is obtained in a character recognition phase, as a step prior to generating the read of the license plate and an evaluation matrix converted into a character string. The invention also proposes a method for comparing two or more license plates and a method for managing ticketless car parks.

Description

[0001] DESCRIPTION

[0002] Procedure for identifying and comparing license plates in ticketless parking facilities

[0003] Technology sector

[0004] The present invention has applications in the field of vehicle traffic control systems; more specifically, in the control of vehicles entering and exiting a parking lot. It can also be applied to the field of devices for the recognition or understanding of images or videos, and for pattern recognition.

[0005] State of the art

[0006] The trend in the parking industry is for parking tickets to be replaced by vehicle registration. In other words, more and more parking facilities are calculating parking time by reading license plates upon entry and exit. This has led to an increasing number of license plate recognition (LPR or ANPR) systems being installed in parking facilities, often replacing or complementing traditional ticketing systems.

[0007] The operation of a ticketless parking system is based on the vehicle's license plate. When a vehicle enters the parking facility, its license plate is automatically read and used as the identifier on the virtual ticket generated by the parking management system. The parking user pays for their stay before leaving, so the parking management system requests the license plate number to know when the vehicle entered and calculate the time it has been parked. To prevent cases where the license plate is not read correctly upon entry, the user is shown several license plate options similar to the one they entered, allowing them to select the image of their ticket and thus ensuring that the payment always corresponds to the actual parking time.

[0008] It's important to note that automatic license plate recognition (APR) systems don't have a 100% reliability rate for several reasons, including license plate defects, lighting issues, camera placement problems, and so on. This can sometimes lead to incorrect license plate readings at the exit, causing the parking management system to receive an incorrect license plate number and preventing the vehicle from exiting automatically. In these cases, since the barrier doesn't open automatically, the user must contact the incident resolution control center, which manually reviews the camera image and corrects the incorrect license plate reading to allow the vehicle to exit.

[0009] Document US2022101031A1 compares license plate images with a database of previously captured images used as a reference. It uses license plate readability and other values ​​such as reliability and color for comparison.

[0010] Document CN107239778A describes an efficient method for license plate recognition, which does not seek to compare two different images of the same license plate to confirm whether it is the same, or not.

[0011] Document CN117953290A describes an object detection method based on low-quality labeled data. These objects are any that can be observed from a traffic surveillance camera.

[0012] Document KR20230135268A describes a device and method for recognizing character strings on vehicle license plates, using convolutional neural networks (CNNs) to optimize processing across various devices. During training, only the content of the license plate is used, without any information about its position, size, or width. Inference is performed with a minimal post-processing algorithm, allowing the recognition of various license plate types with a single algorithm, thus reducing complexity and associated costs.

[0013] In the state of the art, the use of a raw evaluation matrix and an evaluation matrix converted into a string of characters as an identifier (token) or digital fingerprint of the license plate has not been observed, to reduce the probability of error when detecting the characters that make up the license plate.

[0014] Explanation of the invention

[0015] Based on the prior art, the objective of the present invention is to provide a procedure for identifying license plates in ticketless parking facilities that uses a raw score matrix and a score matrix converted into a string of characters as an identifier (token) or fingerprint of the license plate, thus allowing a reduction in the probability of error when detecting the characters.

[0016] In order to achieve the proposed objectives, the invention proposes a procedure that has the characteristics of claim 1.

[0017] The procedure for identifying vehicle license plates in ticketless parking lots essentially comprises:

[0018] - A first phase of capturing an image and reading a license plate, on which a character recognition algorithm is applied, which generates a set of discrete samples, each of which is evaluated by a deep learning training model to determine the character contained in said sample and with it a probability vector in which the probability of said character being each of the possible characters is indicated.

[0019] - Next, a two-dimensional evaluation matrix (score matrix) is formed by concatenating the different probability vectors.

[0020] - Finally, the estimated enrollment is constructed, after post-processing the generated evaluation matrix.

[0021] And, as a digital fingerprint, the license plate is used, the raw evaluation matrix obtained in the character recognition phase as a preliminary step to constructing the license plate reading and the evaluation matrix converted into a string of characters.

[0022] Likewise, the invention proposes a comparison procedure between two license plates identified by the procedure of claim 1.

[0023] Finally, a parking management procedure is also proposed that uses the license plate identification procedure of claim 1 and the license plate comparison procedure mentioned above.

[0024] To complement the description being made and in order to facilitate the understanding of the characteristics of the invention, a set of drawings is included with this descriptive report in which, for illustrative and non-limiting purposes, the following has been represented: Fig. 1 shows the process of automatic license plate recognition by means of video analysis.

[0025] Figures 2.1 and 2.2 represent the complete process in its various stages.

[0026] Fig. 3 shows a raw evaluation matrix, obtained from the reading of the represented license plate.

[0027] Realization of the invention

[0028] The procedure for identifying and comparing license plates in ticketless parking facilities employs an automatic license plate recognition process using video analysis (see Fig. 1), which comprises the following stages: a) License plate detection: This is performed using an application that receives the image to be analyzed as input. The first stage applied to the image is license plate detection, resulting in an estimated region of the license plate's location within the image. b) Detection of characters within the license plate: The next stage processes the license plate region detected in the previous stage and performs a series of operations that result in the region where character recognition is processed. (See Fig. 2.1). c) Character recognition: This process consists of applying a sequential scanning algorithm to generate a set of discrete samples.Each sample is evaluated by the model to determine the character it contains. For each position within the box, the model analyzes which character is present and its probability percentage. This data is obtained from a Deep Learning neural network specifically developed and trained for this function. The model's inference for each sample results in a probability vector indicating the likelihood of each possible character. The concatenation of these vectors creates a two-dimensional matrix where each column corresponds to the recognition model's results for the different positions analyzed within the region containing the license plate text. This two-dimensional matrix is ​​called the score matrix and is the result of combining the probability vectors for each character in each position. (See Fig. 2.2).d) Syntax: The construction of the estimated reading of the registration occurs after a post-processing of the generated evaluation matrix.

[0029] This process can lead to errors in the estimated reading, which can cause the same license plate to have different reading results depending on the perspective, lighting, or size. This explains why there can be circumstances in which the automatic recognition of the same license plate can obtain different results at the entrance and exit of a parking lot depending on the post-processing stages and syntax applied to the score matrix.

[0030] Therefore, a method is proposed to generate a license plate identifier (token) or digital fingerprint using the data obtained in the character recognition phase, as a preliminary step to reading the license plate. It is also proposed to use the raw evaluation matrix and the evaluation matrix converted into a character string as the license plate identifier (token) or digital fingerprint.

[0031] To validate that the evaluation matrix can be used as an identifier (token) of a registration, it is necessary to implement a procedure for comparing two registrations that allows us to state unequivocally whether two different evaluation matrices correspond to the same registration, or not.

[0032] In those cases where the same license plate has two different readings for the same character, it has been observed that the processing of the evaluation matrix causes the loss of much information, since only the character with the highest score is preserved and the rest are lost. For this reason, it has been determined to use the vectors that make up the raw evaluation matrix.

[0033] This comparison process employs an algorithm that comprises the following stages for comparing two score matrices:

[0034] Normalization of evaluation matrices: The two evaluation matrices are normalized so that they have the same dimension and can be compared.

[0035] Comparison of normalized evaluation matrices

[0036] The first difficulty encountered when comparing two evaluation matrices is that they often have different lengths due to the vehicle's tilt relative to the camera and potential erroneous detections. Therefore, the first task is to normalize both evaluation matrices to facilitate comparison. During this stage, the matrices are also binarized.

[0037] Using the normalized and binarized matrices, two scores are calculated that will be used to decide if both evaluation matrices correspond to the same enrollment:

[0038] • Match: Percentage of positions in which the detected character matches.

[0039] • Detection: Percentage of positions where there is a detection.

[0040] These scores define three rules for determining whether two license plates are the same: a) The match score exceeds a minimum value. b) The detection score exceeds a minimum value. c) The match score is greater than the detection score, to prevent sequential license plates from being considered the same.

[0041] For example, two readings are obtained for the license plate 610-WCH: “60CH” and “610WCH”. In this case, the match is 96% and the detection is 87%, and therefore the match is greater than the detection, which would indicate that it is the same license plate. It is observed, then, that although both readings are so different that even when using the Levenshtein distance determination algorithm between them, a value greater than 1 is not obtained, it could be said that both readings correspond to the same license plate.

[0042] The designed algorithm also demonstrates its effectiveness for sequential license plates; for example, when reading the license plates G-118HF and G-119HF, obtaining correct readings: “G118HF” and “G119HF”, the following scores would be obtained: Match: 89%, Detection: 94%, and therefore Match > Detection, so it would be determined that it is not the same license plate.

[0043] When using the vehicle registration identification procedure described, it is also necessary to have an incident resolution procedure in place, which is developed according to the guidelines indicated below:

[0044] When a vehicle enters the parking lot, the license plate reading and its associated score matrix are stored in a database. When the vehicle is about to leave, the license plate recognition system searches the database for the corresponding license plate number. If there is a 100% match with any of the license plates in the database, the read license plate is sent to the parking management system, and the vehicle is removed from the database. If there is not a 100% match between the exit reading and the license plates stored in the database, the system searches for the most similar license plates using Levenshtein's algorithm, which measures the distance between two words. The system then retrieves the evaluation matrices of the license plates with the smallest Levenshtein distance.In the next step, the different candidate evaluation matrices are compared with the evaluation matrix of the license plate reading of the vehicle at the exit. This reveals the license plate recognition result at the entrance for the exiting license plate, allowing the same reading sent at the entrance to be sent to the parking management system. This enables the parking system to automatically close the vehicle's parking space without requiring any human intervention.

[0045] Once the nature of the invention has been sufficiently described, as well as an example of a preferred embodiment, it is noted for the appropriate purposes that the materials, shape, size and arrangement of the elements described may be modified, provided that this does not imply an alteration of the essential characteristics of the invention claimed below.

Claims

CLAIMS 1. Procedure for identifying vehicle license plates in ticketless parking lots, comprising: a first phase of capturing an image and reading a license plate, on which a character recognition algorithm is applied, generating a set of discrete samples, each of which is evaluated by a deep learning training model to determine the character contained in said sample and with it a probability vector indicating the probability that said character is each of the possible characters; forming a two-dimensional evaluation matrix (score matrix) with the concatenation of the different probability vectors; constructing the estimated license plate, after post-processing the generated evaluation matrix;in which, as a digital fingerprint of the license plate, the raw evaluation matrix obtained in the character recognition phase is used, as a preliminary step to constructing the license plate reading and the evaluation matrix converted into a string of characters.; 2. A comparison procedure between two or more registrations, according to claim 1, which allows one to affirm that two evaluation matrices are different, or correspond to the same registration, in which the comparison is made between the characters that make up the registration and / or the evaluation matrices of each registration. 3.- Method, according to claim 2, wherein the processing of the evaluation matrices uses the raw evaluation matrices that include the detections of found characters. 4.- Method, according to claims 2-3, wherein the algorithm for comparing registration numbers comprises, prior to the comparison between evaluation matrices, a stage of normalizing the evaluation matrices. 5.- Method, according to claims 2-4, in which the evaluation matrices are normalized and transformed into binary data.

6. A method, according to claims 2-5, wherein two scores are calculated on the evaluation matrices and used to decide whether the evaluation matrices to be compared correspond to the same enrollment: - Match: percentage of positions in which the detected character matches. - Detection: percentage of positions where detection occurs, determining that two license plates are the same when: - the match score exceeds a minimum value. - the detection score exceeds a minimum value. - the match score is higher than the detection score. 7.- Procedure, according to claims 2-6, in which, when it is observed that two character readings do not allow determining that two license plates are the same, the comparison between the fingerprints between license plates is applied. 8.- Procedure for managing ticketless parking, according to the claims 1-2, in which, when a vehicle enters the parking lot, the license plate reading obtained and the evaluation matrix associated with the image of said license plate are saved in a database, and when the vehicle leaves the parking lot, the license plate recognition system at the exit determines the possible license plate reading and the evaluation matrix associated with the image of said license plate, and then searches for the result obtained in the database of vehicles that are inside the parking lot and: - If there is confirmation that the vehicle's license plate at the exit matches any of the license plates in the stock, the read license plate is sent to the parking management system and the vehicle is removed from the stock; - Otherwise, the search is conducted among the most similar license plates and the evaluation matrices of these results are retrieved, and then the different evaluation matrices of the candidate readings are compared with the evaluation matrix of the output reading until one is found that matches one of the license plates in the stock, in which case the read license plate is sent to the parking management system and the vehicle is removed from the stock.

Images