A method and device for double identification of vehicle identity in a parking lot

Abstract

The application discloses a parking lot vehicle identity double discrimination method and device, and the method comprises the following steps: obtaining a vehicle photo of a vehicle to be entered; inputting the vehicle photo of the vehicle to be entered into a preset license plate recognition system to obtain a license plate number; inputting the license plate number of the vehicle to be entered into a pre-constructed database for matching; if a corresponding license plate number is matched, extracting a vehicle photo from the pre-constructed database; inputting the vehicle photo of the vehicle to be entered and the vehicle photo extracted from the database into a pre-constructed discrimination model for feature extraction; calculating the similarity of the vehicle photo of the vehicle to be entered and the vehicle photo extracted from the database according to the feature extraction result; and if the similarity is within a preset similarity threshold range, allowing the vehicle to be entered to enter. The application can well solve the problem of illegal vehicle entering the parking lot with a fake license plate.

Description

Technical Field

[0001] This invention relates to a method and apparatus for dual identification of vehicles in a parking lot, belonging to the field of computer vision technology. Background Technology

[0002] With the increase in urban parking lots and the continuous development of surveillance camera technology, more and more parking lots are using camera recognition technology to identify vehicles. Currently, most parking lots use license plate recognition to determine vehicle identity; however, in reality, there are often cases of illegal vehicles using counterfeit license plates, rendering license plate recognition sometimes ineffective. How to more accurately identify vehicles using cameras has always been one of the industry's pain points. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and device for dual identification of vehicle identity in parking lots, thereby solving the technical problem of illegal vehicles using counterfeit license plates to enter parking lots.

[0004] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0005] In a first aspect, the present invention provides a method for dual identification of vehicles in a parking lot, comprising:

[0006] Obtain a photo of the vehicle to be entered;

[0007] Input the vehicle photo of the vehicle to be entered into the preset license plate recognition system to obtain the license plate number;

[0008] The license plate number of the vehicle to be entered is entered into a pre-built database for matching. If a matching license plate number is found, the vehicle photo is extracted from the pre-built database.

[0009] The vehicle photos of the vehicles to be entered and the vehicle photos extracted from the database are input into a pre-built discrimination model for feature extraction to obtain feature vectors;

[0010] The similarity between the vehicle photo of the vehicle to be entered and the vehicle photo extracted from the database is calculated based on the feature vector. If the similarity is within the preset similarity threshold range, the vehicle to be entered is allowed to enter.

[0011] Optionally, if no matching license plate number is found or the similarity is not within the preset similarity threshold range, the vehicle to be entered will not be allowed to enter.

[0012] Optionally, the construction of the database includes:

[0013] Obtain vehicle photos and license plate numbers of legitimate vehicles, and generate a storage directory for the vehicle photos in the order they were obtained;

[0014] Create a vehicle information table in the database. The vehicle information table includes a CarID field and a CarNumber field of type char. The CarID field is used to store the storage directory of vehicle photos; the CarNumber field is used to store the license plate number; the CarID field and the CarNumber field together serve as the primary key of the vehicle information table.

[0015] Optionally, the construction of the discriminant model includes:

[0016] Obtain a dataset of vehicle photos, perform data augmentation on the vehicle photos in the dataset to generate training images, and construct a training set;

[0017] For each training image, a sliding window is used to generate patches with overlapping pixels;

[0018] A jigsaw puzzle module is added to the Vision Transformer network model to build the initial discriminative model;

[0019] The initial discriminative model is trained using the training set to obtain and output the trained discriminative model.

[0020] Optionally, the data augmentation process includes random horizontal flipping, filling, random cropping, and random erasing of the vehicle photos.

[0021] Optionally, training the initial discriminative model using the training set includes:

[0022] The following steps are executed iteratively until the preset number of iterations is reached:

[0023] The training image is input into the discriminative model. The N patches of the training image are processed by l-1 Transformer Layers to output the hidden feature Z. l-1 :

[0024]

[0025] In the formula, For category information, The feature values ​​of N patches;

[0026] Category information As a global feature f g ,right Local features are generated by performing shifting and shuffling operations;

[0027] The model loss is calculated based on global and local features, and the model parameters are optimized using the SGD optimizer based on the model loss.

[0028] Optionally, the shift operation includes:

[0029] Will The feature values ​​of the first m patches are moved to the end to generate a sequence.

[0030]

[0031] The mixed washing operation includes:

[0032] A sequence of 1 row and N columns Convert to k rows Column matrix

[0033] For matrix Perform a transpose operation, and after the transpose operation, expand it into a sequence of 1 row and N columns.

[0034] will sequence The feature values ​​of N patches are divided into k groups, and category information is added to the first element of each group. Generate k feature groups, denoted as {f l j |j=1,2,…,k} and used as local features.

[0035] Optionally, the model loss L is:

[0036]

[0037]

[0038]

[0039]

[0040] In the formula, ∈ is a preset constant, P is the number of different vehicles included in each batch of the discrimination model, and p i o i Based on global feature f g Local features f l j The predicted vehicle ID of the i-th vehicle is obtained, where y is the vehicle ID label;

[0041]

[0042]

[0043] In the formula, the global feature f g Let f be the reference. a , and the benchmark f aThe least similar global or local feature for the same vehicle ID is denoted as f. p The most similar global or local feature among different vehicle IDs is denoted as f. n ; local features f l j Let f be the reference. a ′ , and the benchmark f a ′ The least similar global or local feature for the same vehicle ID is denoted as f. p ′ The most similar global or local feature among different vehicle IDs is denoted as f. n ′ .

[0044] Optionally, calculating the similarity between the vehicle photo of the vehicle to be entered and the vehicle photo extracted from the database based on the feature extraction results includes:

[0045] The similarity cosθ is calculated using cosine similarity:

[0046]

[0047] Alternatively, the similarity d can be calculated using normalized Euclidean squared distance:

[0048]

[0049] In the formula, Q i G i These are the i-th dimension values ​​of the feature vectors of the vehicle photo to be entered and the vehicle photo extracted from the database, respectively, where n is the number of dimensions of the feature vector.

[0050] Secondly, the present invention provides a device for dual identification of vehicle identities in a parking lot, the device comprising:

[0051] The photo acquisition module is used to acquire photos of the vehicles to be entered.

[0052] The number acquisition module is used to input the vehicle photo of the vehicle to be entered into the preset license plate recognition system and obtain the license plate number;

[0053] The photo extraction module is used to input the license plate number of the vehicle to be entered into a pre-built database for matching. If a matching license plate number is found, the vehicle photo is extracted from the pre-built database.

[0054] The feature extraction module is used to input the vehicle photos of the vehicles to be entered and the vehicle photos extracted from the database into the pre-built discrimination model to extract features and obtain feature vectors.

[0055] The similarity discrimination module is used to calculate the similarity between the vehicle photo of the vehicle to be entered and the vehicle photo extracted from the database based on the feature vector. If the similarity is within the preset similarity threshold range, the vehicle to be entered is allowed to enter.

[0056] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0057] This invention provides a method and apparatus for dual identification of vehicles in parking lots. It utilizes a license plate recognition system to obtain the license plate number of a vehicle to be entered and compares it with a database. If the license plate number exists, vehicle re-identification technology is used to further identify the vehicle: a discrimination model based on a Vision Transformer network model and a mosaic patch module is used to extract corresponding vehicle features, and the similarity between vehicles is approximately estimated using the distance between feature vectors, thereby achieving dual identification of the vehicle to be entered into the parking lot. This invention effectively solves the problem of illegal vehicles using counterfeit license plates entering parking lots. Attached Figure Description

[0058] Figure 1 This is a flowchart of the method for dual identification of vehicle identity in a parking lot provided in Embodiment 1 of the present invention;

[0059] Figure 2 This is a schematic diagram of the discriminant network provided in Embodiment 1 of the present invention. Detailed Implementation

[0060] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0061] Example 1:

[0062] like Figure 1 As shown, the present invention provides a method for dual identification of vehicles in a parking lot, comprising the following steps:

[0063] 1. Obtain a photo of the vehicle to be entered.

[0064] 2. Input the vehicle photo of the vehicle to be entered into the preset license plate recognition system to obtain the license plate number;

[0065] The license plate recognition system used in this embodiment is a common license plate recognition system on the market.

[0066] 3. Input the license plate number of the vehicle to be entered into the pre-built database for matching. If a matching license plate number is found, extract the vehicle photo from the pre-built database.

[0067] The construction of the database includes:

[0068] S3.1 Obtain vehicle photos and license plate numbers of legitimate vehicles, and generate a storage directory for vehicle photos in the order of acquisition;

[0069] Legitimate vehicles are those already registered in the parking lot. During registration, a photo of the vehicle and its license plate number must be provided. The storage directory is generated in the order of registration, in the format of "car + serial number". If it is the first registered vehicle, its storage directory is car1.

[0070] S3.2 Create a vehicle information table in the database. The vehicle information table includes a CarID field and a CarNumber field of type char. The CarID field is used to store the storage directory of vehicle photos; the CarNumber field is used to store the license plate number; the CarID field and the CarNumber field together serve as the primary key of the vehicle information table.

[0071] 4. Input the vehicle photos of the vehicles to be entered and the vehicle photos extracted from the database into the pre-built discrimination model to extract features and obtain feature vectors;

[0072] The construction of the discriminative model includes:

[0073] S4.1 Obtain a dataset of vehicle photos, perform data augmentation on the vehicle photos in the dataset to generate training images, and construct a training set;

[0074] Data sets can be obtained by taking photos yourself, or you can use publicly available datasets such as VeRi-776 and VehicleID.

[0075] Data augmentation processing includes random horizontal flipping, filling, random cropping, and random erasure of vehicle photos.

[0076] S4.2. Use a sliding window to generate patches with overlapping pixels for each training image;

[0077] Let the window size be A×A and the sliding step size be S. Then, a training image with a resolution of H×W will be divided into... One patch.

[0078] S4.3. Add a jigsaw puzzle module to the Vision Transformer network model to build the initial discriminant model.

[0079] S4.4 Train the initial discriminative model using the training set, obtain the trained discriminative model, and output it.

[0080] Training the initial discriminative model using the training set includes:

[0081] Iteratively execute the following steps (such as) Figure 2(As shown), until the preset number of iterations is reached:

[0082] (1) Input the training image into the discriminant model. The N patches of the training image are processed by l-1 TransformerLayers to output the hidden feature Z. l-1 :

[0083]

[0084] In the formula, For category information, Let l represent the feature values ​​of N patches; l is the total number of Transformer Layers, and N is the total number of patches.

[0085] (2) Category information As a global feature f g ,right Local features are generated by performing shifting and shuffling operations;

[0086] Shift operations include:

[0087] Will The feature values ​​of the first m patches are moved to the end to generate a sequence.

[0088]

[0089] In the formula, m is a preset constant;

[0090] Mixed washing operations include:

[0091] A sequence of 1 row and N columns Convert to k rows Column matrix

[0092] For matrix Perform a transpose operation, and after the transpose operation, expand it into a sequence of 1 row and N columns.

[0093] will sequence The feature values ​​of N patches are divided into k groups, and category information is added to the first element of each group. Generate k feature groups, denoted as {f l j |j=1,2,…,k} and used as local features;

[0094] exist Figure 2 The example given is the puzzle patch module, namely Z. l-1 =[*;1,2,…,9], after processing, f l 1 =[*;9,2,6]、fl 2 =[*;1,8,5]、f l 3 =[*;4,7,3].

[0095] (3) Calculate the model loss based on global and local features, and optimize the model parameters using the SGD optimizer based on the model loss.

[0096] The model loss L is:

[0097]

[0098]

[0099]

[0100]

[0101] In the formula, ∈ is a preset constant, P is the number of different vehicles included in each batch of the discrimination model, and p i o i Based on global feature f g Local features f l j The predicted vehicle ID of the i-th vehicle is obtained, where y is the vehicle ID label;

[0102]

[0103]

[0104] In the formula, the global feature f g Let f be the reference. a , and the benchmark f a The least similar global or local feature for the same vehicle ID is denoted as f. p The most similar global or local feature among different vehicle IDs is denoted as f. n ; local features f l j Let f be the reference. a ′ , and the benchmark f a ′ The least similar global or local feature for the same vehicle ID is denoted as f. p ′ The most similar global or local feature among different vehicle IDs is denoted as f. n ′ .

[0105] 5. Calculate the similarity between the vehicle photo of the vehicle to be entered and the vehicle photo extracted from the database based on the feature vector. If the similarity is within a preset similarity threshold range, the vehicle to be entered is allowed to enter. In this embodiment, the calculation method used is as follows:

[0106] The similarity cosθ is calculated using cosine similarity:

[0107]

[0108] Alternatively, the similarity d can be calculated using normalized Euclidean squared distance:

[0109]

[0110] In the formula, Q i G i These are the i-th dimension values ​​of the feature vectors of the vehicle photo to be entered and the vehicle photo extracted from the database, respectively, where n is the number of dimensions of the feature vector.

[0111] Example 2:

[0112] This invention provides a device for dual identification of vehicles in a parking lot, the device comprising:

[0113] The photo acquisition module is used to acquire photos of the vehicles to be entered.

[0114] The number acquisition module is used to input the vehicle photo of the vehicle to be entered into the preset license plate recognition system and obtain the license plate number;

[0115] The photo extraction module is used to input the license plate number of the vehicle to be entered into a pre-built database for matching. If a matching license plate number is found, the vehicle photo is extracted from the pre-built database.

[0116] The feature extraction module is used to input the vehicle photos of the vehicles to be entered and the vehicle photos extracted from the database into the pre-built discrimination model for feature extraction;

[0117] The similarity discrimination module is used to calculate the similarity between the vehicle photo of the vehicle to be entered and the vehicle photo extracted from the database based on the feature vector. If the similarity is within the preset similarity threshold range, the vehicle to be entered is allowed to enter.

[0118] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0120] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0121] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0122] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for double identification of a vehicle in a parking lot, characterized in that, The method comprises the following steps: acquiring a vehicle photo of a vehicle to be entered; inputting the vehicle photo of the vehicle to be entered into a preset license plate recognition system to acquire a license plate number; inputting the license plate number of the vehicle to be entered into a pre-constructed database for matching, and if a corresponding license plate number is matched, extracting a vehicle photo of the corresponding license plate number from the pre-constructed database; inputting the vehicle photo of the vehicle to be entered and the vehicle photo extracted from the database into a pre-constructed discrimination model for feature extraction to acquire a feature vector; calculating a similarity between the vehicle photo of the vehicle to be entered and the vehicle photo extracted from the database according to the feature vector, and if the similarity is within a preset similarity threshold range, allowing the vehicle to be entered to enter; wherein the construction of the discrimination model comprises: acquiring a data set of vehicle photos, performing data enhancement processing on the vehicle photos in the data set to generate training images, and constructing a training set; generating patches with overlapping pixels by using a sliding window on each training image; increasing a puzzle patch module on the basis of a Vision Transformer network model to construct an initial discrimination model; training the initial discrimination model by using the training set to acquire a trained discrimination model and output the trained discrimination model; the training process comprises: iteratively performing the following steps until a preset iteration number is reached: The training image is input into the discriminator model, and the patches of the training image are processed by the -1 Transformer Layer to output hidden features -1 Transformer Layer to output hidden features : ； In the formula, is category information, is characteristic values of the patches. Category information As a global feature ,right Local features are generated by performing shifting and shuffling operations; calculating a model loss according to global features and local features, and optimizing model parameters by using an SGD optimizer according to the model loss; wherein the shift operation comprises: The middle characteristic values of the patches are moved to the end, generating a sequence : ； the shuffling operation comprises: Line 1 sequence of columns Convert to OK Column matrix , ; on the matrix transpose operation, expanded into a 1 row after the transpose operation sequence of columns ; will sequence middle The feature values ​​of each patch are all divided into Group them and add category information at the beginning of each group. generate There are 3 feature groups, denoted as And as a local feature.

2. The method of claim 1, wherein, if a corresponding license plate number is not matched or the similarity is not within the preset similarity threshold range, not allowing the vehicle to be entered to enter.

3. The method of claim 1, wherein, The construction of the database comprises: acquiring vehicle photos and license plate numbers of legal vehicles, and generating a storage directory of the vehicle photos according to the acquisition order; creating a vehicle information table in the database, wherein the vehicle information table comprises a CarID field and a CarNumber field of a char type, the CarID field is used to store the storage directory of the vehicle photo, the CarNumber field is used to store the license plate number, and the CarID field and the CarNumber field jointly serve as the primary key of the vehicle information table.

4. The method of claim 1, wherein, The data enhancement processing comprises random horizontal flip processing, padding processing, random cropping processing and random erasing processing on the vehicle photo.

5. The method of claim 1, wherein, The model loss is: ； ； ； ； wherein, is a preset constant, is the number of different vehicles contained in each batch of the discriminant model, is the vehicle ID prediction of the i-th vehicle obtained according to the global feature is the vehicle ID prediction of the i-th vehicle obtained according to the local feature is the vehicle ID prediction of the i-th vehicle obtained according to the global feature is the vehicle ID prediction of the i-th vehicle obtained according to the local feature is the vehicle ID label; ； ； In the formula, the global feature is denoted as reference , the global feature is the most dissimilar global feature or local feature in the same vehicle ID, and the global feature is the most similar global feature or local feature in different vehicle IDs. The local feature is denoted as reference , the local feature is the most dissimilar global feature or local feature in the same vehicle ID, and the local feature is the most similar global feature or local feature in different vehicle IDs. ​ 6. The method of claim 1, wherein, The calculation of the similarity between the vehicle photo of the vehicle to be entered and the vehicle photo extracted from the database according to the feature vector comprises: Cosine similarity is used to calculate the similarity : ； or the normalized Euclidean squared distance is used to calculate the similarity : ； In the formula, are the feature vector values of the vehicle photo to be entered into the vehicle and the vehicle photo extracted from the database, respectively, in the i-th dimension, is the dimension number of the feature vector.​ 7. A parking lot vehicle identity double discrimination device, characterized by, The device is configured to perform the steps of the method according to any one of claims 1-6, and the device comprises: a photo acquisition module configured to acquire a vehicle photo of a vehicle to be entered; a number acquisition module configured to input the vehicle photo of the vehicle to be entered into a preset license plate recognition system to acquire a license plate number; a photo extraction module configured to input the license plate number of the vehicle to be entered into a pre-constructed database for matching, and if a corresponding license plate number is matched, extract a vehicle photo of the corresponding license plate number from the pre-constructed database; a feature extraction module configured to input the vehicle photo of the vehicle to be entered and the vehicle photo extracted from the database into a pre-constructed discrimination model for feature extraction to acquire a feature vector; The similarity discrimination module is configured to calculate the similarity between the vehicle photo of the vehicle to be entered and the vehicle photo extracted from the database according to the feature vector, and allow the vehicle to be entered to enter if the similarity is within a preset similarity threshold range.

Images