A license plate recognition method and system based on vehicle feature secondary verification

Abstract

The present application relates to the technical field of license plate recognition, and particularly relates to a license plate recognition method and system based on vehicle feature secondary verification, comprising: obtaining static interference data recognized by a to-be-trained license plate recognition device, and obtaining a first training set; training a first license plate recognition device based on the first training set; performing feature recognition on a vehicle based on a vehicle feature recognition sub-module of the first license plate recognition device; generating a ReID re-identification mark based on vehicle feature data, and associating the ReID re-identification mark with license plate information to obtain vehicle comprehensive feature information; obtaining dynamic interference data recognized by the first license plate recognition device, adding the dynamic interference data to the first training set to obtain a second training set, and training the first license plate recognition device based on the second training set; and sequentially performing license plate recognition and secondary verification based on a second license plate recognition device and the vehicle comprehensive feature information, so that the present application can effectively improve license plate recognition efficiency and reduce false positive rate.

Description

Technical Field

[0001] This invention relates to the field of license plate recognition technology, and in particular to a license plate recognition method and system based on secondary verification of vehicle features. Background Technology

[0002] License plate recognition systems are widely used at highway toll stations. These systems capture vehicle license plate information using high-definition cameras and leverage advanced image processing and recognition technology to achieve rapid and accurate vehicle identification. When a vehicle passes through a toll station, the system automatically reads the license plate number, uploads it to the toll collection system, and completes the toll collection process.

[0003] Before the elimination of provincial border toll stations, highway toll station license plate recognition relied on the exit point, without comparing entry and exit license plates. In actual traffic, due to factors such as license plate damage, recognition angle, and lighting conditions, discrepancies frequently occurred between entry and exit license plates, creating special circumstances and presenting the following problems:

[0004] 1. If the license plate at the entrance / exit does not match, the system will enter a special situation marking process. Instead of billing the card, the system will redirect to online billing. During the online billing process, if the license plate at the entrance / exit does not match, billing will be refused, and only the minimum fee will be charged, resulting in a loss of uncollected toll fees for the road owner.

[0005] 2. In special circumstances, toll collectors need to operate the lane toll collection software to modify and confirm license plates, which causes vehicles in mixed lanes to move slowly and causes congestion at the toll station. Summary of the Invention

[0006] The purpose of this invention is to provide a license plate recognition method and system based on secondary verification of vehicle features, which solves the problems of mismatched license plates at entrances and exits and congestion at toll stations caused by license plate checks in the existing technology.

[0007] To achieve the above objectives, the present invention provides a license plate recognition method based on secondary verification of vehicle features, comprising the following steps:

[0008] S1. Obtain static interference data recognized by the license plate recognition device to be trained, add the static interference data to the original training set of the license plate recognition device to be trained to obtain the first training set, and train the first license plate recognition device based on the first training set.

[0009] Among them, static interference data is the background image that is misdetected as a license plate area by the license plate recognition device to be trained in a scene without vehicles.

[0010] S2. The vehicle feature recognition submodule based on the first license plate recognition device performs feature recognition on the vehicle to obtain vehicle feature data, generates a ReID re-identification identifier based on the vehicle feature data, and associates the ReID re-identification identifier with the license plate information to obtain comprehensive vehicle feature information;

[0011] S3. Obtain the dynamic interference data identified by the first license plate recognition device, add the dynamic interference data to the first training set to obtain the second training set, and train the first license plate recognition device based on the second training set to obtain the second license plate recognition device.

[0012] Among them, dynamic interference data refers to moving images in a vehicle-occupied scene that are misdetected as license plate areas by the first license plate recognition device.

[0013] S4. Based on the second license plate recognition device and the vehicle's comprehensive feature information, license plate recognition and secondary verification are performed sequentially.

[0014] In some embodiments of this application, vehicle feature data includes: vehicle shape, body color, vehicle brand, vehicle model, vehicle outline, sun visor, annual inspection sticker, tissue box, pendant, ornament, passenger seat, and seat belt feature data.

[0015] In some embodiments of this application, in step S1, training the first license plate recognition device based on the first training set includes:

[0016] The Haar features are used to characterize each positive sample in the first training set to obtain the positive sample Haar feature vector;

[0017] The Haar features are used to characterize each negative sample in the first training set to obtain the negative sample Haar feature vector;

[0018] The first license plate classifier is obtained by training the Haar feature vectors of positive and negative samples using the Adaboost algorithm.

[0019] Positive samples are image regions containing license plates, while negative samples are image regions that do not contain license plates.

[0020] In some embodiments of this application, in S2, the vehicle feature recognition submodule of the first license plate recognition device performs feature recognition on the vehicle to obtain vehicle feature data, and generates a ReID re-identification identifier based on the vehicle feature data, specifically including:

[0021] S21. The vehicle feature recognition submodule of the first license plate recognition device acquires images of vehicles passing through the exit and entrance from the camera, and uses a deep convolutional neural network to extract vehicle features from the captured images, and generates a 128-dimensional feature vector based on the vehicle features.

[0022] S22. Generate a 16-bit unique vehicle identifier based on a composite hash algorithm and a 128-dimensional feature vector, and confirm it as a ReID re-identification identifier for rapid retrieval and matching when performing secondary verification of license plates.

[0023] In some embodiments of this application, in step S3, obtaining a second training set and training the first license plate recognition device based on the second training set includes:

[0024] The training parameters are adjusted based on the second training set, and the classifier performance of the first license plate recognition device is optimized based on the adjusted training parameters and combined with the cross-validation algorithm.

[0025] In some embodiments of this application, S4, the sequential license plate recognition and secondary verification based on the second license plate recognition device and the vehicle comprehensive feature information includes:

[0026] S41. A similarity calculation model is constructed using the cosine similarity algorithm and Euclidean distance to obtain a second license plate recognition device generated based on the similarity calculation model. The direction and distance parameters of the 128-dimensional feature vector are used to train the second license plate recognition device.

[0027] S42. Use the trained second license plate recognition device to perform a license plate recognition, and determine whether to perform a second verification based on the license plate recognition result. Specifically: when the license plate recognition result matches the ReID re-identification identifier based on the preset system matching principle, it is confirmed as a valid license plate and passage is allowed; if the license plate recognition result does not match the ReID re-identification identifier, passage is not allowed, and the second license plate recognition device performs a second verification based on vehicle characteristics.

[0028] In some embodiments of this application, a license plate recognition system based on secondary verification of vehicle features is also disclosed, comprising:

[0029] The data acquisition module is used to acquire static interference data recognized by the license plate recognition device to be trained, add the static interference data to the original training set of the license plate recognition device to be trained to obtain the first training set, and train the first license plate recognition device based on the first training set.

[0030] Among them, static interference data is the background image that is misdetected as a license plate area by the license plate recognition device to be trained in a scene without vehicles.

[0031] The first identification module is used to identify the features of the vehicle based on the vehicle feature identification submodule of the first license plate recognition device, obtain vehicle feature data, generate a ReID re-identification identifier based on the vehicle feature data, and associate the ReID re-identification identifier with the license plate information to obtain comprehensive vehicle feature information.

[0032] The second recognition module is used to acquire dynamic interference data recognized by the first license plate recognition device, add the dynamic interference data to the first training set to obtain the second training set, and train the first license plate recognition device based on the second training set to obtain the second license plate recognition device.

[0033] Among them, dynamic interference data refers to moving images in a vehicle-occupied scene that are misdetected as license plate areas by the first license plate recognition device.

[0034] The secondary verification module is used to perform license plate recognition and secondary verification sequentially based on the second license plate recognition device and the vehicle's comprehensive feature information.

[0035] The advantages and beneficial effects of this invention compared to the prior art are:

[0036] In addition to recognizing the license plate information, the license plate recognition method of this invention also extracts vehicle feature information, including various vehicle features such as vehicle image, body color, vehicle brand, vehicle model, as well as vehicle outline, sun visor, annual inspection sticker, tissue box, pendant, ornament, passenger seat, seat belt and other markings. It combines the vehicle feature information to generate a ReID re-identification identifier, and performs secondary verification by combining the license plate recognition with the comprehensive vehicle feature information. Based on the vehicle features, it helps to improve the license plate recognition rate and greatly improve the traffic efficiency of toll stations.

[0037] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0038] Figure 1 This is a schematic diagram illustrating the steps of a license plate recognition method based on secondary verification of vehicle features in an embodiment of the present invention;

[0039] Figure 2 This is a structural block diagram of a license plate recognition system based on secondary verification of vehicle features according to an embodiment of the present invention. Detailed Implementation

[0040] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product is in use. They are used only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," and "connect" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0041] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0042] like Figure 1 As shown, this invention provides a license plate recognition method based on secondary verification of vehicle features, comprising the following steps:

[0043] S1. Obtain static interference data recognized by the license plate recognition device to be trained, add the static interference data to the original training set of the license plate recognition device to be trained to obtain the first training set, and train the first license plate recognition device based on the first training set.

[0044] Among them, static interference data is the background image that is misdetected as a license plate area by the license plate recognition device to be trained in a scene without vehicles.

[0045] S2. The vehicle feature recognition submodule based on the first license plate recognition device performs feature recognition on the vehicle to obtain vehicle feature data, generates a ReID re-identification identifier based on the vehicle feature data, and associates the ReID re-identification identifier with the license plate information to obtain comprehensive vehicle feature information.

[0046] S3. Obtain the dynamic interference data identified by the first license plate recognition device, add the dynamic interference data to the first training set to obtain the second training set, and train the first license plate recognition device based on the second training set to obtain the second license plate recognition device.

[0047] Among them, dynamic interference data refers to moving images in a vehicle-occupied scene that are mistakenly detected as license plate areas by the first license plate recognition device.

[0048] S4. Based on the second license plate recognition device and the vehicle's comprehensive feature information, license plate recognition and secondary verification are performed sequentially.

[0049] It's important to understand that ReID (Re-Identification) technology is a computer vision technique primarily used for identifying and retrieving the same target (such as pedestrians or vehicles) under different cameras or in different scenarios. A ReID re-identification tag is a crucial concept in ReID technology, used to uniquely identify the characteristics of a target object, enabling rapid target location and identification during subsequent retrieval and matching processes. This application utilizes ReID re-identification tags and the license plate recognition results of a second license plate recognition device for secondary verification, effectively improving recognition efficiency and reducing false alarms.

[0050] Specifically, this application improves recognition accuracy by extracting vehicle body features and searching for similarity, which can greatly reduce false alarms and adapt to complex scenarios. Even if the license plate is obscured, the vehicle can still be identified through vehicle body features.

[0051] In some embodiments of this application, vehicle feature data includes: vehicle shape, body color, vehicle brand, vehicle model, vehicle outline, sun visor, annual inspection sticker, tissue box, pendant, ornament, passenger seat, and seat belt feature data.

[0052] In some embodiments of this application, in step S1, training the first license plate recognition device based on the first training set includes:

[0053] The Haar features are used to characterize each positive sample in the first training set to obtain the positive sample Haar feature vector;

[0054] The Haar features are used to characterize each negative sample in the first training set to obtain the negative sample Haar feature vector;

[0055] The first license plate classifier is obtained by training the Haar feature vectors of positive and negative samples using the Adaboost algorithm.

[0056] Positive samples are image regions containing license plates, while negative samples are image regions that do not contain license plates.

[0057] In some embodiments of this application, in S2, the vehicle feature recognition submodule of the first license plate recognition device performs feature recognition on the vehicle to obtain vehicle feature data, and generates a ReID re-identification identifier based on the vehicle feature data, specifically including:

[0058] S21. The vehicle feature recognition submodule of the first license plate recognition device acquires images of vehicles passing through the exit and entrance from the camera, and uses a deep convolutional neural network to extract vehicle features from the captured images, and generates a 128-dimensional feature vector based on the vehicle features.

[0059] S22. Generate a 16-bit unique vehicle identifier based on a composite hash algorithm and a 128-dimensional feature vector, and confirm it as a ReID re-identification identifier for rapid retrieval and matching when performing secondary verification of license plates.

[0060] In some embodiments of this application, in step S3, obtaining a second training set and training the first license plate recognition device based on the second training set includes:

[0061] The training parameters are adjusted based on the second training set, and the classifier performance of the first license plate recognition device is optimized based on the adjusted training parameters and combined with the cross-validation algorithm.

[0062] In some embodiments of this application, S4, the sequential license plate recognition and secondary verification based on the second license plate recognition device and the vehicle comprehensive feature information includes:

[0063] S41. A similarity calculation model is constructed using the cosine similarity algorithm and Euclidean distance to obtain a second license plate recognition device generated based on the similarity calculation model. The direction and distance parameters of the 128-dimensional feature vector are used to train the second license plate recognition device.

[0064] S42. Use the trained second license plate recognition device to perform a license plate recognition, and determine whether to perform a second verification based on the license plate recognition result. Specifically: when the license plate recognition result matches the ReID re-identification identifier based on the preset system matching principle, it is confirmed as a valid license plate and passage is allowed; if the license plate recognition result does not match the ReID re-identification identifier, passage is not allowed, and the second license plate recognition device performs a second verification based on vehicle characteristics.

[0065] In some embodiments of this application, such as Figure 2 As shown, a license plate recognition system based on secondary verification of vehicle features is also disclosed, including:

[0066] The data acquisition module is used to acquire static interference data recognized by the license plate recognition device to be trained, add the static interference data to the original training set of the license plate recognition device to be trained to obtain the first training set, and train the first license plate recognition device based on the first training set.

[0067] Among them, static interference data is the background image that is misdetected as a license plate area by the license plate recognition device to be trained in a scene without vehicles.

[0068] The first identification module is used to identify the features of the vehicle based on the vehicle feature identification submodule of the first license plate recognition device, obtain vehicle feature data, generate a ReID re-identification identifier based on the vehicle feature data, and associate the ReID re-identification identifier with the license plate information to obtain comprehensive vehicle feature information.

[0069] The second recognition module is used to acquire dynamic interference data recognized by the first license plate recognition device, add the dynamic interference data to the first training set to obtain the second training set, and train the first license plate recognition device based on the second training set to obtain the second license plate recognition device.

[0070] Among them, dynamic interference data refers to moving images in a vehicle-occupied scene that are mistakenly detected as license plate areas by the first license plate recognition device.

[0071] The secondary verification module is used to perform license plate recognition and secondary verification sequentially based on the second license plate recognition device and the vehicle's comprehensive feature information.

[0072] The advantages and beneficial effects of this invention compared to the prior art are:

[0073] In addition to recognizing the license plate information, the license plate recognition method of this invention also extracts vehicle feature information, including various vehicle features such as vehicle image, body color, vehicle brand, vehicle model, as well as vehicle outline, sun visor, annual inspection sticker, tissue box, pendant, ornament, passenger seat, seat belt and other markings. It combines the vehicle feature information to generate a ReID re-identification identifier, and performs secondary verification by combining the license plate recognition with the comprehensive vehicle feature information. Based on the vehicle features, it helps to improve the license plate recognition rate and greatly improve the traffic efficiency of toll stations.

[0074] In this application, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. In case of any inconsistency, the meaning set forth in this specification or derived from the content described herein shall prevail. Furthermore, the terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit the scope of this application.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A license plate recognition method based on secondary verification of vehicle features, characterized in that, Includes the following steps: S1. Obtain static interference data recognized by the license plate recognition device to be trained, add the static interference data to the original training set of the license plate recognition device to be trained to obtain the first training set, and train the first license plate recognition device based on the first training set. Among them, static interference data is the background image that is misdetected as a license plate area by the license plate recognition device to be trained in a scene without vehicles. S2. The vehicle feature recognition submodule based on the first license plate recognition device performs feature recognition on the vehicle to obtain vehicle feature data, generates a ReID re-identification identifier based on the vehicle feature data, and associates the ReID re-identification identifier with the license plate information to obtain comprehensive vehicle feature information; S3. Obtain the dynamic interference data identified by the first license plate recognition device, add the dynamic interference data to the first training set to obtain the second training set, and train the first license plate recognition device based on the second training set to obtain the second license plate recognition device. Among them, dynamic interference data refers to moving images in the scene with vehicles that are misdetected as license plate areas by the first license plate recognition device; S4. Based on the second license plate recognition device and the comprehensive vehicle feature information, license plate recognition and secondary verification are performed sequentially. In step S2, the vehicle feature recognition submodule based on the first license plate recognition device performs feature recognition on the vehicle to obtain vehicle feature data, and generates a ReID re-identification identifier based on the vehicle feature data, specifically including: S21. The vehicle feature recognition submodule of the first license plate recognition device acquires images of vehicles passing through the exit and entrance from the camera, and uses a deep convolutional neural network to extract vehicle features from the captured images, and generates a 128-dimensional feature vector based on the vehicle features. S22. Generate a 16-bit unique vehicle identifier based on a composite hash algorithm and a 128-dimensional feature vector, and confirm it as a ReID re-identification identifier for quick retrieval and matching when performing secondary verification of license plates. In step S4, the sequential license plate recognition and secondary verification based on the second license plate recognition device and the vehicle comprehensive feature information includes: S41. A similarity calculation model is constructed using the cosine similarity algorithm and Euclidean distance to obtain a second license plate recognition device generated based on the similarity calculation model. The direction and distance parameters of the 128-dimensional feature vector are used to train the second license plate recognition device. S42. Use the trained second license plate recognition device to perform a license plate recognition, and determine whether to perform a second verification based on the license plate recognition result. Specifically: when the license plate recognition result matches the ReID re-identification identifier based on the preset system matching principle, it is confirmed as a valid license plate and passage is allowed; if the license plate recognition result does not match the ReID re-identification identifier, passage is not allowed, and the second license plate recognition device performs a second verification based on vehicle characteristics.

2. The license plate recognition method based on secondary verification of vehicle features according to claim 1, characterized in that, The vehicle feature data includes: vehicle appearance, body color, vehicle brand, vehicle model, vehicle outline, sun visor, annual inspection sticker, tissue box, pendants, ornaments, passenger seat and seat belt features.

3. The license plate recognition method based on secondary verification of vehicle features according to claim 2, characterized in that, In step S1, training the first license plate recognition device based on the first training set includes: The Haar features are used to characterize each positive sample in the first training set to obtain the positive sample Haar feature vector; The Haar features are used to characterize each negative sample in the first training set to obtain the negative sample Haar feature vector; The first license plate classifier is obtained by training the Haar feature vectors of positive and negative samples using the Adaboost algorithm. Positive samples are image regions containing license plates, while negative samples are image regions that do not contain license plates.

4. The license plate recognition method based on secondary verification of vehicle features according to claim 3, characterized in that, In step S3, obtaining the second training set and training the first license plate recognition device based on the second training set includes: The training parameters are adjusted based on the second training set, and the classifier performance of the first license plate recognition device is optimized based on the adjusted training parameters and combined with the cross-validation algorithm.

5. A license plate recognition system based on secondary verification of vehicle features, characterized in that, include: The data acquisition module is used to acquire static interference data recognized by the license plate recognition device to be trained, add the static interference data to the original training set of the license plate recognition device to be trained to obtain the first training set, and train the first license plate recognition device based on the first training set. Among them, static interference data is the background image that is misdetected as a license plate area by the license plate recognition device to be trained in a scene without vehicles. The first identification module is used to identify the features of the vehicle based on the vehicle feature identification submodule of the first license plate recognition device, obtain vehicle feature data, generate a ReID re-identification identifier based on the vehicle feature data, and associate the ReID re-identification identifier with the license plate information to obtain comprehensive vehicle feature information. The vehicle feature recognition submodule of the first license plate recognition device performs feature recognition on the vehicle to obtain vehicle feature data. The generation of a ReID re-identification identifier based on the vehicle feature data specifically includes: The vehicle feature recognition submodule of the first license plate recognition device acquires images of vehicles passing through the exit and entrance from the camera, and uses a deep convolutional neural network to extract vehicle features from the captured images, and generates a 128-dimensional feature vector based on the vehicle features. A 16-bit unique vehicle identifier is generated based on a composite hash algorithm and a 128-dimensional feature vector, and it is confirmed as a ReID re-identification identifier for rapid retrieval and matching when performing secondary verification of license plates. The second recognition module is used to acquire dynamic interference data recognized by the first license plate recognition device, add the dynamic interference data to the first training set to obtain the second training set, and train the first license plate recognition device based on the second training set to obtain the second license plate recognition device. Among them, dynamic interference data refers to moving images in the scene with vehicles that are misdetected as license plate areas by the first license plate recognition device; The secondary verification module is used to sequentially perform license plate recognition and secondary verification based on the second license plate recognition device and the vehicle's comprehensive feature information. Based on the second license plate recognition device and the vehicle's comprehensive feature information, license plate recognition and secondary verification are performed sequentially, including: A similarity calculation model was constructed using the cosine similarity algorithm and Euclidean distance to obtain a second license plate recognition device generated based on the similarity calculation model. The second license plate recognition device was then trained using the direction and distance parameters of a 128-dimensional feature vector. The trained second license plate recognition device performs a license plate recognition, and determines whether to perform a second verification based on the result of the first license plate recognition. Specifically, if the result of the first license plate recognition matches the ReID re-identification mark based on the preset system matching principle, it is confirmed as a valid license plate and passage is allowed; if the result of the first license plate recognition does not match the ReID re-identification mark, passage is not allowed, and the second license plate recognition device performs a second verification based on vehicle characteristics.

Images