License plate recognition method and system based on vehicle feature secondary verification

Abstract

The invention relates to the technical field of license plate recognition, in particular to a license plate recognition method and system based on vehicle feature secondary verification, and the method comprises the steps: obtaining static interference data recognized by a to-be-trained license plate recognition device, obtaining a first training set, and training a first license plate recognition device based on the first training set; performing feature recognition on the vehicle based on a vehicle feature recognition sub-module of the first license plate recognition device, generating a ReID re-recognition identifier based on the vehicle feature data, and associating the ReID re-recognition identifier with the license plate information to obtain vehicle comprehensive feature information; obtaining dynamic interference data recognized by the first license plate recognition device, adding the dynamic interference data into the first training set to obtain a second training set, and training the first license plate recognition device based on the second training set; license plate recognition and secondary verification are sequentially performed based on the second license plate recognition device and the vehicle comprehensive feature information, the license plate recognition efficiency can be effectively improved, and the false alarm rate is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of license plate recognition, in particular to a license plate recognition method and system based on secondary verification of vehicle features. BACKGROUND

[0002] The license plate recognition system is very common in the application of highway toll stations. This system captures vehicle license plate information through high-definition cameras and realizes fast and accurate identification of vehicle information with the help of advanced image processing and recognition technology. When the vehicle passes through the toll station, the system can automatically read the license plate number and upload it to the charging system to realize the charging process.

[0003] Before the cancellation of provincial boundary toll stations, the license plate recognition of highway toll stations was based on the exit, without comparing the entry and exit license plates. Since 2019, when the provincial boundary toll stations were canceled, the exit of the toll station needs to be compared. Currently, the card-based charging method is preferred on the highways in Hebei Province. If the card-based charging fails, online charging is called (cross-provincial call center online charging), and if the online charging fails, the minimum fee is used as a backup for charging. In the actual traffic process, due to license plate pollution, license plate recognition angle, light irradiation, etc., the entry and exit license plates are often inconsistent, which forms a special situation and has the following problems:

[0004] 1. The entry and exit license plates do not match, the special situation is marked, the card-based charging is not performed, and the online charging is jumped to. During the online charging call process, the entry and exit license plates do not match, which will also refuse to charge, and only the minimum fee can be charged, resulting in a loss of the road owner's unpaid toll amount.

[0005] 2. The special situation requires the toll collector to operate the lane charging software to modify and confirm the license plate, which causes the mixed lane vehicles to travel slowly and causes congestion at the toll station. SUMMARY

[0006] The purpose of the present application is to provide a license plate recognition method and system based on secondary verification of vehicle features, which solves the problems of inconsistent entry and exit license plates and license plate inspection causing congestion at the toll station in the prior art.

[0007] To achieve the above-mentioned purpose, the present application provides a license plate recognition method based on secondary verification of vehicle features, comprising the following steps:

[0008] S1, obtaining static interference data to be recognized by a license plate recognition device, adding the static interference data to the original training set of the license plate recognition device to be trained to obtain a first training set, and training the first license plate recognition device based on the first training set;

[0009] Among them, the static interference data is a background image that is misdetected as a license plate area by the license plate recognition device to be trained in the scene without a vehicle;

[0010] S2, a vehicle feature recognition sub-module based on the first license plate recognition device recognizes the vehicle to obtain vehicle feature data, generates an ReID re-identification mark based on the vehicle feature data, and associates the ReID re-identification mark with the license plate information to obtain vehicle comprehensive feature information;

[0011] S3, obtaining dynamic interference data recognized by the first license plate recognition device, adding the dynamic interference data to the first training set to obtain a second training set, training the first license plate recognition device based on the second training set to obtain a second license plate recognition device;

[0012] The dynamic interference data is a moving image in a vehicle scene on site that is misdetected as a license plate region by the first license plate recognition device.

[0013] S4, license plate recognition and secondary verification are sequentially performed based on the second license plate recognition device and the vehicle comprehensive feature information.

[0014] In some embodiments of the present application, the vehicle feature data includes feature data of a vehicle shape, a vehicle body color, a vehicle brand, a vehicle model, a vehicle contour, a sun visor, an annual inspection mark, a tissue box, a hanging ornament, a decorative ornament, a co-pilot, and a seat belt.

[0015] In some embodiments of the present application, in S1, training the first license plate recognition device based on the first training set includes:

[0016] using Haar features to represent each positive sample in the first training set to obtain a positive sample Haar feature vector;

[0017] using Haar features to represent each negative sample in the first training set to obtain a negative sample Haar feature vector;

[0018] training the positive sample Haar feature vector and the negative sample Haar feature vector using an Adaboost algorithm to obtain a first license plate classifier;

[0019] The positive sample is an image region containing a license plate, and the negative sample is an image region not containing a license plate.

[0020] In some embodiments of the present application, in S2, the vehicle feature recognition sub-module of the first license plate recognition device recognizes the vehicle to obtain vehicle feature data, and generates an ReID re-identification mark based on the vehicle feature data, specifically including:

[0021] S21, the vehicle feature recognition sub-module of the first license plate recognition device obtains the shooting image of the exit and entrance passing vehicle by the camera, extracts the vehicle feature in the shooting image using a deep convolutional neural network, and generates a 128-dimensional feature vector based on the vehicle feature.

[0022] S22, generating a 16-bit unique vehicle identification based on the composite hash algorithm and the 128-dimensional feature vector, and confirming it as a ReID re-identification identification for fast retrieval and matching when performing license plate secondary verification.

[0023] In some embodiments of the present application, in S3, a second training set is obtained, and the first license plate recognition device is trained based on the second training set, including:

[0024] Adjust the training parameters based on the second training set, and optimize the performance of the classifier of the first license plate recognition device based on the adjusted training parameters and in combination with a cross-validation algorithm.

[0025] In some embodiments of the present application, in S4, license plate recognition and secondary verification are sequentially performed based on the second license plate recognition device and the vehicle comprehensive feature information, including:

[0026] S41, a similarity calculation model is constructed using a cosine similarity algorithm and a Euclidean distance, a second license plate recognition device generated based on the similarity calculation model is obtained, and the second license plate recognition device is trained using the direction and distance parameters of the 128-dimensional feature vector;

[0027] S42, the trained second license plate recognition device is used for primary license plate recognition, and whether to perform secondary verification is determined according to the primary license plate recognition result, specifically: when the primary license plate recognition result and the ReID re-identification identification match based on the preset system matching principle, it is confirmed as a valid license plate and allowed to pass; if the primary license plate recognition result and the ReID re-identification identification do not match, it is not allowed to pass, and the second license plate recognition device performs secondary verification based on the vehicle features.

[0028] In some embodiments of the present application, a license plate recognition system based on secondary verification of vehicle features is also disclosed, including:

[0029] A data acquisition module is configured to acquire static interference data identified by a to-be-trained license plate recognition device, add the static interference data to an original training set of the to-be-trained license plate recognition device to obtain a first training set, and train a first license plate recognition device based on the first training set;

[0030] The static interference data is a background image misidentified as a license plate region by the to-be-trained license plate recognition device in a field without a vehicle scene;

[0031] A first identification module is configured to perform feature identification on a vehicle based on a vehicle feature identification submodule of the first license plate recognition device to obtain vehicle feature data, generate a ReID re-identification identification based on the vehicle feature data, and associate the ReID re-identification identification with license plate information to obtain vehicle comprehensive feature information.

[0032] The second identification module is configured to obtain dynamic interference data identified by the first license plate recognition device, add the dynamic interference data to the first training set to obtain a second training set, train the first license plate recognition device based on the second training set, and obtain a second license plate recognition device.

[0033] The dynamic interference data is a moving image of a vehicle scene that is misidentified as a license plate area by the first license plate recognition device.

[0034] The second verification module is configured to sequentially perform license plate recognition and secondary verification based on the second license plate recognition device and the vehicle comprehensive feature information.

[0035] The advantages and beneficial effects of the present application over the prior art are as follows:

[0036] The license plate recognition method of the present application not only identifies the license plate information of the vehicle, but also extracts the vehicle feature information, which includes many vehicle features such as vehicle image, vehicle color, vehicle brand, vehicle model, vehicle outline, sun visor, annual inspection mark, paper towel box, hanging ornament, decorative ornament, co-driver, safety belt, and other marker, etc. The ReID re-identification mark is generated in combination with the vehicle feature information. The license plate recognition is combined with the vehicle comprehensive feature information for secondary verification. The vehicle features are used as the basis to assist in improving the license plate recognition rate and greatly improving the efficiency of the toll station.

[0037] The technical solutions of the present application will be further described in detail below with the aid of the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 The figure is a step schematic diagram of a license plate recognition method based on vehicle feature secondary verification according to an embodiment of the present application.

[0039] Figure 2 The figure is a structure block diagram of a license plate recognition system based on vehicle feature secondary verification according to an embodiment of the present application. DETAILED DESCRIPTION

[0040] In the description of the present application, it should be noted that the terms "upper", "lower", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the present application is usually placed, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In the description of the present application, it should be noted that, unless otherwise explicitly specified and limited, the terms "provided", "mounted", "connected" should be understood broadly, for example, can be fixedly connected, can also be detachably connected, or integrally connected; can be mechanically connected, can also be electrically connected; can be directly connected, can also be indirectly connected through an intermediate medium, can be internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0041] The embodiments of the present application will be described in detail below with reference to the drawings.

[0042] As Figure 1 shown, the present application provides a license plate recognition method based on vehicle feature secondary verification, comprising the following steps:

[0043] S1, obtaining static interference data to be recognized by a to-be-trained license plate recognition device, adding the static interference data to the original training set of the to-be-trained license plate recognition device to obtain a first training set, and training the first license plate recognition device based on the first training set;

[0044] The static interference data is a background image misrecognized as a license plate area by the to-be-trained license plate recognition device in a scene without a vehicle.

[0045] S2, performing feature recognition on the vehicle based on a vehicle feature recognition sub-module of the first license plate recognition device to obtain vehicle feature data, generating a ReID re-identification mark based on the vehicle feature data, and associating the ReID re-identification mark with license plate information to obtain vehicle comprehensive feature information.

[0046] S3, obtaining dynamic interference data recognized by the first license plate recognition device, adding the dynamic interference data to the first training set to obtain a second training set, training the first license plate recognition device based on the second training set to obtain a second license plate recognition device;

[0047] The dynamic interference data is a moving image misrecognized as a license plate area by the first license plate recognition device in a scene with a vehicle.

[0048] S4, sequentially performing license plate recognition and secondary verification based on the second license plate recognition device and the vehicle comprehensive feature information.

[0049] It should be understood that the ReID (Re-Identification) technology is a computer vision technology mainly used for identifying and retrieving the same target (such as pedestrians, vehicles, etc.) under different cameras or different scenes. The ReID identification is an important concept in the ReID technology, which is used to uniquely identify the features of the target object so as to quickly locate and identify the target in the subsequent retrieval and matching process. The present application uses the ReID identification and the license plate recognition result of the second license plate recognition device for secondary verification, which can effectively improve the identification efficiency and reduce false positives.

[0050] Specifically, in the present application, the identification accuracy is improved through vehicle body feature extraction and similarity search, which can greatly reduce false positives and adapt to complex scenes. Even if the license plate is blocked, the vehicle can be identified through the vehicle body features.

[0051] In some embodiments of the present application, the vehicle feature data includes: vehicle shape, vehicle color, vehicle brand, vehicle model, vehicle contour, sun visor, annual inspection mark, tissue box, hanging ornament, decorative ornament, co-driver and safety belt feature data.

[0052] In some embodiments of the present application, in S1, training the first license plate recognition device based on the first training set includes:

[0053] using Haar features to represent each positive sample in the first training set to obtain a positive sample Haar feature vector;

[0054] using Haar features to represent each negative sample in the first training set to obtain a negative sample Haar feature vector;

[0055] training the positive sample Haar feature vector and the negative sample Haar feature vector using the Adaboost algorithm to obtain the first license plate classifier;

[0056] wherein the positive sample is an image region containing a license plate, and the negative sample is an image region not containing a license plate.

[0057] In some embodiments of the present application, in S2, the vehicle feature recognition sub-module of the first license plate recognition device performs feature recognition on the vehicle to obtain vehicle feature data, and generating the ReID identification based on the vehicle feature data specifically includes:

[0058] S21, the vehicle feature recognition sub-module of the first license plate recognition device acquires the shooting images of the exit and entrance passing vehicles by the camera, and extracts the vehicle features in the shooting images using a deep convolutional neural network, and generates a 128-dimensional feature vector based on the vehicle features;

[0059] S22, generating a 16-bit unique vehicle identification based on the composite hash algorithm and the 128-dimensional feature vector, and confirming it as a ReID re-identification identification, which is used for fast retrieval and matching when performing license plate secondary verification.

[0060] In some embodiments of the present application, in S3, a second training set is obtained, and the first license plate recognition device is trained based on the second training set, including:

[0061] The training parameters are adjusted based on the second training set, and the classifier performance of the first license plate recognition device is optimized based on the adjusted training parameters and combined with a cross-validation algorithm.

[0062] In some embodiments of the present application, in S4, the license plate recognition and secondary verification are performed based on the second license plate recognition device and the vehicle comprehensive feature information, including:

[0063] S41, a similarity calculation model is constructed using a cosine similarity algorithm and a Euclidean distance, a second license plate recognition device is generated based on the similarity calculation model, and the second license plate recognition device is trained using the direction and distance parameters of the 128-dimensional feature vector;

[0064] S42, the trained second license plate recognition device is used for primary license plate recognition, and whether to perform secondary verification is determined according to the primary license plate recognition result, specifically: when the primary license plate recognition result and the ReID re-identification identification match based on the preset system matching principle, it is confirmed as a valid license plate and allowed to pass; if the primary license plate recognition result and the ReID re-identification identification do not match, it is not allowed to pass, and the second license plate recognition device performs secondary verification based on the vehicle features.

[0065] In some embodiments of the present application, as shown in Figure 2 A license plate recognition system based on secondary verification of vehicle features is also disclosed, including:

[0066] The data acquisition module is configured to acquire static interference data recognized by the to-be-trained license plate recognition device, add the static interference data to an original training set of the to-be-trained license plate recognition device, obtain a first training set, and train the first license plate recognition device based on the first training set.

[0067] The static interference data is a background image that is misdetected as a license plate area by the to-be-trained license plate recognition device in a scene without a vehicle.

[0068] The first identification module is configured to perform feature recognition on a vehicle based on a vehicle feature recognition submodule of the first license plate recognition device, obtain vehicle feature data, generate a ReID re-identification identification based on the vehicle feature data, and associate the ReID re-identification identification with license plate information to obtain vehicle comprehensive feature information.

[0069] The second identification module is configured to obtain dynamic interference data identified by the first license plate recognition device, add the dynamic interference data to the first training set to obtain a second training set, train the first license plate recognition device based on the second training set, and obtain a second license plate recognition device.

[0070] The dynamic interference data is a moving image of a vehicle scene that is misidentified as a license plate area by the first license plate recognition device.

[0071] The second verification module is configured to sequentially perform license plate recognition and secondary verification based on the second license plate recognition device and vehicle comprehensive feature information.

[0072] The advantages and beneficial effects of the present application relative to the prior art are as follows:

[0073] The license plate recognition method of the present application not only identifies the license plate information of the vehicle, but also extracts vehicle feature information, including vehicle image, vehicle color, vehicle brand, vehicle model, vehicle outline, sun visor, annual inspection mark, paper towel box, pendant, ornament, co-driver, safety belt, and other markers, and generates a ReID re-identification mark in combination with the vehicle feature information. The license plate recognition is combined with the vehicle comprehensive feature information for secondary verification, and the vehicle features are used as the basis to assist in improving the license plate recognition rate and greatly improving the efficiency of the toll station.

[0074] In the present application, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. If there is any inconsistency, the meaning described in the specification or derived from the content described in the specification shall prevail. In addition, the terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0075] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application and are not intended to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can still be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A license plate recognition method based on secondary verification of vehicle features, characterized in that: The following steps are involved: S1. Obtain static interference data recognized by the license plate recognition device to be trained, add the static interference data to an original training set of the license plate recognition device to be trained to obtain a first training set, and train the first license plate recognition device based on the first training set; The static interference data is the background image in the scene without a car that is mistakenly detected as the license plate area by the license plate recognition device to be trained; S2. Performing feature recognition on the vehicle based on the vehicle feature recognition submodule of the first license plate recognition device to obtain vehicle feature data, generating a ReID re-identification identifier based on the vehicle feature data, and associating the ReID re-identification identifier with the license plate information to obtain comprehensive vehicle feature information; S3. Acquire dynamic interference data recognized by the first license plate recognition device, add the dynamic interference data to the first training set to obtain a second training set, and train the first license plate recognition device based on the second training set to obtain a second license plate recognition device; The dynamic interference data is a motion image in the on-site vehicle scene that is mistakenly detected as a license plate area by the first license plate recognition device; S4. Perform license plate recognition and secondary verification in sequence based on the second license plate recognition device and the comprehensive vehicle feature information.

2. A license plate recognition method based on secondary verification of vehicle features according to claim 1, characterized in that: The vehicle characteristic data includes: characteristic data of vehicle appearance, body color, vehicle brand, vehicle model, vehicle outline, sun visor, annual inspection label, tissue box, pendant, ornament, co-pilot seat and seat belt.

3. A license plate recognition method based on secondary verification of vehicle features according to claim 2, characterized in that: In S1, training the first license plate recognition device based on the first training set includes: Use Haar features to characterize each positive sample in the first training set and obtain the positive sample Haar feature vector; Use Haar features to characterize each negative sample in the first training set and obtain the negative sample Haar feature vector; The positive sample Haar feature vector and the negative sample Haar feature vector are trained using the Adaboost algorithm to obtain the first license plate classifier; Among them, the positive sample is the image area containing the license plate, and the negative sample is the image area not containing the license plate.

4. A license plate recognition method based on secondary verification of vehicle features according to claim 3, characterized in that: In S2, the vehicle feature recognition submodule of the first license plate recognition device performs feature recognition on the vehicle to obtain vehicle feature data, and generating a ReID re-identification identifier based on the vehicle feature data specifically includes: S21. The vehicle feature recognition submodule of the first license plate recognition device obtains images captured by the camera of vehicles passing through the exit and entrance, extracts vehicle features from the captured images using a deep convolutional neural network, and generates a 128-dimensional feature vector based on the vehicle features; S22. Generate a 16-bit unique vehicle identifier based on a composite hash algorithm and a 128-dimensional feature vector, and confirm it as a ReID identifier for rapid retrieval and matching when performing secondary verification of the license plate.

5. A license plate recognition method based on secondary verification of vehicle features according to claim 4, characterized in that: In S3, obtaining a second training set, and training the first license plate recognition device based on the second training set includes: Adjustment of training parameters is obtained based on the second training set, and classifier performance of the first license plate recognition device is optimized based on the adjustment of the training parameters and in combination with a cross-validation algorithm.

6. A license plate recognition method based on secondary verification of vehicle features according to claim 5, characterized in that: In S4, performing license plate recognition and secondary verification in sequence based on the second license plate recognition device and the comprehensive vehicle feature information includes: S41, constructing a similarity calculation model using a cosine similarity algorithm and Euclidean distance, obtaining a second license plate recognition device generated based on the similarity calculation model, and training the second license plate recognition device using direction and distance parameters of the 128-dimensional feature vector; S42. Use the trained second license plate recognition device to perform a license plate recognition, and determine whether to perform a secondary verification based on the result of the first license plate recognition. Specifically: when the result of the first license plate recognition is consistent with the ReID re-identification identifier based on the preset system matching principle, it is confirmed as a valid license plate and allowed to pass; if the result of the first license plate recognition is inconsistent with the ReID re-identification identifier, it is not allowed to pass, and the second license plate recognition device performs a secondary verification based on the vehicle characteristics.

7. A license plate recognition system based on secondary verification of vehicle features, characterized in that: include: a data acquisition module, configured to acquire static interference data recognized by the license plate recognition device to be trained, add the static interference data to an original training set of the license plate recognition device to be trained to obtain a first training set, and train the first license plate recognition device based on the first training set; The static interference data is the background image in the scene without a car that is mistakenly detected as the license plate area by the license plate recognition device to be trained; a first recognition module, configured to perform feature recognition on the vehicle based on the vehicle feature recognition submodule of the first license plate recognition device to obtain vehicle feature data, generate a ReID re-identification identifier based on the vehicle feature data, and associate the ReID re-identification identifier with the license plate information to obtain comprehensive vehicle feature information; a second recognition module, configured to obtain dynamic interference data recognized by the first license plate recognition device, add the dynamic interference data to the first training set to obtain a second training set, and train the first license plate recognition device based on the second training set to obtain a second license plate recognition device; The dynamic interference data is a motion image in the on-site vehicle scene that is mistakenly detected as a license plate area by the first license plate recognition device; The secondary verification module is used to perform license plate recognition and secondary verification in sequence based on the second license plate recognition device and the vehicle comprehensive feature information.

Images