Vehicle identification system and method based on license plate recognition

Abstract

The present invention discloses a vehicle identification system and method based on license plate recognition, and the method steps are as follows: obtaining a front vehicle image to be detected; using a trained improved neural network model to extract license plate features to obtain a license plate feature frame; finding a corresponding vehicle suggestion frame according to the license plate feature frame; and matching the vehicle suggestion frame using a mean absolute difference algorithm to obtain a vehicle feature frame. The present invention uses mapping principles and feature points to find a vehicle suggestion frame and uses a mean absolute difference algorithm to match the vehicle suggestion frame to obtain a vehicle feature frame to improve the recognition accuracy of the front vehicle.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent transportation technology, and in particular relates to a vehicle identification system and method based on license plate recognition. Background Art

[0002] In recent years, with the rapid development of deep learning and computer hardware technology, target detection technology has made breakthrough progress in both accuracy and computing speed, and vehicle detection technology has also developed rapidly. Vehicle detection technology is based on the vehicle detection system to obtain images of the vehicle in front and then perform calculations and analysis to obtain the specific shape information of the vehicle in front, and formulate a driving plan based on the information of the vehicle in front and the current road conditions. Therefore, vehicle detection technology is indispensable in the process of vehicle driving.

[0003] The specific detection process of the currently commonly used vehicle detection technology is: obtain the image of the vehicle in front, analyze and calculate the overall vehicle image, and obtain the specific shape information of the vehicle in front. However, this existing vehicle detection technology has the following problems:

[0004] First, because the overall image size of the vehicle in front is large, the detection speed of the existing vehicle detection technology cannot meet the needs of vehicles driving on highways.

[0005] Secondly, when vehicles are driving at high speeds, the risk factor is relatively high, requiring high accuracy in detecting the vehicle ahead and detecting a large number of vehicles in a short period of time. As a result, this vehicle detection technology cannot meet the requirements for detection accuracy during high-speed driving. Summary of the invention

[0006] In view of the above-mentioned deficiencies in the prior art, an object of the present invention is to provide a vehicle identification system and method based on license plate recognition, so as to solve the problems of slow detection speed and poor detection accuracy of the existing vehicle detection technology during high-speed driving of the vehicle.

[0007] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] A vehicle identification system based on license plate recognition of the present invention comprises: an image detection device, a feature extraction device, an information sending device, an intelligent network connection platform, an information receiving device, and an image processing device;

[0009] The image detection device is used to detect the vehicle in front and transmit the image of the vehicle in front to the feature extraction device;

[0010] The feature extraction device is used to extract the license plate features of the front vehicle image, obtain a license plate feature frame, and transmit the license plate feature frame to the information sending device;

[0011] The information sending device is used to transmit the license plate feature frame to the intelligent network platform;

[0012] The intelligent network connection platform searches for a vehicle suggestion frame according to the license plate feature frame, and transmits the vehicle suggestion frame to the information receiving device;

[0013] The information receiving device is used to transmit the vehicle suggestion frame to the image processing device;

[0014] The image processing device obtains a vehicle feature frame according to the vehicle suggestion frame.

[0015] Furthermore, the feature extraction device stores a trained improved neural network model for extracting a license plate feature frame of an image.

[0016] Furthermore, the neural network model is a target detection model with an anchor box.

[0017] Furthermore, the feature extraction device uses a K-means clustering algorithm to cluster the constructed license plate data set to generate a license plate anchor frame suitable for the license plate data set, and uses the generated license plate anchor frame to replace the vehicle anchor frame in the neural network model to obtain an improved neural network model.

[0018] Furthermore, the license plate data set x constructed by using the K-means clustering algorithm i The specific steps of generating a license plate anchor frame suitable for the license plate dataset through clustering are:

[0019] (11) Divide the license plate dataset into n categories (determined according to the number of common license plate sizes on the road section, n ≥ 2);

[0020] (12) Randomly generate n cluster centers y i ;

[0021] (13) Calculate the distance from each point to the cluster center and cluster it to the cluster center closest to the point;

[0022] (14) Calculate the coordinate average of all points in each cluster, and use the obtained coordinate average as the new cluster center;

[0023] (15) Repeat steps (13) and (14) until the new cluster center no longer changes. The final new cluster center z i This is the license plate anchor frame.

[0024] Furthermore, the improved neural network model is trained by a large number of vehicle image data sets, and the trained improved neural network model can obtain a license plate feature frame for an input vehicle image.

[0025] Furthermore, the license plate anchor frame locates the target feature when extracting features to obtain a license plate feature frame.

[0026] Furthermore, the license plate feature frame contains a complete and clear license plate image.

[0027] Furthermore, the intelligent network connection platform includes: an intelligent storage platform and an intelligent computing platform;

[0028] An intelligent storage platform for storing the license plate feature frames of all vehicles on the current vehicle driving road and the vehicle suggestion frames matching the license plate feature frames;

[0029] The intelligent computing platform matches the corresponding vehicle suggestion frame in the intelligent storage platform according to the license plate feature frame transmitted by the information sending device, and transmits the matched vehicle suggestion frame to the information receiving device.

[0030] Furthermore, the vehicle suggestion frame and the vehicle feature frame include a complete and clear vehicle image and vehicle size.

[0031] Furthermore, the sizes of the vehicle suggestion boxes are all the same, the size of the largest vehicle suggestion box is M×N, and the other vehicle suggestion boxes are increased with gray bars to be adjusted to a size of M×N.

[0032] Furthermore, the image processing device performs image matching on the vehicle suggestion frame to obtain a vehicle feature frame.

[0033] Furthermore, the matching process includes: randomly naming n vehicle suggestion boxes as {S1, S2, ..., Sn}, matching the vehicle suggestion boxes in pairs to obtain a matching average error, and putting the obtained matching average error into a matrix D,

[0034]

[0035] The matching process is carried out from left to right row by row in matrix D. If there is SiSj When it is less than the maximum matching error A, the vehicle proposal boxes Si and Sj are used as the final vehicle feature boxes and the matching process is stopped. SiSj Represents the vehicle suggestion frame obtained by matching the vehicle suggestion frames Si and Sj; if all the matching average errors in the matrix D are greater than the maximum matching error A, the vehicle suggestion frame with the largest vehicle size among the n vehicle suggestion frames is taken as the final vehicle feature frame.

[0036] A vehicle identification method based on license plate recognition of the present invention comprises the following steps:

[0037] 1) Obtain the image of the vehicle ahead to be detected;

[0038] 2) Using the trained improved neural network model to extract license plate features and obtain a license plate feature frame;

[0039] 3) Find the corresponding vehicle suggestion frame based on the license plate feature frame;

[0040] 4) Use the mean absolute difference algorithm to match the vehicle suggestion frame to obtain the vehicle feature frame.

[0041] Furthermore, the acquisition of the image of the vehicle to be detected ahead is completed by a vehicle-mounted front camera.

[0042] Furthermore, the feature extraction process includes multiple downsampling and positioning using a vehicle anchor frame.

[0043] Furthermore, the searching for the corresponding vehicle suggestion frame according to the license plate feature frame is performed based on the feature points of the license plate, where the feature points are the license plate number in the license plate. If the feature points of the license plate in the vehicle suggestion frame correspond to the feature points in the license plate feature frame, then the vehicle suggestion frame is the sought vehicle suggestion frame.

[0044] Furthermore, the license plate feature frame and the vehicle suggestion frame are a pair of mappings, that is, one license plate feature frame corresponds to only one vehicle suggestion frame, and one vehicle suggestion frame may correspond to multiple license plate feature frames.

[0045] Furthermore, the step 4) specifically includes: randomly naming the n vehicle suggestion boxes as {S1, S2, ..., Sn}, matching the vehicle suggestion boxes in pairs to obtain a matching average error, and putting the obtained matching average error into a matrix D,

[0046]

[0047] The matching process is carried out from left to right row by row in matrix D. If there is SiSj When it is less than the maximum matching error A, the vehicle proposal boxes Si and Sj are used as the final vehicle feature boxes and the matching process is stopped. SiSj Represents the vehicle suggestion frame obtained by matching the vehicle suggestion frames Si and Sj; if all the matching average errors in the matrix D are greater than the maximum matching error A, the vehicle suggestion frame with the largest vehicle size among the n vehicle suggestion frames is taken as the final vehicle feature frame.

[0048] Furthermore, the vehicle suggestion box is matched by the mean absolute difference algorithm to obtain the matching average error:

[0049]

[0050] Where Si and Sj are vehicle suggestion boxes, M×N is the size of the vehicle suggestion box, the lower left corner of the vehicle suggestion box is taken as the origin, x is the horizontal coordinate, and y is the vertical coordinate.

[0051] Furthermore, the maximum matching error A is matched to all stored vehicle suggestion frames corresponding to the same license plate feature frames by using a mean absolute difference algorithm, and the maximum matching average error is used as the maximum matching error A.

[0052] Beneficial effects of the present invention:

[0053] The present invention extracts features from the license plate of the vehicle in front to obtain a license plate feature frame, then searches for a corresponding vehicle suggestion frame based on the license plate feature frame, and matches the vehicle suggestion frame to obtain a vehicle feature frame; in this method, a neural network model is used to extract features from the license plate of the vehicle in front to obtain a license plate feature frame, and the recognition speed of the vehicle in front is improved by improving the anchor frame in the neural network model; at the same time, the mapping principle and feature points are used to search for the vehicle suggestion frame, and the mean absolute difference algorithm is used to match the vehicle suggestion frame to obtain the vehicle feature frame to improve the recognition accuracy of the vehicle in front. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A schematic structural diagram of a vehicle identification device based on license plate recognition provided in a preferred embodiment of the present invention.

[0055] Figure 2 A schematic diagram of a vehicle suggestion box provided in a preferred embodiment of the present invention.

[0056] Figure 3 A flow chart of a vehicle identification method based on license plate recognition provided in a preferred embodiment of the present invention.

[0057] Figure 4 A schematic diagram of a license plate anchoring frame provided in a preferred embodiment of the present invention.

[0058] Figure 5 A schematic diagram of a flow chart of searching a vehicle suggestion box provided in a preferred embodiment of the present invention.

[0059] Figure 6 A schematic diagram of a vehicle suggestion box matching process provided by a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to facilitate the understanding of those skilled in the art, the present invention is further described below in conjunction with embodiments and drawings. The contents mentioned in the implementation modes are not intended to limit the present invention.

[0061] Reference Figure 1 As shown, a vehicle identification system based on license plate recognition of the present invention includes: an image detection device 101, a feature extraction device 102, an information sending device 103, an intelligent network platform 104, an information receiving device 107, and an image processing device 108;

[0062] The image detection device is used to detect the vehicle in front and transmit the image of the vehicle in front to the feature extraction device;

[0063] The feature extraction device is used to extract the license plate features of the front vehicle image, obtain a license plate feature frame, and transmit the license plate feature frame to the information sending device;

[0064] The information sending device is used to transmit the license plate feature frame to the intelligent network platform;

[0065] The intelligent network connection platform searches for a vehicle suggestion frame according to the license plate feature frame, and transmits the vehicle suggestion frame to the information receiving device;

[0066] The information receiving device is used to transmit the vehicle suggestion frame to the image processing device;

[0067] The image processing device obtains a vehicle feature frame according to the vehicle suggestion frame.

[0068] Wherein, the feature extraction device stores a trained improved neural network model for extracting the license plate feature frame of the image.

[0069] Among them, the neural network model is a target detection model with an anchor box.

[0070] Among them, the feature extraction device uses the K-means clustering algorithm to cluster the constructed license plate data set to generate a license plate anchor frame suitable for the license plate data set, and uses the generated license plate anchor frame to replace the vehicle anchor frame in the neural network model to obtain an improved neural network model.

[0071] Specifically, the K-means clustering algorithm is used to construct the license plate dataset x i The specific steps of generating a license plate anchor frame suitable for the license plate dataset through clustering are:

[0072] (11) Divide the license plate dataset into n categories (determined according to the number of common license plate sizes on the road section, n ≥ 2);

[0073] (12) Randomly generate n cluster centers y i ;

[0074] (13) Calculate the distance from each point to the cluster center and cluster it to the cluster center closest to the point;

[0075] (14) Calculate the coordinate average of all points in each cluster, and use the obtained coordinate average as the new cluster center;

[0076] (15) Repeat steps (13) and (14) until the new cluster center no longer changes. The final new cluster center zi This is the license plate anchor frame.

[0077] In addition, the improved neural network model is trained by a large number of vehicle image data sets, and the trained improved neural network model can obtain a license plate feature frame for an input vehicle image.

[0078] The license plate anchor frame locates the target feature when extracting features to obtain a license plate feature frame.

[0079] The license plate feature frame contains a complete and clear license plate image.

[0080] The intelligent network connection platform includes: an intelligent storage platform and an intelligent computing platform;

[0081] The intelligent storage platform 106 stores the license plate feature frames of all vehicles on the current vehicle driving road and the vehicle suggestion frames matching the license plate feature frames;

[0082] The intelligent computing platform 105 matches the corresponding vehicle suggestion frame in the intelligent storage platform according to the license plate feature frame transmitted by the information sending device, and transmits the matched vehicle suggestion frame to the information receiving device.

[0083] The vehicle suggestion frame and the vehicle feature frame include a complete and clear vehicle image and vehicle size.

[0084] The sizes of the vehicle suggestion boxes are all the same. The largest vehicle suggestion box is M×N in size. The other vehicle suggestion boxes are adjusted to M×N in size by adding gray bars. Figure 2 shown.

[0085] The image processing device performs image matching on the vehicle suggestion frame to obtain a vehicle feature frame.

[0086] In a preferred example, the matching process includes: randomly naming n vehicle suggestion boxes {S1, S2, ..., Sn}, matching the vehicle suggestion boxes in pairs to obtain a matching average error, and putting the obtained matching average error into a matrix D,

[0087] The matching process is carried out from left to right row by row in matrix D. If there is SiSj When it is less than the maximum matching error A, the vehicle proposal boxes Si and Sj are used as the final vehicle feature boxes and the matching process is stopped. SiSj Represents the vehicle suggestion frame obtained by matching the vehicle suggestion frames Si and Sj; if all the matching average errors in the matrix D are greater than the maximum matching error A, the vehicle suggestion frame with the largest vehicle size among the n vehicle suggestion frames is taken as the final vehicle feature frame.

[0088] Reference Figure 3 As shown, a vehicle identification method based on license plate recognition of the present invention comprises the following steps:

[0089] 1) Obtain the image of the vehicle to be detected in front through the on-board front camera;

[0090] 2) Using the trained improved neural network model to extract license plate features and obtain a license plate feature frame; the feature extraction process includes multiple downsampling and positioning using a vehicle anchor frame;

[0091] The constructed license plate data set is clustered using a K-means clustering algorithm to generate a license plate anchor frame suitable for the license plate data set, and the generated license plate anchor frame is used to replace the vehicle anchor frame in the neural network model to obtain an improved neural network model;

[0092] In the preferred example, refer to Figure 4 As shown, the license plate data set x constructed using the K-means clustering algorithm i The specific steps of generating a license plate anchor frame suitable for the license plate dataset through clustering are:

[0093] (11) Divide the license plate dataset into 3 categories (there are 3 common license plate sizes);

[0094] (12) Randomly generate three cluster centers y i ;

[0095] (13) Calculate the distance from each point to the cluster center and cluster it to the cluster center closest to the point;

[0096] (14) Calculate the coordinate average of all points in each cluster, and use the obtained coordinate average as the new cluster center;

[0097] (15) Repeat steps (13) and (14) until the new cluster center no longer changes. The final new cluster center z i This is the license plate anchor frame.

[0098] 3) Find the corresponding vehicle suggestion frame based on the license plate feature frame, refer to Figure 5 As shown;

[0099] The searching for the corresponding vehicle suggestion frame according to the license plate feature frame is performed based on the feature points of the license plate, where the feature points are the license plate number in the license plate. If the feature points of the license plate in the vehicle suggestion frame correspond to the feature points in the license plate feature frame, then the vehicle suggestion frame is the vehicle suggestion frame being searched for;

[0100] The license plate feature frame and the vehicle suggestion frame are a pair of mappings, that is, one license plate feature frame corresponds to only one vehicle suggestion frame, and one vehicle suggestion frame can correspond to multiple license plate feature frames; license plate feature frames 2, 3, and 4 correspond to vehicle suggestion frames 1, 5, and 3 respectively, and license plate feature frames 1 and 5 correspond to the same vehicle suggestion frame, which is vehicle suggestion frame 2.

[0101] 4) Use the mean absolute difference algorithm to match the vehicle suggestion frame to obtain the vehicle feature frame;

[0102] Specifically, the n vehicle suggestion boxes are randomly named {S1, S2, ..., Sn}, the vehicle suggestion boxes are matched in pairs to obtain the matching average error, and the obtained matching average error is put into the matrix D.

[0103]

[0104] The matching process is carried out from left to right row by row in matrix D. If there is SiSj When it is less than the maximum matching error A, the vehicle proposal boxes Si and Sj are used as the final vehicle feature boxes and the matching process is stopped. SiSj Represents the vehicle suggestion frame obtained by matching the vehicle suggestion frames Si and Sj; if all the matching average errors in the matrix D are greater than the maximum matching error A, the vehicle suggestion frame with the largest vehicle size among the n vehicle suggestion frames is taken as the final vehicle feature frame.

[0105] The vehicle suggestion box is matched by the mean absolute difference algorithm to obtain the matching average error:

[0106]

[0107] Where Si and Si are vehicle suggestion boxes, M×N is the size of the vehicle suggestion box, the lower left corner of the vehicle suggestion box is taken as the origin, x is the horizontal coordinate, and y is the vertical coordinate.

[0108] In the preferred example, refer to Figure 6 As shown, the matching process includes: randomly naming the three vehicle suggestion boxes {S1, S2, S3}, matching the vehicle suggestion boxes S1 and S2 to obtain D S1S2 , if D S1S2 <A, then the vehicle suggestion frames S1 and S2 are used as vehicle feature frames, otherwise the vehicle suggestion frames S1 and S3 are matched to obtain D S1S3 , if D S1S3 <A, then the vehicle suggestion frames S1 and S3 are used as vehicle feature frames, otherwise the vehicle suggestion frames S2 and S3 are matched to obtain D S2S3 , if D S2S3<A, then the vehicle suggestion frames S2 and S3 are taken as vehicle feature frames, otherwise the vehicle suggestion frame with the largest vehicle size among the three vehicle suggestion frames is taken as the final vehicle feature frame.

[0109] The maximum matching error A is used to match all stored vehicle suggestion frames corresponding to the same license plate feature frames through a mean absolute difference algorithm, and the maximum matching average error is used as the maximum matching error A.

[0110] The present invention has many specific application paths. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements can be made without departing from the principle of the present invention. These improvements should also be regarded as the protection scope of the present invention.

Claims

1. A vehicle identification system based on license plate recognition, characterized in that: include: Image detection device, feature extraction device, information sending device, intelligent network platform, information receiving device, image processing device; The image detection device is used to detect the vehicle in front and transmit the image of the vehicle in front to the feature extraction device; The feature extraction device is used to extract the license plate features of the front vehicle image, obtain a license plate feature frame, and transmit the license plate feature frame to the information sending device; The information sending device is used to transmit the license plate feature frame to the intelligent network platform; The intelligent network connection platform searches for a vehicle suggestion frame according to the license plate feature frame, and transmits the vehicle suggestion frame to the information receiving device; The information receiving device is used to transmit the vehicle suggestion frame to the image processing device; The image processing device obtains a vehicle feature frame according to the vehicle suggestion frame; The feature extraction device uses a K-means clustering algorithm to cluster the constructed license plate data set to generate a license plate anchor frame suitable for the license plate data set, and uses the generated license plate anchor frame to replace the vehicle anchor frame in the neural network model to obtain an improved neural network model; The intelligent network connection platform includes: an intelligent storage platform and an intelligent computing platform; An intelligent storage platform for storing the license plate feature frames of all vehicles on the current vehicle driving road and the vehicle suggestion frames matching the license plate feature frames; An intelligent computing platform matches a corresponding vehicle suggestion frame in an intelligent storage platform according to the license plate feature frame transmitted by the information sending device, and transmits the matched vehicle suggestion frame to the information receiving device; The matching process includes: randomly naming n vehicle suggestion boxes as {S1, S2, ..., Sn}, matching the vehicle suggestion boxes in pairs to obtain a matching average error, and putting the obtained matching average error into a matrix D, The matching process is carried out from left to right row by row in matrix D. If there is SiSj When it is less than the maximum matching error A, the vehicle proposal boxes Si and Sj are used as the final vehicle feature boxes and the matching process is stopped. SiSj represents the vehicle suggestion frame obtained by matching the vehicle suggestion frames Si and Sj; if all the matching average errors in the matrix D are greater than the matching maximum error A, the vehicle suggestion frame with the largest vehicle size among the n vehicle suggestion frames is taken as the final vehicle feature frame; The vehicle suggestion box is matched by the mean absolute difference algorithm to obtain the matching average error: Where Si and Sj are vehicle suggestion boxes, M×N is the size of the vehicle suggestion box, the lower left corner of the vehicle suggestion box is taken as the origin, x is the horizontal coordinate, and y is the vertical coordinate.

2. The vehicle identification system based on license plate recognition according to claim 1 is characterized in that: The license plate data set x constructed using the K-means clustering algorithm i The specific steps of generating a license plate anchor frame suitable for the license plate dataset through clustering are: (11) Divide the license plate dataset into n categories; (12) Randomly generate n cluster centers y i ; (13) Calculate the distance from each point to the cluster center and cluster it to the cluster center closest to the point; (14) Calculate the coordinate average of all points in each cluster, and use the obtained coordinate average as the new cluster center; (15) Repeat steps (13) and (14) until the new cluster center no longer changes. The final new cluster center z i This is the license plate anchor frame.

3. The vehicle identification system based on license plate recognition according to claim 1 is characterized in that: The improved neural network model is trained by a large number of vehicle image data sets, and the trained improved neural network model can obtain a license plate feature frame for an input vehicle image.

4. A vehicle identification method based on license plate recognition, based on the system of claim 1, characterized in that: Here are the steps: 1) Obtain the image of the vehicle ahead to be detected; 2) Using the trained improved neural network model to extract license plate features and obtain a license plate feature frame; 3) Find the corresponding vehicle suggestion frame based on the license plate feature frame; 4) Use the mean absolute difference algorithm to match the vehicle suggestion frame to obtain the vehicle feature frame.

5. The vehicle identification method based on license plate recognition according to claim 4 is characterized in that: The searching for the corresponding vehicle suggestion frame according to the license plate feature frame is performed based on the feature points of the license plate, where the feature points are the license plate number in the license plate. If the feature points of the license plate in the vehicle suggestion frame correspond to the feature points in the license plate feature frame, then the vehicle suggestion frame is the sought vehicle suggestion frame.

6. The vehicle identification method based on license plate recognition according to claim 5 is characterized in that: The license plate feature frame and the vehicle suggestion frame form a pair of mappings, that is, one license plate feature frame corresponds to only one vehicle suggestion frame, and one vehicle suggestion frame may correspond to multiple license plate feature frames.

Images