License plate recognition credibility judgment method based on multi-algorithm fusion

Abstract

The invention relates to the technical field of license plate recognition, and discloses a license plate recognition credibility judgment method based on multi-algorithm fusion, and the method comprises the following steps: S1, carrying out the image collection and recognition of a vehicle parked at a roadside parking space, and obtaining the license plate information and the vehicle information; s2, scoring and assigning the recognition result of the license plate according to the recognition condition of the vehicle; s3, the distance between the license plate and the camera is scored and assigned; and S4, adding the two scores, and judging whether the identified license plate result needs to be manually rechecked or is credible. A monocular distance measurement algorithm and a vehicle identification algorithm are introduced, a scoring system is added for an identified license plate result, and manual rechecking is introduced when the license plate score is lower than a threshold value; a non-license plate pattern is prevented from being mistakenly recognized as a license plate under the conditions of image distortion, insufficient light and complex shooting scene; therefore, data support is provided for a roadside parking space parking charging application scene, and generation of wrong charging of a parking lot is reduced.

Description

Technical Field

[0001] The invention relates to the technical field of license plate recognition, and in particular to a license plate recognition credibility judgment method based on multi-algorithm fusion. Background Art

[0002] As an effective supplement to other parking lots, roadside parking can provide a certain amount of parking spaces under limited resources; it is convenient for vehicles to enter and exit, and car owners can reduce the walking distance; it can make full use of urban space resources and make full and effective use of urban infrastructure; since roadside parking spaces do not have gates and railings, they usually use manual scanning of license plates and then charge, but this method is time-consuming and labor-intensive. With the development of technology, some cities have adopted the method of scanning and identifying license plates through cameras installed on patrol vehicles and charging; therefore, accurate acquisition of license plate information is an important guarantee for accurately pushing charging orders, reducing erroneous charges, and optimizing user experience.

[0003] Since license plates usually have the following characteristics: 1) a parallelogram area with a specific aspect ratio; 2) the characters and background have obvious contrast, and there are high-contrast edges on the grayscale image; 3) there are two groups of peaks formed by the background part and the character part on the grayscale histogram. Therefore, license plate recognition can usually be divided into four steps: image acquisition and preprocessing, license plate contour recognition and positioning, edge extraction and character segmentation, character recognition and license plate information combination; however, in addition to license plates, other patterns may also have the above characteristics; in the case of image distortion, insufficient light, and complex shooting scenes, the probability of misidentifying non-license plate patterns as license plates is greatly increased, resulting in the occurrence of incorrect charges. Summary of the invention

[0004] The purpose of the present invention is to provide a license plate recognition credibility judgment method based on multi-algorithm fusion to solve the problems raised in the above background technology.

[0005] In the application scenario of roadside parking fee collection, the location of the license plate usually meets the following rules:

[0006] 1. The distance between the license plate and the camera is usually within a fixed range, that is, the distance between the license plate and the camera is usually between 1.5 meters and 2 meters.

[0007] 2. According to the requirements of "Motor Vehicle License Plates of the People's Republic of China", the license plate should be installed on the license plate plate (frame) with the front facing outward and the characters in the positive direction; according to the requirements of GB 15741-1995 "Motor Vehicle and Trailer License Plate Plates (Frames) and Their Positions", the front license plate plate (frame) is located on the front bumper of the vehicle, and the prescribed reference height of the front bumper of the vehicle is 445mm; the lower edge of the license plate on the rear license plate plate (frame) should be no less than 0.30m from the ground, and the upper edge should generally be no more than 1.20m from the ground; therefore, the license plate always appears at the same time as the vehicle and usually does not appear on the upper part of the vehicle.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A license plate recognition credibility judgment method based on multi-algorithm fusion includes the following steps:

[0010] S1. Use the camera installed on the inspection vehicle to inspect the roadside parking spaces, collect images of vehicles parked in the roadside parking spaces and perform image recognition to obtain the license plate information and vehicle information in the images. If the license plate information is not recognized, terminate the algorithm process;

[0011] S2. If the license plate information is recognized, then determine whether the vehicle information is recognized, and assign a score to the license plate recognition result according to the recognition of the vehicle, and determine the positional relationship between the license plate and the vehicle;

[0012] S3. After determining the positional relationship between the license plate and the vehicle, a monocular ranging algorithm based on deep learning is used to obtain the distance between the license plate and the camera, and a score is assigned to the distance;

[0013] S4. Add the recognition result score of the license plate based on the vehicle recognition in step S2 and the score of the distance between the license plate and the camera in step S3. If the scores of the two are less than the set threshold, such as the threshold is denoted as SC and SC is set to 0.8, it is determined that the recognized license plate result needs manual review; if the scores of the two are not less than the set threshold, it is considered that the recognized license plate result is trustworthy and the algorithm process ends.

[0014] As a further solution of the present invention: in the step S2, the specific method of scoring the license plate recognition result according to the recognition situation of the vehicle is as follows:

[0015] S21, record the score of the license plate recognition result as SC1, if no vehicle is recognized, SC1 is defined as 0;

[0016] S22, if the vehicle information is recognized, the four corner coordinates and the center coordinates of the license plate rectangular detection frame and the vehicle rectangular detection frame are calibrated respectively, the overlap degree of the vehicle and license plate detection results is calculated by the coordinate position, and the intersection ratio is used for evaluation, that is, the intersection and union ratio of the vehicle rectangular detection frame and the license plate rectangular detection frame is calculated for evaluation, such as recording the intersection ratio as IoU; when the intersection ratio is less than the set threshold, such as recording the threshold as IoUS, and setting IoUS to 0.01; it is determined that the recognition results of the license plate and the vehicle are not related, and SC1 is defined as 0;

[0017] S23. When the intersection-union ratio is not less than the set threshold, the recognition results of the license plate and the vehicle are determined to match; and the maximum and minimum values ​​of the coordinates of the corner points in the license plate rectangular detection frame and the vehicle rectangular detection frame in the vertical direction are taken to calculate the score SC1 of the license plate recognition result.

[0018] As a further solution of the present invention: in the step S23, the calculation formula of the score SC1 of the license plate recognition result is as follows:

[0019]

[0020] In the above formula (1), e is a natural constant, y max and y min are the maximum and minimum values ​​of the vertical coordinates of the corner points in the license plate rectangular detection frame and the vehicle rectangular detection frame respectively; p is the value of the coordinate of the center point of the license plate rectangular detection frame in the vertical direction; among which, the coordinates of the four corner points of the license plate rectangular detection frame are defined as ((x p1 ,y p1 ),(x p2 ,y p2 ),(x p3 ,y p3 ),(x p4 ,y p4 )), the center point coordinates are defined as (x p ,y p ); the coordinates of the four corner points of the vehicle rectangular detection frame are defined as ((x c1 ,y c1 ),(x c2 ,y c2 ),(x c3 ,y c3 ),(x c4 ,y c4 )), the center point coordinates are defined as (x c ,y c ).

[0021] As a further solution of the present invention: in the step S3, the score of the distance between the license plate and the camera is recorded as SC2, and the calculation formula of the score SC2 of the distance between the license plate and the camera is as follows:

[0022]

[0023] In the above formula (2), D is the distance between the license plate and the camera, and e is a natural constant. Since the camera installed on the inspection vehicle is used to collect images of roadside parking spaces, the distance between the license plate and the camera is usually 1.5 to 2 meters, so D is taken as 1.5 to 2 meters.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] The present invention introduces a monocular ranging algorithm and a vehicle recognition algorithm to add a scoring system to the recognized license plate results, and introduces manual review when the license plate score is lower than a threshold; thereby avoiding the misidentification of non-license plate patterns as license plates in the case of image distortion, insufficient light, and complex shooting scenes; thereby providing data support for roadside parking fee application scenarios and reducing the occurrence of erroneous parking charges. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A flowchart of a license plate recognition credibility judgment method based on multi-algorithm fusion;

[0027] Figure 2 for Figure 1 A partial enlarged view of the

[0028] Figure 3 This is a structural schematic diagram of the license plate rectangular detection frame and the vehicle rectangular detection frame in a license plate recognition credibility judgment method based on multi-algorithm fusion. DETAILED DESCRIPTION

[0029] See also Figure 1-2 In an embodiment of the present invention, a license plate recognition credibility judgment method based on multi-algorithm fusion includes the following steps:

[0030] S1. Use the camera installed on the inspection vehicle to inspect the roadside parking spaces, collect images of vehicles parked in the roadside parking spaces and perform image recognition to obtain the license plate information and vehicle information in the images, such as Figure 3 As shown in the figure (in order to avoid privacy leakage, some numbers of the license plate are blurred), since the license plate information is identified, the next step is carried out;

[0031] S2. Score the license plate recognition result according to the vehicle recognition situation, and record the score as SC1. Since the vehicle has been recognized, the four corner coordinates of the license plate rectangular detection frame are calibrated to ((620,530), (847,530), (847,835), (620,835)), the center coordinates are (734,683), and the four corner coordinates of the vehicle rectangular detection frame are calibrated to ((572,30), (1887,30), (1887,1080) ),(572,1080)), the center coordinates are (1230,555), so the intersection over union ratio IoU is calculated to be 0.05, and the threshold IoUS is set to 0.01, and the union ratio is recorded as IoU>IoUS; then the recognition results of the license plate and the vehicle are determined to match; then the maximum value 1080 and the minimum value 30 of the corner point coordinates in the vertical direction of the license plate rectangular detection frame and the vehicle rectangular detection frame are taken, and the score SC1 of the license plate recognition result is calculated to be 0.488; the calculation process is as follows:

[0032]

[0033] S3. After determining the positional relationship between the license plate and the vehicle, the monocular ranging algorithm based on deep learning is used to obtain the distance between the license plate and the camera. The distance D between the license plate and the camera is 1.53m, and a score is assigned to the distance, and the score is recorded as SC2. SC2 = 0.498 is calculated. The calculation process is as follows:

[0034]

[0035] S4, add SC1 and SC2, the value is 0.987, set the threshold SC to 0.8, 0.987>0.8, then it is considered that the recognized license plate result is trustworthy, and the process ends.

[0036] What is described above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A license plate recognition credibility judgment method based on multi-algorithm fusion, characterized in that: The following steps are involved: S1. Use the camera installed on the inspection vehicle to inspect the roadside parking spaces, collect images of vehicles parked in the roadside parking spaces and perform image recognition to obtain the license plate information and vehicle information in the images. If the license plate information is not recognized, terminate the algorithm process; S2. If the license plate information is recognized, then determine whether the vehicle information is recognized, and assign a score to the license plate recognition result according to the recognition of the vehicle, and determine the positional relationship between the license plate and the vehicle; S3. After determining the positional relationship between the license plate and the vehicle, a monocular ranging algorithm based on deep learning is used to obtain the distance between the license plate and the camera, and a score is assigned to the distance; S4. Add the recognition result score of the license plate based on the vehicle recognition in step S2 and the score of the distance between the license plate and the camera in step S3, and set the threshold SC to 0.

8. If the scores of the two are less than the set threshold SC, it is determined that the recognized license plate result needs manual review; if the scores of the two are not less than the set threshold SC, it is considered that the recognized license plate result is trustworthy, and the algorithm process ends.

2. According to claim 1, a license plate recognition credibility judgment method based on multi-algorithm fusion is characterized in that: In the step S2, the specific method of scoring the license plate recognition result according to the vehicle recognition situation is as follows: S21, record the score of the license plate recognition result as SC1, if no vehicle is recognized, SC1 is defined as 0; S22, if the vehicle information is recognized, the four corner coordinates and the center coordinates of the license plate rectangular detection frame and the vehicle rectangular detection frame are calibrated respectively, the overlap degree of the vehicle and license plate detection results is calculated by the coordinate position, and the intersection over union ratio IoU is used for evaluation, that is, the ratio of the intersection and union of the vehicle rectangular detection frame and the license plate rectangular detection frame is calculated for evaluation, and the threshold IoUS is set to 0.01; when the intersection over union ratio IoU is less than the set threshold IoUS; it is determined that the recognition results of the license plate and the vehicle are not related, and SC1 is defined as 0; S23. When the intersection-over-union ratio IoU is not less than the set threshold IoUS, it is determined that the recognition results of the license plate and the vehicle match; then the maximum and minimum values ​​of the coordinates of the corner points in the license plate rectangular detection frame and the vehicle rectangular detection frame in the vertical direction are taken to calculate the score SC1 of the license plate recognition result.

3. The method for determining the credibility of license plate recognition based on multi-algorithm fusion according to claim 2 is characterized in that: In the step S23, the calculation formula of the score SC1 of the license plate recognition result is as follows: In the above formula (1), e is a natural constant, y max and y min are the maximum and minimum values ​​of the vertical coordinates of the corner points in the license plate rectangular detection frame and the vehicle rectangular detection frame respectively; p is the value of the coordinate of the center point of the license plate rectangular detection frame in the vertical direction; among which, the coordinates of the four corner points of the license plate rectangular detection frame are defined as ((x p1 ,y p1 ),(x p2 ,y p2 ),(x p3 ,y p3 ),(x p4 ,y p4 )), the center point coordinates are defined as (x p ,y p ); the coordinates of the four corner points of the vehicle rectangular detection frame are defined as ((x c1 ,y c1 ),(x c2 ,y c2 ),(x c3 ,y c3 ),(x c4 ,y c4 )), the center point coordinates are defined as (x c ,y c ).

4. The method for determining the credibility of license plate recognition based on multi-algorithm fusion according to claim 1 is characterized in that: In the step S3, the score of the distance between the license plate and the camera is recorded as SC2. The calculation formula of the score SC2 of the distance between the license plate and the camera is as follows: In the above formula (2), D is the distance between the license plate and the camera, and e is a natural constant.

Images