Method for realizing double-layer license plate recognition

Abstract

The invention provides a method for realizing double-layer license plate recognition. The method comprises the following steps: S1, separating a double-layer license plate; comprising the steps of determining a threshold S; separating double-layer license plates according to a threshold value; fixing the height H of the separated upper and lower single-layer license plates as an integer; s2, training the model; s3, adjusting the double-layer license plate; the method includes: determining proportions of upper and lower license plate characters; zooming the image; and S4, detecting the precision of the model. According to the method, the double-layer license plate is divided into two license plates according to a certain proportion to realize the recognition of the double-layer license plate, and the recognition problem of the double-layer license plate is perfectly solved on the premise of not influencing the recognition precision of the single-layer license plate.

Description

Technical Field

[0001] The invention belongs to the technical field of intelligent video processing, and in particular relates to a method for realizing double-layer license plate recognition in license plate recognition. Background Art

[0002] With the development of computer technology and the widespread application of computer vision principles, the method of using computer image processing technology to identify targets in real time has become more and more popular. my country's GDP is growing more and more rapidly, people's living standards are gradually improving, and motor vehicles have become one of the most important modes of transportation. Therefore, the license plate recognition system (License Plate Recognition System, LPRS) has been widely used in various automatic management systems such as intelligent transportation systems and intelligent monitoring systems. It also has a wide range of application value in intelligent park management and highway toll collection systems. The main task of license plate recognition is to be able to extract and recognize the moving car license plate from the complex background, which is one of the core problems in the field of computer vision. Because license plates have different colors and types, coupled with the interference of imaging, occlusion and other factors during movement, license plate recognition has always been one of the most challenging problems in the field of machine vision.

[0003] At present, the application of license plate recognition technology is becoming more and more widespread, and its related needs are also emerging. License plate recognition can be applied to equipment at the entrance of a residential area, such as automatic parking fee collection. When a vehicle enters the residential area, the license plate of the vehicle will be automatically recognized, and when the vehicle leaves the residential area, the fee will be automatically charged according to the parking time.

[0004] The existing methods for realizing double-layer license plate recognition are as follows: First, the histograms corresponding to different thresholds are used to determine the threshold range of the image histogram of the license plate position through a large number of statistical experiments, so as to segment the license plate according to the histogram corresponding to the specific threshold, and then use the pre-set standard character template for pattern matching to identify the characters of the license plate. Secondly, the license plate is located from the image using the binarization method, and then the character features of the license plate are extracted using the boundary tracking technology, and then the characters in the character library are compared using the statistics of the nearest neighbors, and then the adjacent classifier is used to obtain one or several license plate candidate codes, and the license plate number is determined after verifying and checking these numbers; in addition, the current network can only realize the recognition of single-layer license plates.

[0005] Existing methods for identifying double-layer license plates include using a proximity classifier to derive one or several candidate license plate codes, verifying and checking these numbers and then determining the license plate number. Using a classifier to complete the recognition of double-layer license plates can achieve the recognition of double-layer license plates, but the characters of license plates are diverse, and the license plates in the character library cannot contain all license plate numbers in the mainland, so some double-layer license plates cannot be perfectly recognized and correctly recognized; and these traditional image processing methods often cannot perfectly recognize double-layer license plates due to their own factors and different external factors; the current network can only recognize single-layer license plates, and there is currently no complete network that can realize the recognition of double-layer license plates. Summary of the invention

[0006] In order to solve the above problems, the purpose of this application is to propose a method for double-layer license plate recognition, which is mainly aimed at the fact that traditional image processing methods are difficult to perfectly recognize double-layer license plates in reality, such as the matching types between characters are too many, the character library cannot fully contain the information between all characters, and the current network cannot recognize double-layer license plates. These problems cannot be recognized by conventional methods. This method realizes the recognition of double-layer license plates by dividing the double-layer license plate into two license plates according to a certain ratio, and perfectly solves the recognition problem of double-layer license plates without affecting the recognition accuracy of single-layer license plates.

[0007] Specifically, the present invention provides a method for realizing double-layer license plate recognition, the method comprising the following steps:

[0008] S1. Separate double-layer license plate:

[0009] Determine the size of the threshold S;

[0010] Separate double-layer license plates according to thresholds;

[0011] The height H of the separated upper and lower single-layer license plates is fixed to an integer;

[0012] Further including:

[0013] The double-layer license plate has certain standardization, with two characters on the top and five characters on the bottom. Assuming that the image of the license plate is W wide and H high, the original license plate is divided into two layers, upper and lower, according to the ratio of H / 2.75, which is equivalent to two single-layer license plates. Since the coordinates in all images are based on pixels, all coordinate results must be integers;

[0014] If img is the original license plate image; the threshold is S; img1 is the upper license plate; img2 is the lower license plate, the specific calculation method is as follows:

[0015] img1=img[0:H / S,0:W];

[0016] img2 = img[H / S:H,0:W];

[0017] Among them, img[0:H / S,0:W] is the image from the coordinates (0,0) to the area (H / S, W) in the img image;

[0018] img[H / S:H,0:W] is to get the image from (H / S, 0) coordinates to (H, W) in the img image;

[0019] In this way, the double-layer license plate is divided into two single-layer license plates, which are then sent to the CRNN network for training. This can ensure the recognition accuracy of single-layer license plates and realize the recognition of double-layer license plates.

[0020] S2. Training model:

[0021] Since a double-layer license plate is added to the training set, that is, the double-layer license plate is divided into two single-layer license plate data sets, and the double-layer license plate contains the word "hang", which does not exist in ordinary license plates. Due to the addition of the word "hang", the label has changed from the original 74 words to 75 words, and the model needs to be retrained; that is, the word "hang" needs to be added to dict.txt, the double-layer license plate data set is placed in the image directory, the relative path of each image of the added double-layer license plate and the license plate number of each image are written to train.txt, and the CRNN network is used to retrain the model;

[0022] S3. Adjust the double-layer license plate: Step S3 should be performed after training the model in S2:

[0023] Specify the proportion of characters on the upper and lower license plates;

[0024] Scale the image;

[0025] Further including:

[0026] The distribution of double-layer license plates is two characters on the top and five characters on the bottom, while ordinary single-layer license plates have seven or eight characters. If the double-layer license plate is trained in the same way as the single-layer license plate, the character information occupied by the double-layer license plate is not as comprehensive as the single-layer license plate under the same width. Therefore, when testing the double-layer license plate, it is necessary to make some adjustments to the two separated single-layer license plates, that is, to make the information contained in it more comprehensive according to the proportion of characters in the image. The specific implementation is as follows:

[0027] Since the CRNN network structure has no fixed requirements for the width W of the input image, but the height H must be fixed, we only need to scale the width W according to the proportion of the characters in the image:

[0028] img1=cv2.resize(img1,(96,64));

[0029] img2=cv2.resize(img2,(144,64));

[0030] The 64 refers to the height H=64 of the input image of the training model; this makes the character information contained in the two single-layer license plates more comprehensive;

[0031] S4. Detection model accuracy: merge output.

[0032] In step S1, since the width W and height H of the license plate image in the data set are not fixed, (H / S) may not be an integer. The solution is as follows:

[0033] One is to round down or up (H / S) when separating the double-layer license plate:

[0034] math.ceil((H / S));

[0035] math.floor((H / S));

[0036] Among them, math.ceil() is rounded up, and math.floor() is rounded down;

[0037] Second, before separating the double-layer license plate, scale the width W and height H to a uniform size where (H / S) is an integer:

[0038] img=cv2.resize(img,(W,H));

[0039] Among them, cv2.resize() is used to scale the image, where W refers to the width of the image after scaling, and H refers to the height of the image after scaling.

[0040] In step S1, if img is the original license plate image, width W=220; H=64; let the threshold S=2.75; img1 is the upper license plate; img2 is the lower license plate, the specific calculation method is as follows:

[0041] img1=img[0:64 / 2.75,0:220];

[0042] img2=img[64 / 2.75:64,0:220].

[0043] The step S4, testing the trained model, comprises:

[0044] First, the recognition progress of the single-layer license plate needs to be tested. If the accuracy of the single-layer license plate decreases after adding the double-layer license plate, the data of the single-layer license plate or the double-layer license plate needs to be readjusted and the model needs to be retrained.

[0045] The test of the recognition accuracy of the double-layer license plate must be carried out under the condition that the model does not affect the recognition accuracy of the single-layer license plate. It is only necessary to test img1 and img2 according to step S3, and then determine that the recognition result of img1 must be two characters, and the recognition result of img2 must be five characters; finally, the recognition results of img1 and img2 are saved together as the final recognition result of the double-layer license plate.

[0046] The double-layer license plate includes China's standard license plate.

[0047] Therefore, the advantages of this application are: combining the idea of ​​having a certain proportional relationship between all double-layer license plates with the idea of ​​realizing double-layer license plate recognition, ingeniously dividing the double-layer license plate into two single-layer license plates without affecting the recognition accuracy of the original single-layer license plate solves the problem that conventional methods cannot realize double-layer license plate recognition. In particular:

[0048] 1. Divide the double-layer license plate into two single-layer license plates at the upper and lower layers according to the threshold value S = 2.75;

[0049] 2. During the test phase, the width W of the upper license plate is scaled to 96 and the width W of the lower license plate is scaled to 144 according to the ratio of the characters;

[0050] 3. It does not affect the recognition accuracy of ordinary single-layer license plates. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0052] Figure 1 is a schematic diagram of the process of this method.

[0053] Figure 2 This is a schematic diagram of the separation of double-layer license plates.

[0054] Figure 3 It is a schematic diagram of the process of testing a double-layer license plate according to an embodiment of the present application.

[0055] Figure 4 It is a schematic diagram of the network structure of CRNN. DETAILED DESCRIPTION

[0056] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0057] The method of realizing double-layer license plate recognition in this application is based on the fact that the current license plate types are single-layer and double-layer, and can be extended to the recognition of all double-layer license plates in China. A method for recognizing double-layer license plates is proposed, and the method is divided into four steps. The main implementation steps are as follows: Figure 1 As shown below:

[0058] Step S1. Separating double-layer license plates

[0059] Determine the size of the threshold S;

[0060] Separate double-layer license plates according to thresholds;

[0061] The height H of the separated upper and lower single-layer license plates is fixed to an integer;

[0062] Further including:

[0063] The double-layer license plate has certain standardization, with two characters on the top and five characters on the bottom. Assuming that the image of the license plate is W wide and H high, the original license plate is divided into two layers, upper and lower, according to the ratio of H / 2.75, which is equivalent to two single-layer license plates. Since the coordinates in all images are based on pixels, all coordinate results must be integers;

[0064] If img is the original license plate image; the threshold is S; img1 is the upper license plate; img2 is the lower license plate, the specific calculation method is as follows:

[0065] img1=img[0:H / S,0:W];

[0066] img2 = img[H / S:H,0:W];

[0067] Among them, img[0:H / S,0:W] is the image from the coordinates (0,0) to the area (H / S, W) in the img image;

[0068] img[H / S:H,0:W] is to get the image from (H / S, 0) coordinates to (H, W) in the img image;

[0069] In this way, the double-layer license plate is divided into two single-layer license plates, which are then sent to the CRNN network for training. The network structure of the CRNN is as follows: Figure 4 As shown, this can ensure the recognition accuracy of single-layer license plates and realize the recognition of double-layer license plates;

[0070] In particular, China's double-layer license plates have certain norms, such as Figure 2 As shown, if img is the original license plate image, width W = 220; H = 64; let the threshold S = 2.75; img1 is the upper license plate; img2 is the lower license plate, the specific calculation method is as follows:

[0071] img1=img[0:64 / 2.75,0:220];

[0072] img2=img[64 / 2.75:64,0:220].

[0073] Since the width W and height H of the license plate images in the dataset are often not fixed, (H / S) may not be an integer. The solution is as follows:

[0074] One is to round down or up the (H / S) when separating the double-layer license plate:

[0075] math.ceil((H / S))

[0076] math.floor((H / S))

[0077] Among them, math.ceil() is rounded up, and math.floor() is rounded down;

[0078] Second, before separating the double-layer license plate, the width W and height H can be scaled to a uniform size where (H / S) is an integer:

[0079] img = cv2.resize(img,(W,H))

[0080] Among them, cv2.resize() is used to scale the image, where W refers to the width of the image after scaling, and H refers to the height of the image after scaling.

[0081] In this way, the double-layer license plate can be divided into two single-layer license plates, and then sent to the CRNN network for training. This can not only ensure the recognition accuracy of single-layer license plates, but also realize the recognition of double-layer license plates.

[0082] Step S2. Training model

[0083] Since a double-layer license plate is added to the training set, that is, the double-layer license plate is divided into two single-layer license plate data sets, and the double-layer license plate contains the word "Gua", which does not exist in ordinary license plates, the previous pre-training model cannot be loaded during the training process. The previous pre-training model refers to the content in the prior art and does not belong to the part to be explained in this method. The pre-training model can only realize the recognition of single-layer license plates. Due to the addition of the word "Gua", the label is changed from the original 74 characters to 75 characters. The label refers to the specific characters that need to be recognized, and the commonly used ones include: "Jing", "Ji", "A", "3", "Gua", etc., and the model needs to be retrained; that is, the word "Gua" needs to be added to dict.txt, the double-layer license plate data set is placed in the image directory, the relative path of each image of the added double-layer license plate and the license plate number of each image are written to train.txt, and the model is retrained using the CRNN network;

[0084] Step S3. Adjust the double-layer license plate. Step S3 should be performed after the S2 training model:

[0085] Specify the proportion of characters on the upper and lower license plates;

[0086] Scale the image;

[0087] Further including:

[0088] The distribution of double-layer license plates is two characters on the top and five characters on the bottom, while ordinary single-layer license plates have seven or eight characters. If the double-layer license plate is trained in the same way as the single-layer license plate, the character information occupied by the double-layer license plate is not as comprehensive as the single-layer license plate under the same width. Therefore, when testing the double-layer license plate, it is necessary to make some adjustments to the two separated single-layer license plates (that is, according to the proportion of characters in the image, the information contained is more comprehensive). The specific implementation is as follows:

[0089] Since the CRNN network structure has no fixed requirements for the width W of the input image, but the height H must be fixed, it is only necessary to scale the width W according to the proportion of the characters in the image;

[0090] img1=cv2.resize(img1,(96,64));

[0091] img2=cv2.resize(img2,(144,64));

[0092] The 64 refers to the height H=64 of the input image of the training model, which is the height of the image as in step S1. In this way, the character information contained in the two single-layer license plates can be more comprehensive.

[0093] Step S4: Check model accuracy and merge outputs.

[0094] That is, testing the trained model includes:

[0095] First, the recognition progress of the single-layer license plate needs to be tested. If the accuracy of the single-layer license plate decreases after adding the double-layer license plate, the data of the single-layer license plate or the double-layer license plate needs to be readjusted and the model needs to be retrained.

[0096] The test of the recognition accuracy of the double-layer license plate must be carried out under the condition that the model does not affect the recognition accuracy of the single-layer license plate. It is only necessary to test img1 and img2 according to step S3, and then determine that the recognition result of img1 must be two characters, and the recognition result of img2 must be five characters; finally, saving the recognition results of img1 and img2 together is the final recognition result of the double-layer license plate.

[0097] In summary, if Figure 3As shown, the test process of the double-layer license plate is:

[0098] Start; double-layer license plate img; layering; upper single-layer license plate W=96, H=64, enter model training; lower single-layer license plate W=144, H=64, enter model training; upper and lower double layers merged output; end.

[0099] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for realizing double-layer license plate recognition, characterized in that: The method comprises the following steps: S1. Separating the double-layer license plate: Determine the threshold S; Separate double-layer license plates according to thresholds; The height H of the separated upper and lower single-layer license plates is fixed to an integer; Further including: The double-layer license plate has certain standardization, with two characters on the top and five characters on the bottom. Assuming that the image of the license plate is W wide and H high, the original license plate is divided into two layers, upper and lower, according to the ratio of H / 2.75, which is equivalent to two single-layer license plates. Since the coordinates in all images are based on pixels, all coordinate results must be integers; If img is the original license plate image; the threshold is S; img1 is the upper license plate; img2 is the lower license plate, the specific calculation method is as follows: img1=img[0:H / S,0:W]; img2 = img[H / S:H,0:W]; Among them, img[0:H / S,0:W] is the image from the coordinates (0,0) to the area (H / S, W) in the img image; img[H / S:H,0:W] is to get the image from (H / S, 0) coordinates to (H, W) in the img image; In this way, the double-layer license plate is divided into two single-layer license plates, which are then sent to the CRNN network for training. This can ensure the recognition accuracy of single-layer license plates and realize the recognition of double-layer license plates. S2. Training model: Since a double-layer license plate is added to the training set, that is, the double-layer license plate is divided into two single-layer license plate data sets, and the double-layer license plate contains the word "Gua", which does not exist in ordinary license plates. Due to the addition of the word "Gua", the label has changed from the original 74 words to 75 words, and the model needs to be retrained; that is, the word "Gua" needs to be added to dict.txt, the double-layer license plate data set is placed in the image directory, the relative path of each image of the added double-layer license plate and the license plate number of each image are written to train.txt, and the CRNN network is used to retrain the model; S3. Adjust the double-layer license plate: Step S3 should be performed after training the model in S2: Specify the proportion of characters on the upper and lower license plates; Scale the image; Further including: The distribution of double-layer license plates is two characters on the top and five characters on the bottom, while ordinary single-layer license plates have seven or eight characters. If the double-layer license plate is trained in the same way as the single-layer license plate, the character information occupied by the double-layer license plate is not as comprehensive as the single-layer license plate under the same width. Therefore, when testing the double-layer license plate, it is necessary to make some adjustments to the two separated single-layer license plates, that is, to make the information contained in it more comprehensive according to the proportion of characters in the image. The specific implementation is as follows: Since the CRNN network structure has no fixed requirements for the width W of the input image, but the height H must be fixed, we only need to scale the width W according to the proportion of the characters in the image: img1=cv2.resize(img1,(96,64)); img2=cv2.resize(img2,(144,64)); The 64 refers to the height H=64 of the input image of the training model; this makes the character information contained in the two single-layer license plates more comprehensive; S4. Detection model accuracy: merge output.

2. A method for realizing double-layer license plate recognition according to claim 1, characterized in that: In step S1, since the width W and height H of the license plate image in the data set are not fixed, (H / S) may not be an integer. The solution is as follows: First, when separating the double-layer license plate, (H / S) is rounded down or up: math.ceil((H / S)); math.floor((H / S)); Among them, math.ceil() is rounded up, and math.floor() is rounded down; Second, before separating the double-layer license plate, scale the width W and height H to a uniform size where (H / S) is an integer: img=cv2.resize(img,(W,H)); Among them, cv2.resize() is used to scale the image, where W refers to the width of the image after scaling, and H refers to the height of the image after scaling.

3. A method for realizing double-layer license plate recognition according to claim 1, characterized in that: In step S1, if img is the original license plate image, width W=220; H=64; let the threshold S=2.75; img1 is the upper license plate; img2 is the lower license plate, the specific calculation method is as follows: img1=img[0:64 / 2.75,0:220]; img2=img[64 / 2.75:64,0:220].

4. A method for realizing double-layer license plate recognition according to claim 1, characterized in that: The step S4, testing the trained model, comprises: First, the recognition progress of the single-layer license plate needs to be tested. If the accuracy of the single-layer license plate decreases after adding the double-layer license plate, the data of the single-layer license plate or the double-layer license plate needs to be readjusted and the model needs to be retrained. The recognition accuracy of the double-layer license plate must be tested under the condition that the model does not affect the recognition accuracy of the single-layer license plate. It is only necessary to test img1 and img2 according to step S3, and then determine that the recognition result of img1 must be two characters, and the recognition result of img2 must be five characters; Finally, the recognition results of img1 and img2 are saved together to obtain the final recognition result of the double-layer license plate.

5. A method for realizing double-layer license plate recognition according to claim 1, characterized in that: The double-layer license plate includes China's standard license plate.

Images