License Plate Recognition Method, Device, Electronic Device, and Storage Medium

Abstract

The present disclosure provides a license plate recognition method, apparatus, electronic device, and storage medium, which relate to fields such as deep learning, computer vision, and intelligent transportation. The specific implementation solution is as follows: Obtain multiple frames of license plate images of a target vehicle; input each frame of license plate image into a license plate recognition model for license plate recognition to obtain labels corresponding to the positions of each character in each frame of license plate image; according to the labels corresponding to the positions of each character in each frame of license plate image, filter out license plate images containing blurred labels and / or occluded labels from the multiple frames of license plate images to obtain remaining candidate license plate images; recognize the license plate of the target vehicle according to the candidate license plate images. Thus, invalid license plates such as blurred and / or occluded ones can be effectively filtered out, and license plate images with better recognition effects are recognized, improving the recognition accuracy of license plates.

Description

Technical Field

[0001] The present disclosure relates to the field of AI (Artificial Intelligence), specifically to fields such as deep learning, computer vision, and intelligent transportation, and particularly relates to a license plate recognition method, device, electronic device, and storage medium. Background Art

[0002] In fields such as on-street parking and intelligent transportation, it is necessary to accurately recognize the license plate number of a vehicle. However, limited by the shooting environment, the quality of the collected license plate images may vary. When the quality of the license plate image is poor, it may lead to inaccurate recognition of the license plate number, and in severe cases, the license plate number may not be recognized. For example, when the light is very poor, it is easy to cause the imaging of the license plate to be blurred and the license plate to be blocked by passing pedestrians, leaves, and other vehicles, resulting in the license plate number not being recognized. Summary of the Invention

[0003] The present disclosure provides a license plate recognition method, device, electronic device, and storage medium.

[0004] According to one aspect of the present disclosure, a license plate recognition method is provided, including: obtaining multiple frames of license plate images of a target vehicle; inputting each frame of the license plate image into a license plate recognition model for license plate recognition to obtain labels corresponding to the positions of each character in each frame of the license plate image; screening out license plate images containing blurred labels and / or occluded labels from the multiple frames of license plate images according to the labels corresponding to the positions of each character in each frame of the license plate image to obtain remaining candidate license plate images; and recognizing the license plate of the target vehicle according to the candidate license plate images.

[0005] According to another aspect of the present disclosure, a license plate recognition device is provided, including: a first acquisition module for obtaining multiple frames of license plate images of a target vehicle; a first recognition module for inputting each frame of the license plate image into a license plate recognition model for license plate recognition to obtain labels corresponding to the positions of each character in each frame of the license plate image; a screening module for screening out license plate images containing blurred labels and / or occluded labels from the multiple frames of license plate images according to the labels corresponding to the positions of each character in each frame of the license plate image to obtain remaining candidate license plate images; and a second recognition module for recognizing the license plate of the target vehicle according to the candidate license plate images.

[0006] According to yet another aspect of the present disclosure, an electronic device is provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the license plate recognition method proposed in the above aspect of the present disclosure.

[0007] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the license plate recognition method proposed in the above-mentioned aspect of the present disclosure.

[0008] According to still another aspect of the present disclosure, there is provided a computer program product including a computer program which, when executed by a processor, implements the steps of the license plate recognition method proposed in the above-mentioned aspect of the present disclosure.

[0009] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understandable through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0011] Figure 1 is a schematic flowchart of the license plate recognition method provided in Embodiment 1 of the present disclosure;

[0012] Figure 2 is a schematic flowchart of the license plate recognition method provided in Embodiment 2 of the present disclosure;

[0013] Figure 3 is a schematic flowchart of the license plate recognition method provided in Embodiment 3 of the present disclosure;

[0014] Figure 4 is a schematic flowchart of the license plate recognition method provided in Embodiment 4 of the present disclosure;

[0015] Figure 5 is a schematic flowchart of the license plate recognition method provided in Embodiment 5 of the present disclosure;

[0016] Figure 6 is a schematic structural diagram of the license plate recognition device provided in Embodiment 6 of the present disclosure;

[0017] Figure 7 shows a schematic block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to help understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0019] Currently, the average score of license plate images can be calculated, and the license plate image with the highest average score can be selected for license plate number recognition. However, simply recognizing the license plate image with the highest average score results in a large amount of invalid license plate data and low license plate recognition accuracy.

[0020] Therefore, in view of the above problems, the present disclosure proposes a license plate recognition method, apparatus, electronic device, and storage medium.

[0021] The following describes the license plate recognition method, apparatus, electronic device, and storage medium according to the embodiments of the present disclosure with reference to the accompanying drawings.

[0022] Figure 1 It is a schematic flowchart of the license plate recognition method provided by Embodiment 1 of the present disclosure.

[0023] In the embodiments of the present disclosure, it is exemplified that the license plate recognition method is configured in a license plate recognition apparatus, and the license plate recognition apparatus can be applied to any electronic device so that the electronic device can perform the license plate recognition function.

[0024] Among them, the electronic device can be any device with computing capabilities. For example, it can be a personal computer (PC for short), a mobile terminal, etc. The mobile terminal can be a hardware device such as a mobile phone, a tablet computer, a personal digital assistant, a wearable device, etc. with various operating systems, touch screens, and / or display screens.

[0025] As Figure 1 shown, the license plate recognition method may include the following steps:

[0026] Step 101, obtain multiple frames of license plate images of the target vehicle.

[0027] In the embodiments of the present disclosure, the target vehicle can be any vehicle that needs to perform license plate recognition.

[0028] In the embodiments of the present disclosure, the license plate of the target vehicle can be image-captured by an image acquisition device to obtain multiple frames of license plate images of the target vehicle. Among them, the number of image acquisition devices can be one or more, and the present disclosure does not make specific limitations.

[0029] As an example, it is exemplarily described that the license plate recognition method is applied to a parking system. The target vehicle can be a vehicle driving into or out of a parking lot, and the target vehicle can be captured by an image acquisition device in the parking system of the parking lot to obtain multiple frames of license plate images of the target vehicle.

[0030] Step 102, input each frame of license plate image into a license plate recognition model for license plate recognition to obtain the labels corresponding to the positions of each character in each frame of license plate image.

[0031] In the embodiments of the present disclosure, the license plate recognition model is a trained model. After training, the license plate recognition model has learned the correspondence between the positions of each character in the license plate image and the labels. Among them, the labels may include blank labels and / or blurred labels and / or character labels. The character labels can be used to indicate that there is a character at the corresponding character position in the image. The blank labels are used to indicate that the corresponding character position in the image is blank or occluded. The blurred labels represent that the corresponding character position is blurred.

[0032] For example, when the license plate in the image is clear and there are no passing pedestrians, leaves, or other vehicles blocking the license plate, the labels output by the license plate recognition model may all be character labels. When the image is blurred or there are passing pedestrians, leaves, or other vehicles blocking the license plate, the labels output by the license plate recognition model may include blurred labels and / or blank labels.

[0033] It should be understood that since the license plate character lengths of different target vehicles are different, for license plate images with different license plate character lengths, the number of blank labels corresponding to the character positions in the license plate images that the license plate recognition model allows to output is also different. For example, the license plate character length in the license plate image of target vehicle 1 is 8 bits, the set output length of the license plate recognition model is 8 bits, and the number of blank labels corresponding to the character positions in the license plate image that the license plate recognition model allows to output is 0. When the license plate recognition model outputs a blank label, it can be determined that the blank label is an occlusion label. Another example is that the license plate character length in the license plate image of target vehicle 2 is 7 bits, the set output length of the license plate recognition model is 8 bits, and the number of blank labels corresponding to the character positions in the license plate image that the license plate recognition model allows to output is 1. When the license plate recognition model outputs 2 or more blank labels, 1 of the blank labels can be determined as a blank label according to the order of the blank labels determined during the training of the license plate recognition model, and the remaining blank labels are occlusion labels. Therefore, when the number of blank labels output by the license plate recognition model is greater than the number of blank labels allowed to be output, the blank label can be corrected to determine the occlusion label corresponding to each character position in the license plate image.

[0034] In the embodiments of the present disclosure, a license plate recognition model can be used to perform license plate recognition on each frame of license plate image to obtain labels corresponding to the positions of each character in each frame of license plate image. It can be understood that when the labels corresponding to the positions of each character are all character labels, the license plate of the target vehicle can be directly determined according to the character labels corresponding to the positions of each character. However, when there are blurred labels and / or occluded labels among the labels corresponding to multiple character positions in the license plate image, since the characters at the target character positions corresponding to the blurred labels and / or occluded labels cannot be recognized, the license plate of the target vehicle cannot be determined. Therefore, in the embodiments of the present disclosure, in order to accurately recognize the license plate of the target vehicle, when there are blurred labels and / or occluded labels among the labels corresponding to multiple character positions, step 103 can be executed.

[0035] Step 103: According to the labels corresponding to the positions of each character in each frame of license plate image, filter out the license plate images containing blurred labels and / or occluded labels from multiple frames of license plate images to obtain the remaining candidate license plate images.

[0036] In the embodiments of the present disclosure, each frame of license plate image can be screened for blurred labels and / or occluded labels, the license plate images containing blurred labels and / or occluded labels can be filtered out, and the license plate images not containing blurred labels and / or occluded labels can be retained as candidate license plate images.

[0037] Step 104: Recognize the license plate of the target vehicle according to the candidate license plate images.

[0038] In the embodiments of the present disclosure, the candidate license plate images can be one frame or multiple frames. When the candidate license plate image is one frame, the target license plate of the target vehicle can be determined according to the labels corresponding to the positions of each character in the candidate license plate image; when the candidate license plate images are multiple frames, one frame of target license plate image can be determined from the multiple frames of candidate license plate images, and then the target license plate of the target vehicle can be determined according to the labels corresponding to the positions of each character in the target license plate image.

[0039] In summary, by filtering out the license plate images containing blurred labels and / or occluded labels from multiple frames of license plate images and recognizing the remaining candidate license plate images, the license plate of the target vehicle is obtained. Thereby, invalid license plates such as blur and / or occlusion can be effectively filtered out, and license plate images with better recognition effects can be recognized, improving the recognition accuracy of the license plate.

[0040] In order to accurately recognize the license plate of the target vehicle from the candidate license plate images. As Figure 2 shown, Figure 2It is a schematic flowchart of the license plate recognition method provided in the second embodiment of the present disclosure. In the embodiment of the present disclosure, when there are multiple frames of candidate license plate images, the target license plate image can be determined from the candidate license plate images, and the target license plate of the target vehicle can be determined according to the labels corresponding to the positions of each character in the target license plate image. Figure 2 The illustrated embodiment may include the following steps:

[0041] Step 201, obtain multiple frames of license plate images of the target vehicle.

[0042] Step 202, input each frame of the license plate image into a license plate recognition model for license plate recognition to obtain the labels corresponding to the positions of each character in each frame of the license plate image.

[0043] Step 203, according to the labels corresponding to the positions of each character in each frame of the license plate image, filter out the license plate images containing blurred labels and / or occluded labels from the multiple frames of license plate images to obtain the remaining candidate license plate images.

[0044] Step 204, when there are multiple frames of candidate license plate images, determine the target license plate image from the multiple frames of candidate license plate images according to the label confidence corresponding to the positions of each character in each frame of the candidate license plate image.

[0045] It should be understood that when the license plate recognition model outputs the labels corresponding to the positions of each character in each frame of the license plate image, the license plate recognition model can also output the label confidence (probability) of each character position. The higher the label confidence, the higher the confidence of the license plate image corresponding to the license plate. Therefore, the target license plate image can be determined from the multiple frames of candidate license plate images according to the label confidence corresponding to the positions of each character in the candidate license plate images.

[0046] Optionally, for any one frame of candidate license plate image, determine the license plate confidence corresponding to the one frame of candidate license plate image according to the label confidence of each character position in the one frame of candidate license plate image; sort the license plate confidences corresponding to each candidate license plate image to obtain the sorting result of each candidate license plate image; determine the target license plate image according to the sorting result.

[0047] That is to say, in order to obtain a license plate image with a better recognition effect, the label confidence levels of each character position in any frame of candidate license plate images can be sorted according to a preset rule, and the license plate confidence level corresponding to this frame of candidate license plate image can be determined according to the sorting result. For example, the label confidence levels of each character position can be sorted in descending order, and the minimum value of the label confidence level can be used as the license plate confidence level corresponding to this frame of candidate license plate image. Furthermore, the license plate confidence levels corresponding to the candidate license plate images are sorted to obtain the sorting results of each candidate license plate image, and the target license plate image is determined according to the sorting results. For example, the license plate confidence levels corresponding to each candidate license plate image can be sorted from high to low, and the candidate license plate image with the highest license plate confidence level can be used as the target license plate image.

[0048] Step 205: Determine the target license plate of the target vehicle according to the labels corresponding to each character position in the target license plate image.

[0049] Furthermore, the target license plate of the target vehicle can be determined according to the character labels corresponding to each character position in the target license plate image.

[0050] It should be noted that the execution processes of steps 201 to 203 can refer to the execution processes of the above embodiments and will not be elaborated here.

[0051] In summary, when there are multiple frames of candidate license plate images, the target license plate image is determined from the multiple frames of candidate license plate images according to the label confidence levels corresponding to each character position in each frame of candidate license plate image; and the target license plate of the target vehicle is determined according to the labels corresponding to each character position in the target license plate image. Thus, the target license plate image with a better recognition effect can be accurately determined from the candidate license plate images, reducing the cost of manual review. Furthermore, the license plate image with a better recognition is recognized to determine the target license plate of the target vehicle, saving relevant resources and improving the license plate recognition accuracy of the vehicle.

[0052] In order to enable the license plate recognition model to accurately output the labels corresponding to each character in the license plate image, as Figure 3 shown, Figure 3 is a schematic flowchart of the license plate recognition method provided in Embodiment 3 of the present disclosure. In the embodiment of the present disclosure, the license plate recognition model can be trained so that the license plate recognition model learns the corresponding relationship between each character position and the label in the license plate image. Figure 3 The embodiment shown may include the following steps:

[0053] Step 301: Obtain multiple frames of license plate images of the target vehicle.

[0054] Step 302: Obtain multiple sample license plate images.

[0055] In the embodiments of the present disclosure, the sample license plate image can be collected online. For example, through web crawler technology, an image containing a vehicle license plate can be collected online as the sample license plate image. Alternatively, the sample license plate image can also be an image containing a vehicle license plate collected offline, or the sample license plate image can also be a synthetic image, etc. The embodiments of the present disclosure do not limit this.

[0056] Step 303: For each sample license plate image, label each character position in the sample license plate image to obtain the annotation label corresponding to each character position; wherein, the annotation label includes at least one of a character label, a blur label, and a blank label.

[0057] Further, each character position of each sample license plate image can be labeled to obtain the annotation label corresponding to each character position. Among them, the annotation label can include at least one of a character label, a blur label, and a blank label.

[0058] It should be noted that, in order to improve the training effect of the model, the label can be manually annotated. Or, in order to reduce the labor cost and improve the training efficiency of the model, the label can also be automatically annotated. For example, through an annotation model, each character position of the sample license plate image can be automatically annotated. The present disclosure does not limit this. Further, after automatically annotating each character position of the sample license plate image, the labeled label in the sample license plate image can also be audited manually to improve the accuracy of the sample annotation result, thereby improving the training effect of the model.

[0059] Step 304: Obtain the prediction label corresponding to each character position output by the license plate recognition model.

[0060] Furthermore, input the sample license plate image into the license plate recognition model to obtain the prediction label corresponding to each character position output by the license plate recognition model.

[0061] Step 305: Train the license plate recognition model according to the difference between the prediction label and the corresponding annotation label in the sample license plate image to minimize the difference.

[0062] In the embodiments of the present disclosure, after obtaining the prediction label corresponding to each character position output by the license plate recognition model, the prediction label can be compared with the corresponding annotation label in the sample license plate image to determine the difference between the prediction label and the corresponding annotation label in the sample license plate image, and the parameters of the license plate recognition model can be adjusted according to this difference to minimize this difference.

[0063] Step 306: Input each frame of license plate image into the license plate recognition model for license plate recognition to obtain the label corresponding to each character position in each frame of license plate image.

[0064] Step 307: According to the labels corresponding to the positions of each character in each frame of license plate image, filter out the license plate images containing blurred labels and / or occluded labels from multiple frames of license plate images to obtain the remaining candidate license plate images.

[0065] Step 308: Identify the license plate of the target vehicle according to the candidate license plate images.

[0066] It should be noted that the execution processes of steps 301, 306 - 308 can refer to the execution processes of the above embodiments and will not be elaborated here.

[0067] In summary, by obtaining multiple sample license plate images; for each sample license plate image, performing label annotation on the positions of each character in the sample license plate image to obtain the annotation labels corresponding to the positions of each character; wherein, the annotation labels include at least one of character labels, blurred labels, and blank labels; obtaining the prediction labels corresponding to the positions of each character output by the license plate recognition model; training the license plate recognition model according to the difference between the prediction labels and the corresponding annotation labels in the sample license plate image to minimize the difference. Thus, by training the license plate recognition model, the license plate recognition model can learn the correspondence between the positions of each character in the license plate image and the labels, and the license plate recognition model can accurately output the labels corresponding to each character in the license plate image.

[0068] To accurately determine the occluded labels corresponding to the positions of each character in the vehicle image, as Figure 4 shown, Figure 4 is a schematic flowchart of the license plate recognition method provided in Embodiment 4 of the present disclosure. In the embodiment of the present disclosure, when the number of blank labels corresponding to the positions of each character in the license plate image is greater than the corresponding allowable number of blank labels, the blank labels to be corrected in the license plate image can be corrected. Figure 4 The shown embodiment may include the following steps:

[0069] Step 401: Obtain multiple frames of license plate images of the target vehicle.

[0070] Step 402: Input each frame of license plate image into the license plate recognition model for license plate recognition to obtain the labels corresponding to the positions of each character in each frame of license plate image.

[0071] Step 403: Determine the first number of blank labels corresponding to the positions of each character in the license plate image.

[0072] In the embodiment of the present disclosure, the labels corresponding to the positions of each character in the license plate image are statistically counted to obtain the first number of blank labels corresponding to the positions of each character in the license plate image.

[0073] Step 404: Determine a second quantity based on the difference between the lengths of the characters in the license plate image and the set output quantity of the license plate recognition model.

[0074] In the embodiments of the present disclosure, the output quantity of the license plate recognition model can be preset. For example, the set output length of the vehicle recognition model can be set to 8. Since the character lengths of the license plate images of different target vehicles are different, and the number of blank labels allowed to be output by the license plate recognition model is also different, the second quantity can be determined based on the difference between the lengths of the characters in the license plate image and the set output quantity of the license plate recognition model.

[0075] For example, the license plate character length in the license plate image of target vehicle 1 is 8 digits, the set output length of the license plate recognition model is 8 digits, and the number of blank labels corresponding to the character positions in the license plate image allowed to be output by the license plate recognition model is 0; for another example, the license plate character length of target vehicle 2 is 7 digits, the set output length of the license plate recognition model is 8 digits, and the number of blank labels corresponding to the character positions in the license plate image allowed to be output by the license plate recognition model is 1.

[0076] Step 405: When the first quantity is greater than the second quantity, correct the labels corresponding to each character position in the license plate image according to the difference between the second quantity and the first quantity.

[0077] That is to say, when the number of blank labels corresponding to each character position output by the current license plate recognition model is greater than the number of blank labels allowed to be output by the license plate recognition model, the blank labels to be corrected in the license plate image can be corrected.

[0078] Optionally, when the first quantity is greater than the second quantity, select the difference quantity of blank labels to be corrected from the labels corresponding to each character position in the license plate image in a preset order; correct the blank labels to be corrected into occlusion labels.

[0079] In the embodiments of the present disclosure, when training the license plate recognition model, the order of outputting blank labels by the license plate recognition model can be set. For example, when the license plate character length in the license plate image is 7 digits and the set output length of the license plate recognition model is 8 digits, the first digit output by the license plate recognition model can be set as a blank label or the last digit output by the license plate recognition model can be set as a blank label.

[0080] Furthermore, in order to accurately determine the occlusion labels corresponding to the positions of each character in the vehicle image, when the first quantity is greater than the second quantity, the difference between the first quantity and the second quantity can be obtained, that is, the quantity of blank labels corresponding to the positions of each character output by the current license plate recognition model and the quantity of blank labels allowed to be output by the license plate recognition model are obtained, and the difference quantity of blank labels is selected from the blank labels corresponding to the positions of each character output by the current license plate recognition model in a preset order as the blank labels to be corrected, and the blank labels to be corrected are corrected to occlusion labels. For example, if the length of the license plate characters of the target vehicle is 6 digits, the set output length of the license plate recognition model is 8 digits, the quantity of blank labels corresponding to the positions of characters in the license plate image allowed to be output by the license plate recognition model is 2, and the quantity of blank labels output by the current license plate recognition model is 3, 1 blank label can be selected from the blank labels in the order from front to back as the blank label to be corrected, and the blank label to be corrected is corrected to an occlusion label.

[0081] Step 406: According to the labels corresponding to the positions of each character in each frame of license plate image, filter out the license plate images containing blurred labels and / or occlusion labels from multiple frames of license plate images to obtain the remaining candidate license plate images.

[0082] Step 407: Identify the license plate of the target vehicle according to the candidate license plate images.

[0083] It should be noted that the execution processes of steps 401-402 and 406-407 can refer to the execution processes of the above embodiments, which will not be elaborated here.

[0084] In summary, by determining the first quantity of blank labels corresponding to the positions of each character in the license plate image; determining the second quantity according to the difference between the lengths of each character in the license plate image and the set output quantity of the license plate recognition model; when the first quantity is greater than the second quantity, correcting the labels corresponding to the positions of each character in the license plate image according to the difference between the second quantity and the first quantity. Thus, the occlusion labels corresponding to the positions of each character in the vehicle image can be accurately determined.

[0085] To illustrate the above embodiments more clearly, examples are given below for illustration.

[0086] For example, Figure 5As shown, a deep learning algorithm can be used to recognize license plate images, obtaining labels (including character labels and / or blank labels and / or blurred labels) and confidence levels corresponding to the positions of each character. According to the labels corresponding to the positions of each character in the license plate image, license plate images containing blurred labels and / or occlusion labels (incomplete license plates) are filtered out from multiple frames of license plate images to obtain the remaining candidate license plate images. Furthermore, the minimum value of the label confidence levels of each character position in the candidate license plate image is used as the license plate confidence level corresponding to the candidate license plate image. Finally, the license plate confidence levels corresponding to each candidate license plate image can be sorted from high to low, and the candidate license plate image with the highest license plate confidence level can be used as the target license plate image. And according to the character labels corresponding to the positions of each character in the target license plate image, the target license plate of the target vehicle is determined.

[0087] Among them, the license plate recognition model adopted by the deep learning algorithm can be a multi-label classifier, including character labels, blank labels, and blurred labels corresponding to each character in daily license plates (such as including blank, blurred, and abbreviations of 34 provincial-level administrative regions, 10 digits, 26 letters, "gua", "jun", "jing", "shi", etc. labels). The input of this classifier can be a license plate picture, and the output can be the label to which each character belongs, as well as the corresponding confidence level.

[0088] The output of the license plate recognition model adopted by the deep learning algorithm can be 8 characters, covering the maximum number of characters in national standard license plates, and can also support outputs of less than 8 characters (such as for cars, it can support outputs of 7 characters). The present disclosure does not limit this. The order of the output characters corresponds one by one to the actual license plate order, and the area of the license plate image can be located according to the order of the characters.

[0089] The license plate recognition method of the embodiments of the present disclosure includes obtaining multiple frames of license plate images of a target vehicle; inputting each frame of license plate image into a license plate recognition model for license plate recognition to obtain the labels corresponding to the positions of each character in each frame of license plate image; according to the labels corresponding to the positions of each character in each frame of license plate image, filtering out license plate images containing blurred labels and / or occlusion labels from multiple frames of license plate images to obtain the remaining candidate license plate images; and recognizing the license plate of the target vehicle according to the candidate license plate images. This method filters out license plate images containing blurred labels and / or occlusion labels from multiple frames of license plate images, recognizes the remaining candidate license plate images, and obtains the license plate of the target vehicle. Thus, invalid license plates such as blurred and / or occluded ones can be effectively filtered out, and license plate images with better recognition effects are recognized, improving the recognition accuracy of license plates.

[0090] Corresponding to the license plate recognition method provided in the above Figures 1 to 5 embodiment, the present disclosure also provides a license plate recognition device. Since the license plate recognition device provided in the embodiments of the present disclosure is the same as the above Figures 1 to 5The license plate recognition method provided by the embodiment corresponds to the license plate recognition method provided by the embodiment, so the implementation manner of the license plate recognition method is also applicable to the license plate recognition device provided by the embodiments of the present disclosure, and will not be described in detail in the embodiments of the present disclosure.

[0091] Figure 6 FIG. 6 is a schematic structural diagram of a license plate recognition device provided by Embodiment 6 of the present disclosure.

[0092] As Figure 6 shown, the license plate recognition device 600 may include: a first acquisition module 610, a first recognition module 620, a screening module 630, and a second recognition module 640.

[0093] Among them, the first acquisition module 610 is configured to acquire multiple frames of license plate images of a target vehicle; the first recognition module 620 is configured to input each frame of license plate image into a license plate recognition model for license plate recognition; the screening module 630 is configured to screen out license plate images containing blurred labels and / or occluded labels from the multiple frames of license plate images according to the labels corresponding to the positions of each character in each frame of license plate image, so as to obtain remaining candidate license plate images to obtain the labels corresponding to the positions of each character in each frame of license plate image; the second recognition module 640 is configured to recognize the license plate of the target vehicle according to the candidate license plate images.

[0094] In a possible implementation manner of the embodiments of the present disclosure, each of the labels output by the license plate recognition model has a corresponding label confidence level; the second recognition module 640 is configured to: in the case where there are multiple frames of candidate license plate images, determine a target license plate image from the multiple frames of candidate license plate images according to the label confidence levels corresponding to the positions of each character in each frame of candidate license plate image; and determine the target license plate of the target vehicle according to the labels corresponding to the positions of each character in the target license plate image.

[0095] In a possible implementation manner of the embodiments of the present disclosure, the second recognition module 640 is further configured to: for any one frame of candidate license plate image, determine the license plate confidence level corresponding to one frame of candidate license plate image according to the label confidence levels of the positions of each character in one frame of candidate license plate image; sort the license plate confidence levels corresponding to the respective candidate license plate images to obtain a sorting result of the respective candidate license plate images; and determine the target license plate image according to the sorting result.

[0096] In a possible implementation manner of the embodiments of the present disclosure, the license plate recognition device 600 further includes: a second acquisition module, a labeling module, a third acquisition module, and a training module.

[0097] Among them, the second acquisition module is used to acquire a plurality of sample license plate images; the annotation module is used to, for each sample license plate image, perform label annotation on each character in the sample license plate image to obtain the annotation label corresponding to each character; among them, the annotation label includes at least one of a character label, a blur label, and a blank label; the third acquisition module is used to acquire the prediction label corresponding to each character output by the license plate recognition model; the training module is used to train the license plate recognition model according to the difference between the prediction label and the corresponding annotation label in the sample license plate image to minimize the difference.

[0098] In a possible implementation manner of the embodiment of the present disclosure, the license plate recognition device 600 further includes: a determination module and a correction module.

[0099] Among them, the determination module is used to determine the first quantity of the labels corresponding to the positions of each character in the license plate image as blank labels; the determination module is further used to determine the second quantity according to the difference between the lengths of each character in the license plate image and the set output quantity of the license plate recognition model; the correction module is used to, when the first quantity is greater than the second quantity, correct the labels corresponding to the positions of each character in the license plate image according to the difference between the second quantity and the first quantity.

[0100] In a possible implementation manner of the embodiment of the present disclosure, the correction module is used to: when the first quantity is greater than the second quantity, select the difference quantity of blank labels to be corrected from the labels corresponding to the positions of each character in the license plate image in a preset order; correct the blank labels to be corrected to occlusion labels.

[0101] The license plate recognition device in the embodiment of the present disclosure acquires multiple frames of license plate images of a target vehicle; inputs each frame of license plate image into a license plate recognition model for license plate recognition to obtain the labels corresponding to the positions of each character in each frame of license plate image; filters out the license plate images containing blur labels and / or occlusion labels from the multiple frames of license plate images according to the labels corresponding to the positions of each character in each frame of license plate image to obtain the remaining candidate license plate images; recognizes the license plate of the target vehicle according to the candidate license plate images. The device can realize filtering out the license plate images containing blur labels and / or occlusion labels from the multiple frames of license plate images, recognizing the remaining candidate license plate images, and obtaining the license plate of the target vehicle. Thus, invalid license plates such as blur and / or occlusion can be effectively filtered out, and license plate images with better recognition effects can be recognized, improving the recognition accuracy of the license plate.

[0102] To implement the above embodiments, the present disclosure also provides an electronic device, which may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the license plate recognition method proposed in any of the above embodiments of the present disclosure.

[0103] To implement the above embodiments, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the license plate recognition method proposed in any of the above embodiments of the present disclosure.

[0104] To implement the above embodiments, the present disclosure also provides a computer program product, which includes a computer program that implements the steps of the license plate recognition method proposed in any of the above embodiments of the present disclosure when executed by a processor.

[0105] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0106] Figure 7 A schematic block diagram of an example electronic device that can be used to implement the embodiments of the present disclosure is shown. Among them, the electronic device may include a server and a client in the above embodiments. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0107] As Figure 7 shown, the device 700 includes a computing unit 701, which can execute various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 702 or a computer program loaded from a storage unit 707 into a RAM (Random Access Memory) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An I / O (Input / Output) interface 705 is also connected to the bus 704.

[0108] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as a keyboard, a mouse, etc.; output unit 707, such as various types of displays, speakers, etc.; storage unit 708, such as a disk, an optical disc, etc.; and communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0109] Computing unit 701 can be various general-purpose and / or dedicated processing components with processing and computing capabilities. Some examples of computing unit 701 include, but are not limited to, CPU (Central Processing Unit), GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSP (Digital Signal Processor), and any suitable processor, controller, microcontroller, etc. Computing unit 701 executes the various methods and processes described above, such as the above license plate recognition method. For example, in some embodiments, the above license plate recognition method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by computing unit 701, one or more steps of the license plate recognition method described above can be executed. Alternatively, in other embodiments, computing unit 701 can be configured to execute the above license plate recognition method by any other suitable means (e.g., by means of firmware).

[0110] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SoCs (System On Chip), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0111] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0112] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a RAM, a ROM, an EPROM (Electrically Programmable Read-Only Memory), or a flash memory, an optical fiber, a CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0113] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or an LCD (Liquid Crystal Display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball), by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0114] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, and a blockchain network.

[0115] A computer system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services (Virtual Private Server). The server may also be a server of a distributed system or a server combined with a blockchain.

[0116] Herein, it should be noted that artificial intelligence is a discipline that studies to make a computer simulate certain thinking processes and intelligent behaviors of humans (such as learning, reasoning, thinking, planning, etc.), and there are both hardware-level technologies and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include several major directions such as computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, and knowledge graph technology.

[0117] According to the technical solution of the embodiment of the present disclosure, by screening out license plate images containing blurred labels and / or occlusion labels from multiple frames of license plate images, and identifying the remaining candidate license plate images, the license plate of the target vehicle is obtained. Thus, invalid license plates such as blur and / or occlusion can be effectively filtered out, and license plate images with better recognition effects are identified, improving the recognition accuracy of the license plate.

[0118] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added, or deleted. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solution disclosed in the present disclosure can be achieved, and no limitation is made herein.

[0119] The above specific embodiments do not constitute a limitation to the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A license plate recognition method, comprising: Performing image acquisition on the license plate of a target vehicle to obtain multiple frames of license plate images of the target vehicle; Inputting each frame of the license plate image into a license plate recognition model for license plate recognition to obtain labels corresponding to the positions of each character in each frame of the license plate image, where the labels include character labels, blur labels, occlusion labels, and / or blank labels; Determining a first quantity of blank labels corresponding to the positions of each character in the license plate image, and determining a second quantity according to the difference between the lengths of each character in the license plate image and the set output quantity of the license plate recognition model; In the case where the first quantity is greater than the second quantity, determining the difference between the second quantity and the first quantity, determining the blank labels to be corrected according to the difference between the second quantity and the first quantity, and correcting the blank labels to be corrected to occlusion labels; Filtering out license plate images containing blur labels and / or occlusion labels from the multiple frames of license plate images according to the labels corresponding to the positions of each character in each frame of the license plate image to obtain remaining candidate license plate images; If the candidate license plate image is one frame, determining the license plate of the target vehicle according to the labels corresponding to the positions of each character in the candidate license plate image; otherwise, determining a frame of target license plate image from the remaining candidate license plate images, and determining the target license plate of the target vehicle according to the labels corresponding to the positions of each character in the target license plate image.

2. The method according to claim 1, wherein Each of the labels output by the license plate recognition model has a corresponding label confidence; determining a frame of target license plate image from the remaining candidate license plate images includes: In the case where there are multiple frames of candidate license plate images, determining the target license plate image from the multiple frames of candidate license plate images according to the label confidences corresponding to the positions of each character in each frame of the candidate license plate image.

3. The method according to claim 2, wherein The determining the target license plate image from the multiple frames of candidate license plate images according to the label confidences corresponding to the positions of each character in each frame of the candidate license plate image includes: For any one frame of candidate license plate image, determining the license plate confidence corresponding to the one frame of candidate license plate image according to the label confidences of the positions of each character in the one frame of candidate license plate image; Sorting the license plate confidences corresponding to each of the candidate license plate images to obtain a sorting result of each of the candidate license plate images; Determining the target license plate image according to the sorting result.

4. The method according to any one of claims 1-3, wherein, Before inputting the license plate image into the license plate recognition model for license plate recognition to obtain the labels corresponding to the positions of each character in the license plate image, further comprising: Obtaining multiple sample license plate images; For each sample license plate image, performing label annotation on the positions of each character in the sample license plate image to obtain annotation labels corresponding to the positions of each character; Obtaining predicted labels corresponding to the positions of each character output by the license plate recognition model; Training the license plate recognition model according to the difference between the predicted labels and the corresponding annotation labels in the sample license plate image to minimize the difference.

5. The method according to claim 1, wherein Determine the blank labels to be corrected according to the difference between the second quantity and the first quantity, including: selecting the blank labels to be corrected in the quantity of the difference from the labels corresponding to each character position in the license plate image in a preset order.

6. A license plate recognition device, comprising: A first acquisition module, configured to collect images of the license plate of a target vehicle to obtain multiple frames of license plate images of the target vehicle; A first recognition module, configured to input each frame of the license plate image into a license plate recognition model for license plate recognition to obtain labels corresponding to each character position in each frame of the license plate image, where the labels include character labels, blurred labels, occlusion labels, and / or blank labels; A determination module, configured to determine a first quantity of blank labels corresponding to each character position in the license plate image, and determine a second quantity according to the difference between the length of each character in the license plate image and the set output quantity of the license plate recognition model; A correction module, configured to, when the first quantity is greater than the second quantity, determine the difference between the second quantity and the first quantity, determine the blank labels to be corrected according to the difference between the second quantity and the first quantity, and correct the blank labels to be corrected to occlusion labels; A screening module, configured to screen out license plate images containing blurred labels and / or occlusion labels from the multiple frames of license plate images according to the labels corresponding to each character position in each frame of the license plate image to obtain reserved candidate license plate images; A second recognition module, configured to, if the candidate license plate image is one frame, determine the license plate of the target vehicle according to the labels corresponding to each character position in the candidate license plate image, otherwise, determine a frame of target license plate image from the reserved candidate license plate images, and determine the license plate of the target vehicle according to the labels corresponding to each character position in the target license plate image.

7. The apparatus according to claim 6, wherein, Each of the labels output by the license plate recognition model has a corresponding label confidence; determining a frame of target license plate image from the reserved candidate license plate images includes: When there are multiple frames of candidate license plate images, determining the target license plate image from the multiple frames of candidate license plate images according to the label confidence corresponding to each character position in each frame of the candidate license plate image.

8. The apparatus according to claim 7, wherein, The determining the target license plate image from the multiple frames of candidate license plate images according to the label confidence corresponding to each character position in each frame of the candidate license plate image includes: For any one frame of candidate license plate image, determining the license plate confidence corresponding to the one frame of candidate license plate image according to the label confidence of each character position in the one frame of candidate license plate image; Sorting the license plate confidences corresponding to each of the candidate license plate images to obtain a sorting result of each of the candidate license plate images; Determining the target license plate image according to the sorting result.

9. The apparatus according to any one of claims 6-8, wherein, The device further includes: A second acquisition module, configured to acquire multiple sample license plate images; A labeling module, configured to, for each of the sample license plate images, perform label labeling on each character in the sample license plate image to obtain labeled labels corresponding to each character; A third acquisition module, configured to acquire prediction labels corresponding to each character output by the license plate recognition model; A training module, configured to train the license plate recognition model according to the difference between the prediction labels and the corresponding annotation labels in the sample license plate image, so as to minimize the difference.

10. The apparatus according to claim 6, wherein, Determining the blank labels to be corrected according to the difference between the second quantity and the first quantity: including: selecting the difference quantity of blank labels to be corrected from the labels corresponding to each character position in the license plate image in a preset order.

11. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the license plate recognition method according to any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the license plate recognition method according to any one of claims 1-5.

13. A computer program product, comprising a computer program, where the computer program implements the steps of the license plate recognition method according to any one of claims 1-5 when being executed by a processor.

Images