License plate recognition method, device, storage medium and electronic device

Abstract

The present application discloses a license plate recognition method, device, storage medium and electronic device, including: acquiring license plate images in actual scenes, and identifying rare license plate images in all acquired license plate images through unsupervised learning; performing text recognition on the rare license plate images using a text recognition model, and using the text recognition results as pseudo labels of the rare license plate images; performing uncertainty estimation on the pseudo labels to obtain uncertainty estimation results; using the rare license plate images and their pseudo labels as training data to train a generation model for generating rare license plate images; wherein the loss function value in the process of training the generation model is determined based on the uncertainty estimation results of the pseudo labels; generating rare license plate images using the generation model of the rare license plate images for training the license plate recognition model; and performing license plate recognition using the license plate recognition model. The application of the present application can effectively improve the recognition accuracy of rare license plates.

Description

Technical Field

[0001] The present application relates to image recognition technology, and in particular to a license plate recognition method, device, storage medium and electronic device. Background Art

[0002] In recent years, with the rapid progress of deep learning technology, it has achieved remarkable results in many fields, and license plate recognition technology is one of its important applications. Driven by big data and artificial intelligence, the accuracy of license plate recognition has soared to more than 99%, almost reaching the level of human eye recognition. However, there are still some challenges behind the high accuracy.

[0003] An important feature of license plate recognition technology based on deep learning is that it is data-driven, and its performance is highly dependent on the richness and diversity of training data. However, in reality, there are many types and styles of license plates, and the license plate standards in different regions and countries are also different. This leads to a phenomenon: common types of license plates have a high frequency of occurrence and their data samples are relatively rich, so the model has a higher accuracy rate when recognizing these license plates. However, for those rare types of license plates, it is difficult for the model to learn enough features during training, resulting in a gap between the accuracy of the model in recognizing rare types of license plates and the accuracy of recognizing common types of license plates. Summary of the invention

[0004] The present application provides a license plate recognition method, device, storage medium and electronic device, which can effectively improve the recognition accuracy of rare types of license plates.

[0005] To achieve the above objectives, this application adopts the following technical solutions:

[0006] A license plate recognition method, comprising:

[0007] Obtain license plate images in real scenes, and identify rare license plate images among all acquired license plate images through unsupervised learning;

[0008] Using a text recognition model to perform text recognition on the rare license plate image, and using the text recognition result as a pseudo label of the rare license plate image;

[0009] Performing uncertainty estimation on the pseudo-label to obtain an uncertainty estimation result;

[0010] The rare license plate image and its pseudo-label are used as training data to train a generative model for generating rare license plate images; wherein the loss function value in the process of training the generative model is determined based on the uncertainty estimation result of the pseudo-label;

[0011] Generate rare license plate images using the rare license plate image generation model for training a license plate recognition model;

[0012] The license plate recognition model is used to perform license plate recognition.

[0013] Preferably, the step of determining rare license plate images from all acquired license plate images by unsupervised learning comprises:

[0014] Extracting image features and color vectors from all license plate images; wherein the color vector is used to represent the color distribution law in the license plate image;

[0015] Based on the image features and color vectors of the license plate images, all license plate images are clustered, and license plate images belonging to the rare license plate classification are determined as rare license plate images.

[0016] Preferably, after extracting the image features and before clustering, the method further comprises: performing dimensionality reduction processing on the extracted image features using principal component analysis;

[0017] The clustering is performed based on the image features after the dimensionality reduction process.

[0018] Preferably, after setting weights for the pseudo labels, the method further comprises:

[0019] Based on the uncertainty estimation result of the pseudo-label, data enhancement is performed on the rare license plate image to generate a new license plate image, and the license plate image newly generated by data enhancement is used for training the generation model; wherein, the smaller the uncertainty of the pseudo-label of the rare license plate image, the greater the number of new license plate images generated by data enhancement.

[0020] Preferably, estimating the uncertainty of the pseudo-label comprises:

[0021] Using the text recognition model to perform multiple text recognition on the rare license plate image corresponding to the pseudo label, each time the text recognition process is performed, some nodes in the text recognition model are randomly closed by dropout;

[0022] Based on the predicted labels obtained by the multiple text recognitions, calculating the confidence mean and variance of the predicted labels;

[0023] The uncertainty estimate is determined based on the mean and the variance.

[0024] Preferably, the mean value is based on and variance Determine the uncertainty estimate ,include:

[0025] Wherein, λ is the weight of the confidence mean.

[0026] A license plate recognition device comprises: a rare license plate extraction unit, a text recognition and uncertainty estimation unit, a rare license plate generation unit and a license plate recognition unit;

[0027] The rare license plate extraction unit is used to obtain license plate images in actual scenes and identify rare license plate images from all acquired license plate images through unsupervised learning;

[0028] The text recognition and uncertainty assessment unit is used to perform text recognition on the rare license plate image using a text recognition model, and use the text recognition result as a pseudo label of the rare license plate image; and is also used to perform uncertainty estimation on the pseudo label to obtain an uncertainty estimation result;

[0029] The rare license plate generation unit is used to train a generation model for generating rare license plate images using the rare license plate images and their pseudo labels as training data; wherein the loss function value in the process of training the generation model is determined based on the uncertainty estimation result of the pseudo labels;

[0030] The license plate recognition unit is used to train a license plate recognition model using the generated rare license plate image; and is also used to perform license plate recognition using the license plate recognition model.

[0031] Preferably, in the rare license plate extraction unit, determining rare license plate images from all acquired license plate images by unsupervised learning comprises:

[0032] Extracting image features and color vectors from all license plate images; wherein the color vector is used to represent the color distribution law in the license plate image;

[0033] Based on the image features and color vectors of the license plate images, all license plate images are clustered, and license plate images belonging to the rare license plate classification are determined as rare license plate images.

[0034] Preferably, the rare license plate extraction unit is further used to perform dimensionality reduction processing on the extracted image features using principal component analysis; and is also used to perform clustering based on the image features after dimensionality reduction processing.

[0035] Preferably, the device further includes a data enhancement unit, which is used to perform data enhancement on the rare license plate image to generate a new license plate image, and use the license plate image newly generated by data enhancement for training the generation model; wherein, the smaller the uncertainty of the pseudo-label of the rare license plate image, the greater the number of new license plate images generated by data enhancement.

[0036] Preferably, in the text recognition and uncertainty estimation unit, the performing uncertainty estimation on the pseudo-label comprises:

[0037] Using the text recognition model to perform multiple text recognition on the rare license plate images corresponding to the pseudo labels, and randomly closing some nodes in the text recognition model by dropout for each prediction;

[0038] Based on the predicted labels obtained by the multiple text recognitions, calculating the confidence mean and variance of the predicted labels;

[0039] The uncertainty estimate is determined based on the mean and the variance.

[0040] Preferably, in the text recognition and uncertainty estimation unit, the and variance Determine the uncertainty estimate ,include: ;

[0041] Wherein, λ is the weight of the confidence mean.

[0042] The present application provides a computer-readable storage medium having computer instructions stored thereon, and when the instructions are executed by a processor, any of the above-mentioned license plate recognition methods can be implemented.

[0043] The present application discloses an electronic device, which includes at least a computer-readable storage medium and a processor;

[0044] The processor is used to read executable instructions from the computer-readable storage medium and execute the instructions to implement any of the above-mentioned license plate recognition methods.

[0045] It can be seen from the above technical solution that in this application, license plate images in actual scenes are obtained, and rare license plate images are determined in all the obtained license plate images through unsupervised learning; the obtained rare license plate images are subjected to text recognition using a text recognition model, and the text recognition results are used as pseudo labels for these rare license plate images. Considering the accuracy of the pseudo labels, the uncertainty of the pseudo labels is estimated to obtain the uncertainty estimation results of the pseudo labels. Next, these rare license plate images and their pseudo labels are used as training data to train a generative model for generating rare license plate images, and the loss function value in the model training process is determined based on the uncertainty estimation results of the pseudo labels to control the role of the corresponding pseudo label data in the model training, that is, the lower the uncertainty of the pseudo label data, the greater the corresponding role in the training, thereby effectively controlling the accuracy of the rare license plate image generation model. Then, the rare license plate images are generated using the generation model of rare license plate images, and the accuracy of the generation model is controlled through the above-mentioned processing, so that rare license plate images with higher accuracy can be effectively generated; finally, these automatically generated and large number of rare license plate images are used to train the license plate recognition model, so as to effectively improve the recognition ability of the trained license plate recognition model for rare license plate images, and then use the trained license plate recognition model to perform license plate recognition, which can effectively improve the recognition accuracy of rare types of license plates. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of the basic process of the license plate recognition method in this application;

[0047] Figure 2 This is a schematic diagram of a specific process of a license plate recognition method in a specific embodiment of the present application;

[0048] Figure 3 It is a basic structural diagram of a license plate recognition device;

[0049] Figure 4 It is a schematic diagram of the basic structure of the electronic device in this application. DETAILED DESCRIPTION

[0050] In order to make the objectives, technical means and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings.

[0051] The basic idea of ​​this application is to generate a generative model for rare license plate images through training, so as to automatically generate a large amount of rare license plate image data for training the license plate recognition model, so as to improve the recognition ability of the license plate recognition model for rare types of license plates.

[0052] Figure 1 Schematic diagram of the basic process of the license plate recognition method in this application. Figure 1 As shown, the method includes:

[0053] Step 101 , obtaining license plate images in actual scenes, and determining rare license plate images among all obtained license plate images through unsupervised learning.

[0054] In order to train a generative model for rare license plate images in this application, it is necessary to first construct training samples for the generative model. Specific training samples include rare license plate images and their corresponding license plate content (that is, labels). If the labels of rare license plate images are set manually, the accuracy of the labels can be guaranteed, but the workload is too large, which affects the system performance. Based on this, this application considers the method of automatically setting labels for rare license plate images to obtain more rare license plate images and their corresponding label data, so as to maximize the scale of training samples.

[0055] This step is used to identify rare license plate images from actual scenes. The unsupervised learning method, for example, can be to extract features and perform cluster analysis on the acquired license plate images, and then find the classification of rare license plate types and their corresponding image data from the clustering results as rare license plate images.

[0056] The rare license plate images determined in the above manner are all actual images.

[0057] Step 102: Perform text recognition on the rare license plate image using a text recognition model, and use the text recognition result as a pseudo label of the rare license plate image.

[0058] Rare license plate images for rare license plate image generation models are obtained through the above step 101. This step is used to automatically set labels for these rare license plate images.

[0059] Specifically, text recognition is performed on the acquired rare license plate images, and the automatically recognized license plate content is used as the label of the corresponding license plate image. Since the accuracy of text recognition is not 100%, the labels obtained in this way are referred to as pseudo-labels below, so as to distinguish them from the label information of the manually marked rare license plate images.

[0060] Step 103: perform uncertainty estimation on the pseudo-label to obtain an uncertainty estimation result of the pseudo-label.

[0061] As mentioned above, the pseudo-labels determined by text recognition may be inaccurate. If these inaccurate pseudo-labels are used directly to train the generative model of rare license plate images, it will inevitably affect the accuracy of the corresponding generative model, thereby affecting the accuracy of the generated rare license plate images. When these generated rare license plate images are further used to train the license plate recognition model, it will also affect the accuracy of the license plate recognition model, and thus affect the accuracy of rare license plate recognition using the license plate recognition model.

[0062] Taking the above problems into consideration, in this application, uncertainty estimation is performed on the pseudo-label, and the role of the corresponding pseudo-label in the training of the generation model of rare license plate images is controlled based on the uncertainty estimation results obtained; when the uncertainty of the pseudo-label is smaller, it means that the accuracy of the pseudo-label is higher, then it can play a greater role in the training of the generation model; conversely, when the uncertainty of the pseudo-label is larger, it means that the accuracy of the pseudo-label is lower, then its role in the training of the generation model can be subject to certain restrictions, thereby effectively improving the accuracy of the generation model of rare license plate images, and then improving the accuracy of the generated rare license plate images and the accuracy of the ticket recognition model.

[0063] This step is used to estimate the uncertainty of the pseudo-label and obtain the uncertainty estimation result, thereby marking the uncertainty of the corresponding pseudo-label. Specifically, the stronger the uncertainty of the pseudo-label, the lower the confidence of the pseudo-label.

[0064] The specific method of estimating the uncertainty of the pseudo-label can be set according to actual needs.

[0065] Step 104 , using the rare license plate images and their pseudo labels as training data, to train a generative model for generating rare license plate images.

[0066] Among them, the value of the loss function in the process of training the generation model is determined based on the uncertainty estimation result of the pseudo-label.

[0067] Rare license plate images and corresponding pseudo labels have been determined through step 103. This step uses these rare license plate images and pseudo labels as training data to train the generative model of rare license plate images. In the generative model training process, the loss function value in the training process is determined based on the uncertainty estimation result of the pseudo label. Specifically, the weight of the pseudo label can be determined by the uncertainty estimation result, and then the weight is used to perform weighted processing on the loss function, so as to control the influence of the corresponding pseudo label data on the model parameter update process, thereby achieving the control of the role played by different pseudo labels in the training process through the uncertainty estimation results. In particular, the stronger the uncertainty represented by the uncertainty estimation result, the smaller the corresponding weight.

[0068] Of course, when training the generative model of rare license plate images, in addition to using the rare license plate images and their pseudo labels obtained in the above steps, other actual rare license plate images and manually calibrated real labels can be further used as training samples for training the generative model. Furthermore, with reference to the pseudo labels, weights can also be set for the real labels to control their role in the training process. Generally, the real labels have higher accuracy and should play a greater role in the model training process. Therefore, the weight of the real labels is generally greater than or equal to the weight of the pseudo labels.

[0069] By controlling the pseudo-label weights, a generative model for rare license plate images with higher accuracy can be generated.

[0070] Step 105, using the rare license plate image generation model to generate rare license plate images for training the license plate recognition model.

[0071] The generative model of rare license plate images generated by the aforementioned steps can automatically generate a large number of rare license plate images. Through the control of the aforementioned steps, the accuracy of the generative model can ensure the accuracy of the generated rare license plate images. These large and accurate rare license plate images can be used together with the true value license plate images as training data to form a training set for training the license plate recognition model. The number of license plate categories in the training set is balanced, which can effectively solve the problem of low recognition accuracy of rare license plates caused by unbalanced data samples.

[0072] Step 106, performing license plate recognition using the license plate recognition model.

[0073] Through step 105, a license plate recognition model with higher recognition accuracy for rare types of license plates is trained and generated. Using such a license plate recognition model for license plate recognition can effectively improve the recognition accuracy of rare types of license plates.

[0074] In addition, in the license plate recognition process, steps 101-105 are completed once in advance before license plate recognition to generate a license plate recognition model; after the license plate recognition model is generated, the model is saved, and whenever license plate recognition is required, the license plate image can be directly input into the pre-generated license plate recognition model to obtain the license plate recognition result; there is no need to perform a license plate recognition model training process before each license plate recognition.

[0075] So far, Figure 1 The license plate recognition process shown is finished. Figure 1 The method shown can utilize the generative model of rare license plate images to automatically generate a large number of accurate rare license plate images, thereby effectively expanding the sample size of rare license plate samples in license plate recognition model training, generating a license plate recognition model with stronger recognition capabilities for rare types of license plates, and improving the accuracy of license plate recognition.

[0076] The specific implementation of the license plate recognition method in this application is described below through specific embodiments. Figure 2 This is a schematic diagram of the specific process of the license plate recognition method in the specific embodiment of this application. Figure 2 As shown, the method includes:

[0077] Step 201, obtaining a license plate image in an actual scene and performing image preprocessing.

[0078] This step is used to capture and preprocess license plate images in actual scenarios.

[0079] Specifically, image data containing license plates are collected from on-site data to form an image data set; a license plate detection algorithm is used to detect the license plate position from the collected image and extract the license plate area; the extracted license plate area is further standardized to ensure the uniformity of the input data.

[0080] Step 202: extract image features and color vectors from the preprocessed license plate image, and perform normalization processing.

[0081] In this embodiment, considering that in the license plate recognition process, the distribution law of license plate colors is obviously different for different types of license plates, the color distribution law can be used as one of the bases for license plate classification. Of course, when unsupervised learning classification is usually performed, it can also be performed based on the extracted image features. Based on this, in this application, unsupervised learning is performed based on the image features and color vectors of the license plate image to identify rare types of license plate images.

[0082] Specifically, in this step, image feature extraction is performed on the one hand, and color vector extraction is performed on the other hand. The color vector is used to represent the color distribution law of the license plate image. Generally, color vector extraction obtains a low-dimensional color vector, and image feature extraction obtains a high-dimensional license plate feature vector.

[0083] The extracted color vector (specifically, it can be a pixel sampling vector) and license plate feature vector are normalized to ensure that the data is suitable for the unsupervised learning algorithm.

[0084] Step 203 , clustering is performed based on the extracted image features and color vectors to determine license plate images belonging to the rare type of license plate classification as rare license plate images.

[0085] After determining the image features (i.e., license plate feature vector) and color vector of the license plate, image clustering can be performed to divide all license plate images into several categories, wherein the number of categories of the license plate format can be pre-set as the number of clusters. Specific clustering can use clustering algorithms such as K-means to obtain the corresponding cluster centers. According to the characteristics of the rare type license plate set in advance, the rare type license plate classification and the cluster center of the corresponding classification are determined, and the license plate image belonging to the corresponding classification is determined, which is called a rare license plate image. Specifically, the license plate image within the set range from the cluster center of the rare type license plate classification can be regarded as a license plate image belonging to the rare type license plate classification, i.e., a rare license plate image. In addition, when determining the rare type license plate classification, for example, the classification obtained by clustering can be set as the rare license plate classification, or the cluster category with the set color feature can be set as the rare license plate type classification. It can be set specifically according to actual needs, and this application does not limit this.

[0086] In addition, the dimension of the image features extracted from the license plate image is relatively high, and it takes more time and resources to process. The corresponding image features include a variety of different categories of image feature information, and some feature information can be regarded as noise for license plate recognition. Based on this, in this embodiment, preferably, before clustering, principal component analysis (PCA) can be used to further reduce the dimension of the image features, filter out the noise in the feature vector, make the main features in the license plate feature vector more prominent, and reduce the interference information in the features. After the dimensionality reduction process, cluster analysis is performed using the image features after dimensionality reduction.

[0087] Through the processing of the above steps 201-203, the purpose of identifying and mining rare license plate images in license plate images in actual scenes is achieved.

[0088] Step 204, using the text recognition model to perform text recognition on the rare license plate image, using the text recognition result as a pseudo label of the rare license plate image, and performing uncertainty estimation on the pseudo label to obtain an uncertainty estimation result of the pseudo label.

[0089] This step is used to set pseudo labels for rare license plate images and perform uncertainty estimation for the pseudo labels.

[0090] For all rare license plate images, text recognition processing is performed one by one to obtain pseudo labels corresponding to all rare license plate images. Among them, the text recognition processing using the text recognition model can be implemented using various existing text recognition models in an existing manner, which will not be repeated here.

[0091] As mentioned above, in order to mark the role of pseudo-label data in the training of the generative model, it is necessary to estimate the uncertainty of the pseudo-label and determine the uncertainty estimation result of the pseudo-label. Specifically, the smaller the uncertainty of the pseudo-label, the greater the probability that the pseudo-label is the correct label, and the generative model should try to learn more corresponding pseudo-label data; the greater the uncertainty of the pseudo-label, the smaller the probability that the pseudo-label is the correct label, and the generative model should try to learn less corresponding pseudo-label data. The specific uncertainty estimation can set the estimation method according to actual needs. In this embodiment, an exemplary estimation method is given, which uses the idea based on MC Dropout to estimate uncertainty. Specifically, it can be carried out according to the following steps:

[0092] 1) Use the text recognition model to perform multiple text recognition on rare license plate images corresponding to pseudo labels. Each time the text recognition process is processed, some nodes in the text recognition model are randomly closed by dropout.

[0093] The uncertainty estimation process can be performed together with the process of determining pseudo labels by performing text recognition on rare license plate images. In the text recognition process of a rare license plate image A, the text recognition model can be used to perform multiple text recognition processes, wherein pseudo labels are obtained after processing using the complete text recognition model. At the same time, in order to estimate the uncertainty, in other times of text recognition processes using the text recognition model, some nodes in the text recognition model can be randomly closed to complete the text recognition process. The output result obtained is called a predicted label to distinguish it from a pseudo label. The reason for such processing is that when performing text recognition on a license plate image, by closing some nodes of the text recognition model, the corresponding text recognition processing result will be disturbed to a certain extent, so that the output prediction label may deviate from the pseudo label, and the confidence of the corresponding prediction label may also change; in more detail, if the prediction label that is the same as the pseudo label can still be output after some nodes are closed, it means that the stability of the pseudo label is good, and the probability that the pseudo label is the correct label is greater; if the output prediction label and its confidence change greatly after some nodes are closed, it means that the stability of the pseudo label is not very good, and the probability that the pseudo label is the correct label is smaller; based on this, the influence of closing some nodes on the output result can be reflected by the prediction label and its confidence, and the stability of the pseudo label can also be reflected. Therefore, in this embodiment, the uncertainty of the pseudo label is estimated based on the prediction label and its confidence;

[0094] 2) Based on the predicted labels obtained from multiple text recognitions, calculate the confidence mean and confidence variance of the predicted labels;

[0095] 3) Determine the uncertainty estimation result based on the confidence mean and confidence variance.

[0096] In more detail, the mean confidence of the predicted labels The calculation formula can be as follows:

[0097]

[0098] T is the number of predictions of MC Dropout, It is the confidence of the predicted label obtained by performing the t-th text recognition process using the text recognition model, and t is the prediction number index.

[0099] Confidence variance of the predicted labels The calculation formula can be as follows:

[0100]

[0101] in, and are the predicted labels obtained from the t-th and t-1-th text recognition, respectively. is a variance function, which is used to calculate the uncertainty of the predicted label. The specific function form can be set as needed, for example:

[0102]

[0103] in, is the edit distance between the predicted labels obtained by the t-th and t-1-th text recognition, which can usually be the similarity between the two predicted labels. is the length of the predicted label obtained by the tth text recognition, that is, the length of the predicted license plate content.

[0104] The specific calculation formula for determining the uncertainty estimation result based on the confidence mean and confidence variance can be:

[0105]

[0106] Among them, λ is the weight of the confidence mean.

[0107] Step 205: perform data enhancement on the rare license plate image according to the uncertainty estimation result of the pseudo label.

[0108] In order to further expand the richness and quantity of training samples of the rare license plate image generation model, data enhancement can be performed based on rare license plate images to generate more rare license plate images. Specifically, for pseudo-label data with smaller uncertainty represented by the uncertainty estimation results, it indicates that the confidence of the pseudo-label is higher, and more data enhancement can be performed based on these data with higher confidence to generate more training samples; for pseudo-label data with stronger uncertainty represented by the uncertainty estimation results, it indicates that the confidence of the pseudo-label is slightly lower, then less data enhancement is performed based on these data with low confidence to generate fewer training samples to limit its impact on the model training process.

[0109] The specific data enhancement method can adopt various existing methods, such as data enhancement by controlling brightness change, pixel jitter, etc. For the license plate image obtained after data enhancement, the corresponding pseudo label and the uncertainty estimation result of the pseudo label are set to be the same as the original rare license plate image for reference data enhancement.

[0110] Step 206 , using the rare license plate images and their pseudo labels determined in step 203 and the rare license plate images and their pseudo labels generated by data enhancement as training data, to train a generative model for generating rare license plate images.

[0111] In this embodiment, an artificial intelligence generated content (AIGC) generation model is used as a generation model for rare license plate images. The training data uses the actual rare license plate images obtained, the data-enhanced rare license plate images, and the corresponding pseudo labels. Of course, in order to expand the scale of training data and introduce more reliable data, the actual rare license plate images and the real labels marked by humans can usually be used as training data for training and generating the AIGC generation model.

[0112] During the training process of the AIGC generation model, a training label (pseudo label or real label) is input into the AIGC generation model for processing to generate a rare license plate image, which is called the first rare license plate image. The first rare license plate image is compared with the rare license plate image corresponding to the label to determine the value of the loss function. Finally, the parameters of the AIGC generation model are updated based on the value of the loss function, and the next training label is input to generate a rare license plate image, and so on, until the training end condition is met, and the trained AIGC generation model is obtained.

[0113] In the above training process, in order to control the role of the corresponding pseudo-label in the parameter update process based on the uncertainty of the pseudo-label, the uncertainty estimation result of the pseudo-label is used when determining the loss function value. Specifically, based on the uncertainty estimation result of the pseudo-label in the training data, a weight can be set for the pseudo-label, and the loss function of the corresponding training process is weighted according to the weight of the pseudo-label, so that the obtained loss function value reflects the uncertainty of the pseudo-label, thereby controlling the parameter update process. Among them, the stronger the uncertainty of the pseudo-label, the smaller the weight of the corresponding pseudo-label, and the corresponding weighted result of the loss function value is smaller, and the impact on the parameter update process is smaller. For the real label, in order to ensure the consistency of the processing process, a weight can also be set for the real label, and the weight is usually greater than or equal to the weight of the pseudo-label. Thus, the training of the AIGC generation model can be controlled by the uncertainty estimation result of the pseudo-label, so as to obtain a more accurate AIGC generation model.

[0114] Step 207, using the rare license plate image generation model to generate rare license plate images for training the license plate recognition model.

[0115] Step 208: Perform license plate recognition using the license plate recognition model.

[0116] The processing of steps 207 and 208 is the same as that of steps 105 and 106, respectively, and will not be repeated here.

[0117] So far, Figure 2 The method flow shown ends. The license plate recognition method given in the above specific embodiment can obtain low-dimensional color features and high-dimensional semantic features from the actual license plate image, use principal component analysis PCA to reduce dimensionality and noise, and use unsupervised algorithms to mine rare types of license plates from massive unlabeled field data; at the same time, a pseudo-label data joint training scheme based on uncertainty estimation is proposed. According to the Monte Carlo dropout method, the uncertainty of each pseudo-label sample is evaluated, and then according to the uncertainty evaluation result, the data enhancement and loss weight of the pseudo-label sample in the training process are dynamically adjusted, so as to better train the AIGC generation model, and use the generation model to effectively solve the problem of unbalanced license plate type samples, and effectively improve the accuracy of rare license plate recognition.

[0118] The above is the specific implementation of the license plate recognition method in this application. This application also provides a license plate recognition device, which can be used to implement the above license plate recognition method. Figure 3 Figure 1 is a basic structural diagram of a license plate recognition device. Figure 3 As shown, the device includes: a rare license plate extraction unit, a text recognition and uncertainty estimation unit, a rare license plate generation unit and a license plate recognition unit.

[0119] The rare license plate extraction unit is used to obtain license plate images in actual scenes and identify rare license plate images from all acquired license plate images through unsupervised learning;

[0120] A text recognition and uncertainty assessment unit is used to perform text recognition on rare license plate images using a text recognition model, and use the text recognition result as a pseudo label of the rare license plate image; and is also used to perform uncertainty estimation on the pseudo label to obtain an uncertainty estimation result;

[0121] A rare license plate generation unit is used to train a generation model for generating rare license plate images using rare license plate images and their pseudo labels as training data; wherein the loss function value in the training generation model process is determined based on the uncertainty estimation result of the pseudo labels;

[0122] The license plate recognition unit is used to train the license plate recognition model using the generated rare license plate images; and is also used to perform license plate recognition using the license plate recognition model.

[0123] Optionally, in the rare license plate extraction unit, the process of determining the rare license plate image from all acquired license plate images by unsupervised learning may specifically include:

[0124] Extract image features and color vectors from all license plate images; the color vector is used to represent the color distribution law of the license plate image;

[0125] Based on the image features and color vectors of the license plate images, all license plate images are clustered, and license plate images belonging to the rare license plate classification are determined as rare license plate images.

[0126] Optionally, the rare license plate extraction unit can be further used to perform dimensionality reduction processing on the extracted image features using a principal component analysis method; accordingly, clustering is performed based on the image features after the dimensionality reduction processing.

[0127] Optionally, the device may further include a data enhancement unit, used to perform data enhancement on the rare license plate images to generate new license plate images, and use the license plate images newly generated by data enhancement for training the generation model; wherein, the smaller the uncertainty of the pseudo-labels of the rare license plate images, the greater the number of new license plate images generated by data enhancement.

[0128] Optionally, in the text recognition and uncertainty estimation unit, the processing of uncertainty estimation on the pseudo-label may specifically include:

[0129] Use the text recognition model to perform multiple text recognition on rare license plate images corresponding to pseudo labels. Each prediction is made by randomly closing some nodes in the text recognition model through dropout.

[0130] Based on the predicted labels obtained from multiple text recognitions, the confidence mean and variance of the predicted labels are calculated;

[0131] The uncertainty estimation result is determined based on the confidence mean and confidence variance.

[0132] Among them, in the text recognition and uncertainty estimation unit, based on the mean and variance Determine the uncertainty estimate The specific processing can be: ; where λ is the weight of the confidence mean.

[0133] The present application also provides a computer-readable storage medium, which stores instructions, and when the instructions are executed by a processor, the steps in the license plate recognition method described above can be executed. In practical applications, the computer-readable medium can be included in each device / apparatus / system in the above embodiments, or it can exist independently without being assembled into the device / apparatus / system. Instructions are stored in the computer-readable storage medium, and when the instructions stored in the computer-readable storage medium are executed by a processor, the steps in the license plate recognition method described above can be executed.

[0134] According to the embodiments disclosed in the present application, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof, but it is not used to limit the scope of protection of the present application. In the embodiments disclosed in the present application, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0135] Figure 4 This application also provides an electronic device. Figure 4 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0136] The electronic device may include a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, and a computer program stored in the memory and executable on the processor. When the program in the memory 402 is executed, the method for license plate recognition may be implemented.

[0137] Specifically, in practical applications, the electronic device may further include components such as a power supply 403, an input / output unit 404, etc. Those skilled in the art may understand that Figure 4 The structure of the electronic device shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0138] The processor 401 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the server and processes data by running or executing software programs and / or modules stored in the memory 402 and calling data stored in the memory 402, thereby monitoring the electronic device as a whole.

[0139] The memory 402 can be used to store software programs and modules, that is, the above-mentioned computer-readable storage medium. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0140] The electronic device also includes a power supply 403 for supplying power to various components, which can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 403 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, etc.

[0141] The electronic device may also include an input and output unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, and optical signal inputs related to user settings and function control. The input unit output 404 may also be used to display information input by the user or information provided to the user and various graphical user interfaces, which may be composed of graphics, text, icons, videos, and any combination thereof.

[0142] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A license plate recognition method, characterized in that: include: Obtain license plate images in real scenes, and identify rare license plate images among all acquired license plate images through unsupervised learning; Using a text recognition model to perform text recognition on the rare license plate image, and using the text recognition result as a pseudo label of the rare license plate image; Performing uncertainty estimation on the pseudo-label to obtain an uncertainty estimation result; The rare license plate image and its pseudo-label are used as training data to train a generative model for generating rare license plate images; wherein, based on the uncertainty estimation result of the pseudo-label, the weight of the pseudo-label is determined, and the weight is used to perform weighted processing on the loss function in the training process of the generative model, and the stronger the uncertainty represented by the uncertainty estimation result, the smaller the weight; Generate rare license plate images using the rare license plate image generation model for training a license plate recognition model; The license plate recognition model is used to perform license plate recognition.

2. The method according to claim 1, characterized in that The method of determining rare license plate images from all acquired license plate images by unsupervised learning includes: Extracting image features and color vectors from all license plate images; wherein the color vector is used to represent the color distribution law in the license plate image; Based on the image features and color vectors of the license plate images, all license plate images are clustered, and license plate images belonging to the rare license plate classification are determined as rare license plate images.

3. The method according to claim 2, characterized in that After extracting the image features and before clustering, the method further includes: performing dimensionality reduction processing on the extracted image features using principal component analysis; The clustering is performed based on the image features after the dimensionality reduction process.

4. The method according to claim 1, characterized in that After estimating the uncertainty of the pseudo-label, the method further comprises: Based on the uncertainty estimation result of the pseudo-label, data enhancement is performed on the rare license plate image to generate a new license plate image, and the license plate image newly generated by data enhancement is used for training the generation model; wherein, the smaller the uncertainty of the pseudo-label of the rare license plate image, the greater the number of new license plate images generated by data enhancement.

5. The method according to claim 1, characterized in that The estimating uncertainty of the pseudo-label comprises: Using the text recognition model to perform multiple text recognition on the rare license plate image corresponding to the pseudo label, each time the text recognition process is performed, some nodes in the text recognition model are randomly closed by dropout; Based on the predicted labels obtained by the multiple text recognitions, calculating the confidence mean and variance of the predicted labels; The uncertainty estimate is determined based on the mean and the variance.

6. The method according to claim 5, characterized in that The mean value based on and variance Determine the uncertainty estimate ,include: ; Wherein, λ is the weight of the confidence mean.

7. A license plate recognition device, characterized in that: include: Rare license plate extraction unit, text recognition and uncertainty estimation unit, rare license plate generation unit and license plate recognition unit; The rare license plate extraction unit is used to obtain license plate images in actual scenes and identify rare license plate images from all acquired license plate images through unsupervised learning; The text recognition and uncertainty assessment unit is used to perform text recognition on the rare license plate image using a text recognition model, and use the text recognition result as a pseudo label of the rare license plate image; It is also used to estimate the uncertainty of the pseudo-label to obtain an uncertainty estimation result; The rare license plate generation unit is used to train a generation model for generating rare license plate images using the rare license plate images and their pseudo labels as training data; wherein the weight of the pseudo label is determined based on the uncertainty estimation result of the pseudo label, and the weight is used to perform weighted processing on the loss function in the generation model process, and the stronger the uncertainty represented by the uncertainty estimation result, the smaller the weight; The license plate recognition unit is used to train a license plate recognition model using the generated rare license plate image; and is also used to perform license plate recognition using the license plate recognition model.

8. The device according to claim 7, characterized in that In the rare license plate extraction unit, determining rare license plate images from all acquired license plate images through unsupervised learning includes: Extracting image features and color vectors from all license plate images; wherein the color vector is used to represent the color distribution law in the license plate image; Based on the image features and color vectors of the license plate images, all license plate images are clustered, and license plate images belonging to the rare license plate classification are determined as rare license plate images.

9. The device according to claim 8, characterized in that The rare license plate extraction unit is further used to perform dimensionality reduction processing on the extracted image features using the principal component analysis method; and is also used to perform clustering based on the image features after the dimensionality reduction processing.

10. The device according to claim 7, characterized in that The device further includes a data enhancement unit, which is used to perform data enhancement on the rare license plate image to generate a new license plate image, and use the license plate image newly generated by data enhancement for training the generation model; wherein, the smaller the uncertainty of the pseudo-label of the rare license plate image, the greater the number of license plate images newly generated by data enhancement.

11. The device according to claim 7, characterized in that In the text recognition and uncertainty estimation unit, the step of estimating the uncertainty of the pseudo-label includes: Using the text recognition model to perform multiple text recognition on the rare license plate images corresponding to the pseudo labels, and randomly closing some nodes in the text recognition model by dropout for each prediction; Based on the predicted labels obtained by the multiple text recognitions, calculating the confidence mean and variance of the predicted labels; The uncertainty estimate is determined based on the mean and the variance.

12. The device according to claim 11, characterized in that In the text recognition and uncertainty estimation unit, the and variance Determine the uncertainty estimate ,include: ; Wherein, λ is the weight of the confidence mean.

13. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by the processor, the license plate recognition method described in any one of claims 1 to 6 can be implemented.

14. An electronic device, characterized in that: The electronic device includes at least a computer-readable storage medium and also includes a processor; The processor is used to read executable instructions from the computer-readable storage medium and execute the instructions to implement the license plate recognition method described in any one of claims 1 to 6 above.

Images