License plate character classification method and system based on multi-model guidance

Abstract

The present invention discloses a license plate character classification method and system based on multi-model guidance. The method comprises: inputting a first license plate character data set into a first classification model for learning to obtain a first classification probability for each category; inputting the first license plate character data set into an initial license plate character classification model for learning to obtain a first classification probability for each category; inputting a second license plate character data set into a second classification model for learning to obtain a second classification probability for each category; inputting the second license plate character data set into the initial license plate character classification model for learning to obtain a second classification probability for each category; calculating a first divergence value based on the first classification probability and the first license plate character classification probability; calculating a second divergence value based on the second classification probability and the second license plate character classification probability; constructing an overall loss function based on the first divergence value and the second divergence value for training to obtain a license plate character classification model.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a license plate character classification method and system based on multi-model guidance. Background Art

[0002] License plate recognition technology plays an important role in many scenarios, such as urban traffic management, vehicle identification, parking fee management, and violation handling. License plate recognition technology currently has a variety of implementation algorithms. Traditional methods perform two-dimensional target detection of license plate characters based on video images. Traditional methods use a small amount of data, and the amount of data for different character types varies greatly. Some data collection is difficult, resulting in a long-tail distribution of data distribution characteristics when classifying license plate characters. It can also be understood that a small number of character categories occupy the majority of the character data, and a large number of character categories occupy a small number of character data, making the overall data distribution present a long-tail distribution. As a result, when using traditional methods to classify license plate characters, the features of most categories distributed at the tail are not fully learned, which easily leads to classification errors. Summary of the Invention

[0003] The purpose of the present invention is to solve the technical problem of classification errors caused by insufficient feature learning of long-tail distribution data by traditional methods. To achieve the above purpose, the present invention provides a license plate character classification method and system based on multi-model guidance.

[0004] The present invention provides a license plate character classification method based on multi-model guidance, comprising:

[0005] Constructing a first license plate character data group and a second license plate character data group, wherein the number of character categories in the first license plate character data group is greater than the number of character categories in the second license plate character data group;

[0006] Inputting the first license plate character data set into a first classification model for learning to obtain a first classification probability for each category, and inputting the first license plate character data set into an initial license plate character classification model for learning to obtain a first license plate character classification probability for each category;

[0007] Inputting the second license plate character data set into a second classification model for learning to obtain a second classification probability for each category, and inputting the second license plate character data set into the initial license plate character classification model for learning to obtain a second license plate character classification probability for each category; wherein the number of convolution modules of the feature extraction network in the first classification model is less than the number of convolution modules of the feature extraction network in the second classification model;

[0008] Calculating a first scatter value based on the first classification probability and the first license plate character classification probability, and calculating a second scatter value based on the second classification probability and the second license plate character classification probability;

[0009] constructing an overall loss function according to the first divergence value and the second divergence value, and training the initial license plate character classification model to obtain a license plate character classification model;

[0010] The license plate character image is classified according to the license plate character classification model to obtain a classification result.

[0011] In one embodiment, the method further comprises:

[0012] Constructing a third license plate character data set, wherein the number of character categories in the second license plate character data set is greater than the number of character categories in the third license plate character data set;

[0013] Inputting the third license plate character data set into a third classification model for learning to obtain a third classification probability for each category, and inputting the third license plate character data set into the initial license plate character classification model for learning to obtain a third license plate character classification probability for each category;

[0014] Calculating a third divergence value based on the third classification probability and the third license plate character classification probability;

[0015] constructing the overall loss function according to the first divergence value, the second divergence value, and the third divergence value, and training the initial license plate character classification model to obtain a license plate character classification model;

[0016] Among them, the number of convolution modules in the second classification model is less than the number of convolution modules in the third classification model, and the feature extraction network of the initial license plate character classification model is the same as the feature extraction network of the second classification model.

[0017] In one embodiment, before constructing an overall loss function based on the first divergence value, the second divergence value, and the third divergence value and training the initial license plate character classification model to obtain the license plate character classification model, the method further includes:

[0018] Constructing a first verification data set, a second verification data set, and a third verification data set, wherein the number of character categories in the first verification data set is greater than the number of character categories in the second verification data set, and the number of character categories in the second verification data set is greater than the number of character categories in the third verification data set;

[0019] Verifying the initial license plate character classification model based on the first verification data set to obtain a first initial classification accuracy, and verifying the first classification model based on the first verification data set to obtain a first classification accuracy;

[0020] If the first classification accuracy is greater than twice the first initial classification accuracy, the first weight coefficient of the first classification model is 1; if the first classification accuracy is less than twice the first initial classification accuracy, the first weight coefficient is 0.7 to 0.9;

[0021] The overall loss function is constructed according to the first divergence value, the first weight coefficient, the second divergence value, and the third divergence value.

[0022] In one embodiment, the method further comprises:

[0023] Verifying the initial license plate character classification model based on the second verification data set to obtain a second initial classification accuracy, and verifying the second classification model based on the second verification data set to obtain a second classification accuracy;

[0024] If the second classification accuracy is greater than twice the second initial classification accuracy, the second weight coefficient of the second classification model is 1; if the second classification accuracy is less than twice the second initial classification accuracy, the second weight coefficient is 0.7 to 0.9;

[0025] The overall loss function is constructed according to the first divergence value, the first weight coefficient, the second divergence value, the second weight coefficient and the third divergence value.

[0026] In one embodiment, the method further comprises:

[0027] Verifying the initial license plate character classification model based on the third verification data set to obtain a third initial classification accuracy rate, and verifying the third classification model based on the third verification data set to obtain a third classification accuracy rate;

[0028] If the third classification accuracy is greater than twice the third initial classification accuracy, the third weight coefficient of the third classification model is 1; if the third classification accuracy is less than twice the third initial classification accuracy, the third weight coefficient is 0.7 to 0.9;

[0029] The overall loss function is constructed according to the first divergence value, the first weight coefficient, the second divergence value, the second weight coefficient, the third divergence value and the third weight coefficient.

[0030] The present invention provides a license plate character classification system based on multi-model guidance, comprising:

[0031] a data construction module, configured to construct a first license plate character data group and a second license plate character data group, wherein the number of character categories in the first license plate character data group is greater than the number of character categories in the second license plate character data group;

[0032] A first license plate character data group training module is used to input the first license plate character data group into a first classification model for learning to obtain a first classification probability for each category, and input the first license plate character data group into an initial license plate character classification model for learning to obtain a first license plate character classification probability for each category;

[0033] a second license plate character data set training module, configured to input the second license plate character data set into a second classification model for learning to obtain a second classification probability for each category, and input the second license plate character data set into the initial license plate character classification model for learning to obtain a second license plate character classification probability for each category; wherein the number of convolution modules of the feature extraction network in the first classification model is less than the number of convolution modules of the feature extraction network in the second classification model;

[0034] a divergence calculation module, configured to calculate a first divergence value based on the first classification probability and the first license plate character classification probability, and to calculate a second divergence value based on the second classification probability and the second license plate character classification probability;

[0035] a classification model generation module, configured to construct an overall loss function based on the first divergence value and the second divergence value, and train the initial license plate character classification model to obtain a license plate character classification model;

[0036] The classification module is used to classify the license plate character image according to the license plate character classification model to obtain a classification result.

[0037] In one embodiment, the system further comprises:

[0038] The data construction module is used to construct a third license plate character data group, wherein the number of character categories in the second license plate character data group is greater than the number of character categories in the third license plate character data group;

[0039] a third license plate character data group training module, configured to input the third license plate character data group into a third classification model for learning to obtain a third classification probability for each category, and input the third license plate character data group into the initial license plate character classification model for learning to obtain a third license plate character classification probability for each category;

[0040] The divergence calculation module is used to calculate a third divergence value based on the third classification probability and the third license plate character classification probability;

[0041] The classification model generation module is used to construct the overall loss function according to the first divergence value, the second divergence value and the third divergence value, train the initial license plate character classification model, and obtain a license plate character classification model;

[0042] Among them, the number of convolution modules in the second classification model is less than the number of convolution modules in the third classification model, and the feature extraction network of the initial license plate character classification model is the same as the feature extraction network of the second classification model.

[0043] In one embodiment, the system further comprises:

[0044] a verification data construction module, configured to construct a first verification data group, a second verification data group, and a third verification data group, wherein the number of character categories in the first verification data group is greater than the number of character categories in the second verification data group, and the number of character categories in the second verification data group is greater than the number of character categories in the third verification data group;

[0045] a first verification module, configured to verify the initial license plate character classification model based on the first verification data set to obtain a first initial classification accuracy, and to verify the first classification model based on the first verification data set to obtain a first classification accuracy;

[0046] a first weight coefficient acquisition module, configured to, if the first classification accuracy is greater than twice the first initial classification accuracy, set the first weight coefficient of the first classification model to 1; and if the first classification accuracy is less than twice the first initial classification accuracy, set the first weight coefficient to 0.7 to 0.9;

[0047] The classification model generation module is used to construct the overall loss function according to the first divergence value, the first weight coefficient, the second divergence value and the third divergence value.

[0048] In one embodiment, the system further comprises:

[0049] a second verification module, configured to verify the initial license plate character classification model based on the second verification data set to obtain a second initial classification accuracy, and verify the second classification model based on the second verification data set to obtain a second classification accuracy;

[0050] a second weight coefficient acquisition module, configured to, if the second classification accuracy is greater than twice the second initial classification accuracy, set the second weight coefficient of the second classification model to 1; and if the second classification accuracy is less than twice the second initial classification accuracy, set the second weight coefficient to 0.7 to 0.9;

[0051] The classification model generation module is used to construct the overall loss function based on the first divergence value, the first weight coefficient, the second divergence value, the second weight coefficient and the third divergence value.

[0052] In one embodiment, the system further comprises:

[0053] a third verification module, configured to verify the initial license plate character classification model based on the third verification data set to obtain a third initial classification accuracy, and to verify the third classification model based on the third verification data set to obtain a third classification accuracy;

[0054] a third weight coefficient acquisition module, configured to, if the third classification accuracy is greater than twice the third initial classification accuracy, set the third weight coefficient of the third classification model to 1; and if the third classification accuracy is less than twice the third initial classification accuracy, set the third weight coefficient to 0.7 to 0.9;

[0055] The classification model generation module is used to construct the overall loss function based on the first divergence value, the first weight coefficient, the second divergence value, the second weight coefficient, the third divergence value and the third weight coefficient.

[0056] In the above-mentioned license plate character classification method and system based on multi-model guidance, a first license plate character data group and a second license plate character data group are respectively constructed according to the number of each license plate character category, and different first classification models and second classification models are respectively trained using the first license plate character data group and the second license plate character data group. The features of the first license plate character data group and the second license plate character data group of different license plate character categories learned by the first classification model and the second classification model are passed to the initial license plate character classification model. Through guided learning by the first classification model and the second classification model, the initial license plate character classification model gradually acquires the capabilities of the first classification model and the second classification model, and the overall loss function constructed by the first divergence value and the second divergence value is used to regulate the model parameters in the training process, thereby solving the problem of easy classification errors caused by insufficient learning of long-tail distribution data using a single model in the traditional method, and improving the classification accuracy of the license plate character classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1The figure is a flowchart of the steps of the license plate character classification method based on multi-model guidance provided by the present invention.

[0058] Figure 2 It is a structural diagram of the license plate character classification system based on multi-model guidance provided by the present invention. DETAILED DESCRIPTION

[0059] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments.

[0060] See Figure 1 The present invention provides a license plate character classification method based on multi-model guidance, comprising:

[0061] S10, constructing a first license plate character data group and a second license plate character data group, wherein the number of character categories in the first license plate character data group is greater than the number of character categories in the second license plate character data group;

[0062] S20, inputting the first license plate character data set into a first classification model for learning to obtain a first classification probability for each category, and inputting the first license plate character data set into an initial license plate character classification model for learning to obtain a first license plate character classification probability for each category;

[0063] S30, inputting the second license plate character data set into the second classification model for learning to obtain a second classification probability for each category, and inputting the second license plate character data set into the initial license plate character classification model for learning to obtain a second license plate character classification probability for each category; wherein the number of convolution modules of the feature extraction network in the first classification model is less than the number of convolution modules of the feature extraction network in the second classification model;

[0064] S40, calculating a first scatter value based on the first classification probability and the first license plate character classification probability, and calculating a second scatter value based on the second classification probability and the second license plate character classification probability;

[0065] S50, constructing an overall loss function based on the first divergence value and the second divergence value, training the initial license plate character classification model, and obtaining a license plate character classification model;

[0066] S60, classifying the license plate character image according to the license plate character classification model to obtain a classification result.

[0067] In this embodiment, in S10, the first license plate character data group and the second license plate character data group are distinguished according to the number of each license plate character category. The license plate character categories include 86 license plate character categories. The character categories with a large number are divided into a group to form a first license plate character data group. In one embodiment, the first license plate character data group includes character categories such as Arabic numerals 0 to 9 and English letters A. The character categories with a small number are divided into a group to form a second license plate character data group. In one embodiment, the second license plate character data group includes English letters B to Z, Chinese characters such as Beijing and Shanghai. The number corresponding to each character category in the first license plate character data group is greater than the number corresponding to each character category in the second license plate character data group. By constructing the first license plate character data group and the second license plate character data group, the data is grouped and divided.

[0068] In S20 and S30, the amount of data for each character category in the first license plate character data group is large, and most of them are Arabic numerals. When learning the data, it is simpler than the data in the second license plate character data group, and a lightweight first classification model is used to implement classification. The amount of data for each character category in the second license plate character data group is smaller than the amount of data for each character category in the first license plate character data group, and a second classification model that is more complex than the first classification model is used for learning. Both the first classification model and the second classification model include two parts: a feature extraction network and a classifier. The number of convolution modules of the feature extraction network in the first classification model is smaller than the number of convolution modules of the feature extraction network in the second classification model, indicating that the second classification model is more complex than the first classification model and can extract more features. For different data groups, classification models with different structures are used for learning. In one embodiment, the feature extraction network of the first classification model uses a ResNet-18 network based on a residual structure for feature extraction, and performs parameter learning by stacking multiple convolution layers-normalization layers-nonlinear activation function layers. The feature extraction network of the second classification model uses a ResNet-34 network for feature extraction. In one embodiment, the classifier may use a Softmax multi-class classifier to perform classification after feature extraction to obtain a class with the largest probability value.

[0069] The initial license plate character classification model consists of two parts: a feature extraction network and a classifier. The first classification probability, the first license plate character classification probability, the second classification probability, and the second license plate character classification probability are the results of the classifiers of each model. Each category has a predicted probability, and the model's classifier then outputs the category with the highest probability value.

[0070] After the first license plate character data set is input into the first classification model for learning, a first classification probability for each category is obtained. After the first license plate character data set is input into the initial license plate character classification model for learning, a first license plate character classification probability for each category is obtained. This reflects the predicted probability distribution results of the first classification model and the initial license plate character classification model for the first license plate character data set, respectively. The first divergence value corresponding to each category is calculated using the first classification probability and the first license plate character classification probability, thereby achieving training supervision for the first classification model and the initial license plate character classification model.

[0071] Similarly, the second license plate character data set is input into the second classification model for learning, and the second classification probability for each category is obtained. The second license plate character data set is input into the initial license plate character classification model for learning, and the second license plate character classification probability for each category is obtained. This reflects the predicted probability distribution results of the second classification model and the initial license plate character classification model for the second license plate character data set. The second divergence value corresponding to each category is calculated using the second classification probability and the second license plate character classification probability, thereby achieving training supervision for the second classification model and the initial license plate character classification model.

[0072] The first classification probability of each category predicted by the first classification model and the second classification probability of each category predicted by the second classification model are used as supplementary learning references for the initial license plate character classification model. The ability learned by the first classification model and the second classification model is transferred to the initial license plate character classification model through the method in this embodiment. According to the different numbers of the first license plate character data group and the second license plate character data group, the training and learning of classification models of different complexities are carried out to reduce the computational complexity of the model parameters, thereby realizing the transfer of learning capabilities between network structures of different sizes, learning more feature representations between more data categories and within the same data category, and obtaining a license plate character classification model with low computational complexity and strong learning ability through learning and training, which is then used to classify license plate character images to obtain classification results.

[0073] In S40, in one embodiment, when calculating the first divergence value and the second divergence value, KL divergence can be used for calculation. KL divergence, that is, relative entropy, is used to measure the difference between two probability distributions. When the difference between the two probability distributions is smaller, the KL divergence value is smaller, and the loss function value is smaller, which means that the initial license plate character classification model has learned the capabilities of the first classification model and the second classification model. The KL divergence value is expressed as:

[0074]

[0075] Where n is the number of samples, represents the first license plate character classification probability or the second license plate character classification probability corresponding to the initial license plate character classification model, The first classification probability or the second classification probability corresponding to the first classification model or the second classification model is represented. Thus, the first divergence value is calculated based on the first classification probability and the first license plate character classification probability, and the second divergence value is calculated based on the second classification probability and the second license plate character classification probability.

[0076] In S50, an overall loss function is constructed based on the first divergence value and the second divergence value, and the initial license plate character classification model is trained to adjust the model parameters so that the initial license plate character classification model gradually learns the capabilities of the first classification model and the second classification model.

[0077] In S60, the license plate character image can be understood as a character image that needs to be detected. The license plate character classification model obtained through training is used to classify the license plate character image to obtain a corresponding classification result.

[0078] The present invention provides a license plate character classification method based on multi-model guidance. According to the number of each license plate character category, a first license plate character data group and a second license plate character data group are respectively constructed, and different first classification models and second classification models are respectively trained using the first license plate character data group and the second license plate character data group. The features of the first license plate character data group and the second license plate character data group of different license plate character categories learned by the first classification model and the second classification model are passed to the initial license plate character classification model. Through guided learning by the first classification model and the second classification model, the initial license plate character classification model gradually acquires the capabilities of the first classification model and the second classification model, and the overall loss function constructed by the first divergence value and the second divergence value is used to regulate the model parameters in the training process, thereby solving the problem of easy classification errors caused by insufficient learning of long-tail distribution data using a single model in the traditional method, and improving the classification accuracy of the license plate character classification model.

[0079] In S10, the first license plate character data set and the second license plate character data set are derived from license plate character datasets formed from different categories. The license plate character datasets are images of license plate characters of motor vehicles and non-motor vehicles captured through real road traffic scene monitoring. The images cover license plate character images under different viewing angles, different backgrounds, and different lighting intensities, and specifically include different types of license plates, such as cars, new energy vehicles, buses, motorcycles, electric vehicles, and large trucks.

[0080] The license plate character dataset collects image frames containing vehicle license plates from surveillance videos, performs license plate detection through a license plate character object detection algorithm to obtain the position information of each license plate character, and saves the intercepted license plate character images as individual ones from the image frames according to the position information, or intercepts the images of each license plate character manually. Classify each obtained license plate character image. In one embodiment, there are 86 license plate character categories in total, including 10 Arabic numerals (0 to 9), 24 capital letters (A to Z, excluding I and O), and 52 Chinese characters. Among the 86 license plate character categories, the data volumes of different character types vary greatly. The proportion of Arabic numerals is relatively large, with more of the numbers 6 and 8. Capital letters come second, and Chinese characters are the least. License plate characters such as "Tibetan", "Provisional", "Police", "Diplomatic", and "Student" are less in actual life and are difficult to collect data for.

[0081] In one embodiment, the license plate character classification method based on multi-model guidance further includes:

[0082] In S10, construct a third license plate character data group, where the number corresponding to the character category in the second license plate character data group is greater than the number corresponding to the character category in the third license plate character data group;

[0083] S310, input the third license plate character data group into the third classification model for learning to obtain the third classification probability of each category, and input the third license plate character data group into the initial license plate character classification model for learning to obtain the third license plate character classification probability of each category;

[0084] In S40, calculate the third divergence value according to the third classification probability and the third license plate character classification probability;

[0085] In S50, construct an overall loss function according to the first divergence value, the second divergence value, and the third divergence value, and train the initial license plate character classification model to obtain a license plate character classification model;

[0086] Among them, the number of convolutional modules in the feature extraction network of the second classification model is less than the number of convolutional modules in the feature extraction network of the third classification model, and the feature extraction network of the initial license plate character classification model is the same as the feature extraction network of the second classification model. ??

[0087] In this embodiment, the character categories with a small number are further divided into a group to form the third license plate character data group. The number of character categories corresponding to the second license plate character data group is greater than the number of character categories corresponding to the third license plate character data group. The character categories in the third license plate character data group include Chinese character categories such as "New", "Tibet", "Police", "School", "Diplomatic", etc. In S10, the first license plate character data group, the second license plate character data group, and the third license plate character data group are constructed, forming three data groups with different data volumes of large, medium, and small, and corresponding to the first classification model, the second classification model, and the third classification model respectively, realizing three different types of network structures of small, medium, and large.

[0088] For the third license plate character data group, the data volume of each category is small, it is difficult to collect, and most of them are Chinese character license plate character categories, making it difficult to learn. Therefore, by using a larger network structure in the third classification model to learn a small amount of data, it is possible to better learn the features of each category in the third license plate character data group to ensure that more abundant and effective features are extracted. The third classification model includes two parts: a feature extraction network and a classifier. The network structure in the feature extraction network of the third classification model is different from the network structures in the first classification model and the second classification model. In one embodiment, the feature extraction network of the third classification model can adopt a backbone network based on ResNet-50, and the classifier can adopt a soft Max classifier.

[0089] After the third license plate character data group is input into the third classification model for learning, the third classification probability of each category is obtained. After the third license plate character data group is input into the initial license plate character classification model for learning, the third license plate character classification probability of each category is obtained, reflecting the prediction probability distribution results of the third classification model and the initial license plate character classification model for the third license plate character data group respectively. Calculate the third divergence value corresponding to each category through the third classification probability and the third license plate character classification probability to realize the training supervision of the third classification model and the initial license plate character classification model.

[0090] Use the third classification probability of each category predicted by the third classification model as a supplementary learning reference for the initial license plate character classification model. Through the method in this embodiment, transfer the ability learned by the third classification model to the initial license plate character classification model. According to the different quantity situations of the first license plate character data group, the second license plate character data group, and the third license plate character data group, train and learn classification models with different degrees of complexity, reduce the computational complexity of the model parameters, and thus realize the transfer of learning ability between different-sized network structures, learn more feature representations between different data categories and within the same data category, and obtain a license plate character classification model with low computational complexity and strong learning ability through learning and training, and then use it to classify license plate character images to obtain classification results.

[0091] An overall loss function is constructed based on the first divergence value, the second divergence value, and the third divergence value, and the initial license plate character classification model is trained to adjust the model parameters so that the initial license plate character classification model gradually learns the capabilities of the first classification model, the second classification model, and the third classification model. By transferring the features for different license plate character categories learned by the first classification model, the second classification model, and the third classification model to the same initial license plate character classification model for guided learning, the initial license plate character classification model gradually learns the capabilities of the first classification model, the second classification model, and the third classification model, and exceeds the learning capabilities of the first classification model, the second classification model, and the third classification model, thereby improving the overall accuracy of license plate character classification.

[0092] In one embodiment, at S510, an overall loss function is constructed based on the first divergence value, the second divergence value, and the third divergence value to train the initial license plate character classification model. Before obtaining the license plate character classification model, the license plate character classification method based on multi-model guidance further includes:

[0093] S501, constructing a first verification data set, a second verification data set, and a third verification data set, wherein the number of character categories in the first verification data set is greater than the number of character categories in the second verification data set, and the number of character categories in the second verification data set is greater than the number of character categories in the third verification data set;

[0094] S502, verifying the initial license plate character classification model based on the first verification data set to obtain a first initial classification accuracy, and verifying the first classification model based on the first verification data set to obtain a first classification accuracy;

[0095] S503, if the first classification accuracy is greater than twice the first initial classification accuracy, the first weight coefficient of the first classification model is 1; if the first classification accuracy is less than twice the first initial classification accuracy, the first weight coefficient is 0.7 to 0.9;

[0096] In S50 , an overall loss function is constructed according to the first divergence value, the first weight coefficient, the second divergence value, and the third divergence value.

[0097] In this embodiment, a first verification data set, a second verification data set, and a third verification data set are constructed to constrain the weight parameters of the first classification model, the second classification model, and the third classification model, respectively. The weight of the first classification model is constrained by the first verification data set. In one embodiment, during the training of the initial license plate character classification model, one training cycle refers to a training process in which all training data are sampled once. The training cycle can be set to 120 or 180 training cycles, and the training effect of the student model is verified every ten training cycles.

[0098] The first validation data set is input into the initial license plate character classification model for validation, and the predicted data is output. This data is then compared with the first validation data set to obtain a first initial classification accuracy. Similarly, the first validation data set is input into the first classification model for validation, and the predicted data is output. This data is then compared with the first validation data set to obtain a first classification accuracy. Thus, the respective accuracy rates of the two models for the same validation data set are calculated.

[0099] Based on the difference between the first classification accuracy and the first initial classification accuracy, a first weight coefficient is assigned to obtain a weight coefficient for the first classification model. If the first weight coefficient is 1, it means that the initial license plate character classification model can fully trust the guidance of the first classification model. If the first weight coefficient is between 0.7 and 0.9, it means that the initial license plate character classification model can partially trust the guidance of the first classification model, and the initial license plate character classification model can also achieve better classification capabilities.

[0100] In one embodiment, the license plate character classification method based on multi-model guidance further includes:

[0101] S504, verifying the initial license plate character classification model based on the second verification data set to obtain a second initial classification accuracy, and verifying the second classification model based on the second verification data set to obtain a second classification accuracy;

[0102] S505, if the second classification accuracy is greater than twice the second initial classification accuracy, the second weight coefficient of the second classification model is 1; if the second classification accuracy is less than twice the second initial classification accuracy, the second weight coefficient is 0.7 to 0.9;

[0103] In S50, an overall loss function is constructed according to the first divergence value, the first weight coefficient, the second divergence value, the second weight coefficient and the third divergence value.

[0104] In this embodiment, the second validation data set is input into the initial license plate character classification model for validation, and the predicted data is output. This data is then combined with the second validation data set to obtain a second initial classification accuracy. Similarly, the second validation data set is input into the second classification model for validation, and the predicted data is output. This data is then combined with the second validation data set to obtain a second classification accuracy. Thus, the respective accuracy rates of the two models for the same validation data set are calculated.

[0105] Based on the comparison between the second classification accuracy and the second initial classification accuracy, a second weight coefficient is assigned to obtain a weight coefficient for the second classification model. If the second weight coefficient is 1, it means that the initial license plate character classification model can fully trust the guidance of the second classification model. If the second weight coefficient is between 0.7 and 0.9, it means that the initial license plate character classification model can partially trust the guidance of the second classification model, and the initial license plate character classification model can also achieve better classification capabilities.

[0106] In one embodiment, the license plate character classification method based on multi-model guidance further includes:

[0107] S506, verifying the initial license plate character classification model based on the third verification data set to obtain a third initial classification accuracy, and verifying the third classification model based on the third verification data set to obtain a third classification accuracy;

[0108] S507, if the third classification accuracy is greater than twice the third initial classification accuracy, the third weight coefficient of the third classification model is 1; if the third classification accuracy is less than twice the third initial classification accuracy, the third weight coefficient is 0.7 to 0.9;

[0109] In S50 , an overall loss function is constructed according to the first divergence value, the first weight coefficient, the second divergence value, the second weight coefficient, the third divergence value, and the third weight coefficient.

[0110] In this embodiment, the third validation data set is input into the initial license plate character classification model for validation, and the predicted data is output. This data is then combined with the third validation data set to calculate a third initial classification accuracy. Similarly, the third validation data set is input into the third classification model for validation, and the predicted data is output. This data is then combined with the third validation data set to calculate a third classification accuracy. Thus, the respective accuracy rates of the two models for the same validation data set are calculated.

[0111] The third classification accuracy rate is compared with the third initial classification accuracy rate, and a third weight coefficient is assigned to obtain a weight coefficient for the third classification model. If the third weight coefficient is 1, it means that the initial license plate character classification model can fully trust the guidance of the third classification model. If the third weight coefficient is between 0.7 and 0.9, it means that the initial license plate character classification model can partially trust the guidance of the third classification model, and the initial license plate character classification model can also achieve better classification capabilities.

[0112] According to the first divergence value, the first weight coefficient, the second divergence value, the second weight coefficient, the third divergence value and the third weight coefficient, an overall loss function L=∈1L1+∈2L2+∈3L3 is constructed, where ∈1, ∈2, ∈3 represent the first weight coefficient, the second weight coefficient and the third weight coefficient respectively, and L1, L2, L3 represent the first divergence value, the second divergence value and the third divergence value respectively. According to the first divergence value, the first weight coefficient, the second divergence value, the second weight coefficient, the third divergence value and the third weight coefficient, an overall loss function is constructed to train the initial license plate character classification model and adjust the model parameters so that the initial license plate character classification model gradually learns the capabilities of the first classification model, the second classification model and the third classification model.

[0113] See Figure 2 The present invention provides a license plate character classification system 100 based on multi-model guidance. The license plate character classification system 100 based on multi-model guidance includes a data construction module 10, a first license plate character data group training module 20, a second license plate character data group training module 30, a divergence calculation module 40, a classification model generation module 50 and a classification module 60. The data construction module 10 is used to construct a first license plate character data group and a second license plate character data group, and the number of character categories corresponding to the first license plate character data group is greater than the number of character categories corresponding to the second license plate character data group. The first license plate character data group training module 20 is used to input the first license plate character data group into the first classification model for learning to obtain the first classification probability of each category, and input the first license plate character data group into the initial license plate character classification model for learning to obtain the first license plate character classification probability of each category. The second license plate character data group training module 30 is used to input the second license plate character data group into the second classification model for learning to obtain the second classification probability of each category, and input the second license plate character data group into the initial license plate character classification model for learning to obtain the second license plate character classification probability of each category; wherein, the number of convolution modules of the feature extraction network in the first classification model is less than the number of convolution modules of the feature extraction network in the second classification model.

[0114] The divergence calculation module 40 is configured to calculate a first divergence value based on the first classification probability and the first license plate character classification probability, and to calculate a second divergence value based on the second classification probability and the second license plate character classification probability. The classification model generation module 50 is configured to construct an overall loss function based on the first divergence value and the second divergence value, train an initial license plate character classification model, and obtain a license plate character classification model. The classification module 60 is configured to classify the license plate character image based on the license plate character classification model and obtain a classification result.

[0115] In this embodiment, the relevant description of the data construction module 10 can refer to the relevant description of S10 in the above embodiment. The relevant description of the first license plate character data group training module 20 can refer to the relevant description of S20 in the above embodiment. The relevant description of the second license plate character data group training module 30 can refer to the relevant description of S30 in the above embodiment. The relevant description of the divergence calculation module 40 can refer to the relevant description of S40 in the above embodiment. The relevant description of the classification model generation module 50 can refer to the relevant description of S50 in the above embodiment. The relevant description of the classification module 60 can refer to the relevant description of S60 in the above embodiment.

[0116] In one embodiment, the license plate character classification system 100 based on multi-model guidance further includes a data construction module, a third license plate character data set training module, a divergence calculation module, and a classification model generation module. The data construction module is used to construct a third license plate character data set, wherein the number of character categories corresponding to the second license plate character data set is greater than the number of character categories corresponding to the third license plate character data set. The third license plate character data set training module is used to input the third license plate character data set into a third classification model for learning to obtain a third classification probability for each category, and input the third license plate character data set into an initial license plate character classification model for learning to obtain a third license plate character classification probability for each category.

[0117] The divergence calculation module is configured to calculate a third divergence value based on the third classification probability and the third license plate character classification probability. The classification model generation module is configured to construct an overall loss function based on the first divergence value, the second divergence value, and the third divergence value, and to train the initial license plate character classification model to obtain a license plate character classification model. The number of convolutional modules in the second classification model is smaller than the number of convolutional modules in the third classification model, and the feature extraction network of the initial license plate character classification model is the same as the feature extraction network of the second classification model.

[0118] In this embodiment, the relevant description of the data construction module can refer to the relevant description in S10 of the above embodiment. The relevant description of the third license plate character data group training module can refer to the relevant description in S310 of the above embodiment. The relevant description of the divergence calculation module can refer to the relevant description in S40 of the above embodiment. The relevant description of the classification model generation module can refer to the relevant description in S50 of the above embodiment.

[0119] In one embodiment, the license plate character classification system 100 based on multi-model guidance further includes a verification data construction module, a first verification module, a first weight coefficient acquisition module, and a classification model generation module. The verification data construction module is used to construct a first verification data group, a second verification data group, and a third verification data group. The number of character categories corresponding to the first verification data group is greater than the number of character categories corresponding to the second verification data group, and the number of character categories corresponding to the second verification data group is greater than the number of character categories corresponding to the third verification data group. The first verification module is used to verify the initial license plate character classification model based on the first verification data group to obtain a first initial classification accuracy rate, and to verify the first classification model based on the first verification data group to obtain a first classification accuracy rate.

[0120] The first weight coefficient acquisition module is configured to set the first weight coefficient of the first classification model to 1 if the first classification accuracy is greater than twice the first initial classification accuracy, and to set the first weight coefficient to 0.7 to 0.9 if the first classification accuracy is less than twice the first initial classification accuracy. The classification model generation module is configured to construct an overall loss function based on the first divergence value, the first weight coefficient, the second divergence value, and the third divergence value.

[0121] In this embodiment, the relevant description of the verification data construction module can refer to the relevant description of S501 in the above embodiment. The relevant description of the first verification module can refer to the relevant description of S502 in the above embodiment. The relevant description of the first weight coefficient acquisition module can refer to the relevant description of S503 in the above embodiment. The relevant description of the classification model generation module can refer to the relevant description in S50 in the above embodiment.

[0122] In one embodiment, the multi-model guided license plate character classification system 100 further includes a second verification module, a second weight coefficient acquisition module, and a classification model generation module. The second verification module is configured to verify the initial license plate character classification model based on a second verification data set to obtain a second initial classification accuracy, and to verify the second classification model based on the second verification data set to obtain a second classification accuracy.

[0123] The second weight coefficient acquisition module is configured to set the second weight coefficient of the second classification model to 1 if the second classification accuracy is greater than twice the second initial classification accuracy, and to set the second weight coefficient to 0.7 to 0.9 if the second classification accuracy is less than twice the second initial classification accuracy. The classification model generation module is configured to construct an overall loss function based on the first divergence value, the first weight coefficient, the second divergence value, the second weight coefficient, and the third divergence value.

[0124] In this example, the description of the second verification module can refer to the description of S504 in the above embodiment. The description of the second weight coefficient acquisition module can refer to the description of S505 in the above embodiment. The description of the classification model generation module can refer to the description of S50 in the above embodiment.

[0125] In one embodiment, the license plate character classification system 100 based on multi-model guidance also includes a third verification module, a third weight coefficient acquisition module and a classification model generation module. The third verification module is used to verify the initial license plate character classification model according to the third verification data set to obtain a third initial classification accuracy, and to verify the third classification model according to the third verification data set to obtain a third classification accuracy. The third weight coefficient acquisition module is used to: if the third classification accuracy is greater than twice the third initial classification accuracy, the third weight coefficient of the third classification model is 1; if the third classification accuracy is less than twice the third initial classification accuracy, the third weight coefficient is 0.7 to 0.9. The classification model generation module is used to construct an overall loss function based on the first divergence value, the first weight coefficient, the second divergence value, the second weight coefficient, the third divergence value and the third weight coefficient.

[0126] In this embodiment, the description of the third verification module can refer to the description of S506 in the above embodiment. The description of the third weight coefficient acquisition module can refer to the description of S507 in the above embodiment. The description of the classification model generation module can refer to the description of S50 in the above embodiment.

[0127] In the various embodiments described above, the specific order or hierarchy of steps in the disclosed processes is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of this disclosure. The accompanying method claims provide elements of the various steps in an exemplary order and are not intended to be limited to a specific order or hierarchy.

[0128] Those skilled in the art will also appreciate that the various illustrative logical blocks, modules, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of the two. To clearly demonstrate the interchangeability of hardware and software, the various illustrative components, modules, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0129] The various illustrative logic blocks or modules described in the embodiments of the present invention can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0130] The steps of the methods or algorithms described in the embodiments of the present invention may be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. For example, the storage medium may be connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium may also be integrated into the processor. The processor and storage medium may be provided in an ASIC, which may be provided in a user terminal. Alternatively, the processor and storage medium may also be provided in different components in the user terminal.

[0131] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A license plate character classification method based on multi-model guidance, characterized in that: include: Constructing a first license plate character data group and a second license plate character data group, wherein the number of character categories in the first license plate character data group is greater than the number of character categories in the second license plate character data group; Inputting the first license plate character data set into a first classification model for learning to obtain a first classification probability for each category, and inputting the first license plate character data set into an initial license plate character classification model for learning to obtain a first license plate character classification probability for each category; Inputting the second license plate character data set into a second classification model for learning to obtain a second classification probability for each category, and inputting the second license plate character data set into the initial license plate character classification model for learning to obtain a second license plate character classification probability for each category; wherein the number of convolution modules of the feature extraction network in the first classification model is less than the number of convolution modules of the feature extraction network in the second classification model; Calculating a first scatter value based on the first classification probability and the first license plate character classification probability, and calculating a second scatter value based on the second classification probability and the second license plate character classification probability; constructing an overall loss function according to the first divergence value and the second divergence value, and training the initial license plate character classification model to obtain a license plate character classification model; The license plate character image is classified according to the license plate character classification model to obtain a classification result.

2. The license plate character classification method based on multi-model guidance according to claim 1 is characterized in that: The method further comprises: Constructing a third license plate character data set, wherein the number of character categories in the second license plate character data set is greater than the number of character categories in the third license plate character data set; Inputting the third license plate character data set into a third classification model for learning to obtain a third classification probability for each category, and inputting the third license plate character data set into the initial license plate character classification model for learning to obtain a third license plate character classification probability for each category; Calculating a third divergence value based on the third classification probability and the third license plate character classification probability; constructing the overall loss function according to the first divergence value, the second divergence value, and the third divergence value, and training the initial license plate character classification model to obtain a license plate character classification model; Among them, the number of convolution modules in the second classification model is less than the number of convolution modules in the third classification model, and the feature extraction network of the initial license plate character classification model is the same as the feature extraction network of the second classification model.

3. The license plate character classification method based on multi-model guidance according to claim 2 is characterized in that: Before constructing an overall loss function based on the first divergence value, the second divergence value, and the third divergence value, and training the initial license plate character classification model to obtain the license plate character classification model, the method further includes: Constructing a first verification data set, a second verification data set, and a third verification data set, wherein the number of character categories in the first verification data set is greater than the number of character categories in the second verification data set, and the number of character categories in the second verification data set is greater than the number of character categories in the third verification data set; Verifying the initial license plate character classification model based on the first verification data set to obtain a first initial classification accuracy, and verifying the first classification model based on the first verification data set to obtain a first classification accuracy; If the first classification accuracy is greater than twice the first initial classification accuracy, the first weight coefficient of the first classification model is 1; if the first classification accuracy is less than twice the first initial classification accuracy, the first weight coefficient is 0.7 to 0.9; The overall loss function is constructed according to the first divergence value, the first weight coefficient, the second divergence value, and the third divergence value.

4. The license plate character classification method based on multi-model guidance according to claim 3 is characterized in that: The method further comprises: Verifying the initial license plate character classification model based on the second verification data set to obtain a second initial classification accuracy, and verifying the second classification model based on the second verification data set to obtain a second classification accuracy; If the second classification accuracy is greater than twice the second initial classification accuracy, the second weight coefficient of the second classification model is 1; if the second classification accuracy is less than twice the second initial classification accuracy, the second weight coefficient is 0.7 to 0.9; The overall loss function is constructed according to the first divergence value, the first weight coefficient, the second divergence value, the second weight coefficient and the third divergence value.

5. The license plate character classification method based on multi-model guidance according to claim 4 is characterized in that: The method further comprises: Verifying the initial license plate character classification model based on the third verification data set to obtain a third initial classification accuracy rate, and verifying the third classification model based on the third verification data set to obtain a third classification accuracy rate; If the third classification accuracy is greater than twice the third initial classification accuracy, the third weight coefficient of the third classification model is 1; if the third classification accuracy is less than twice the third initial classification accuracy, the third weight coefficient is 0.7 to 0.9; The overall loss function is constructed according to the first divergence value, the first weight coefficient, the second divergence value, the second weight coefficient, the third divergence value and the third weight coefficient.

6. A license plate character classification system based on multi-model guidance, characterized in that: include: a data construction module, configured to construct a first license plate character data group and a second license plate character data group, wherein the number of character categories in the first license plate character data group is greater than the number of character categories in the second license plate character data group; A first license plate character data group training module is used to input the first license plate character data group into a first classification model for learning to obtain a first classification probability for each category, and input the first license plate character data group into an initial license plate character classification model for learning to obtain a first license plate character classification probability for each category; a second license plate character data set training module, configured to input the second license plate character data set into a second classification model for learning to obtain a second classification probability for each category, and input the second license plate character data set into the initial license plate character classification model for learning to obtain a second license plate character classification probability for each category; wherein the number of convolution modules of the feature extraction network in the first classification model is less than the number of convolution modules of the feature extraction network in the second classification model; a divergence calculation module, configured to calculate a first divergence value based on the first classification probability and the first license plate character classification probability, and to calculate a second divergence value based on the second classification probability and the second license plate character classification probability; a classification model generation module, configured to construct an overall loss function based on the first divergence value and the second divergence value, and train the initial license plate character classification model to obtain a license plate character classification model; The classification module is used to classify the license plate character image according to the license plate character classification model to obtain a classification result.

7. The license plate character classification system based on multi-model guidance according to claim 6 is characterized in that: The system further comprises: The data construction module is used to construct a third license plate character data group, wherein the number of character categories in the second license plate character data group is greater than the number of character categories in the third license plate character data group; a third license plate character data group training module, configured to input the third license plate character data group into a third classification model for learning to obtain a third classification probability for each category, and input the third license plate character data group into the initial license plate character classification model for learning to obtain a third license plate character classification probability for each category; The divergence calculation module is used to calculate a third divergence value based on the third classification probability and the third license plate character classification probability; The classification model generation module is used to construct the overall loss function according to the first divergence value, the second divergence value and the third divergence value, train the initial license plate character classification model, and obtain a license plate character classification model; Among them, the number of convolution modules in the second classification model is less than the number of convolution modules in the third classification model, and the feature extraction network of the initial license plate character classification model is the same as the feature extraction network of the second classification model.

8. The license plate character classification system based on multi-model guidance according to claim 7 is characterized in that: The system further comprises: a verification data construction module, configured to construct a first verification data group, a second verification data group, and a third verification data group, wherein the number of character categories in the first verification data group is greater than the number of character categories in the second verification data group, and the number of character categories in the second verification data group is greater than the number of character categories in the third verification data group; a first verification module, configured to verify the initial license plate character classification model based on the first verification data set to obtain a first initial classification accuracy, and to verify the first classification model based on the first verification data set to obtain a first classification accuracy; a first weight coefficient acquisition module, configured to, if the first classification accuracy is greater than twice the first initial classification accuracy, set the first weight coefficient of the first classification model to 1; and if the first classification accuracy is less than twice the first initial classification accuracy, set the first weight coefficient to 0.7 to 0.9; The classification model generation module is used to construct the overall loss function according to the first divergence value, the first weight coefficient, the second divergence value and the third divergence value.

9. The license plate character classification system based on multi-model guidance according to claim 8, characterized in that: The system further comprises: a second verification module, configured to verify the initial license plate character classification model based on the second verification data set to obtain a second initial classification accuracy, and verify the second classification model based on the second verification data set to obtain a second classification accuracy; a second weight coefficient acquisition module, configured to, if the second classification accuracy is greater than twice the second initial classification accuracy, set the second weight coefficient of the second classification model to 1; and if the second classification accuracy is less than twice the second initial classification accuracy, set the second weight coefficient to 0.7 to 0.9; The classification model generation module is used to construct the overall loss function based on the first divergence value, the first weight coefficient, the second divergence value, the second weight coefficient and the third divergence value.

10. The license plate character classification system based on multi-model guidance according to claim 9 is characterized in that: The system further comprises: a third verification module, configured to verify the initial license plate character classification model based on the third verification data set to obtain a third initial classification accuracy, and to verify the third classification model based on the third verification data set to obtain a third classification accuracy; a third weight coefficient acquisition module, configured to, if the third classification accuracy is greater than twice the third initial classification accuracy, set the third weight coefficient of the third classification model to 1; and if the third classification accuracy is less than twice the third initial classification accuracy, set the third weight coefficient to 0.7 to 0.9; The classification model generation module is used to construct the overall loss function based on the first divergence value, the first weight coefficient, the second divergence value, the second weight coefficient, the third divergence value and the third weight coefficient.

Images