A license plate recognition model training method, device, storage medium and electronic device

Abstract

The embodiments of the present invention provide a license plate recognition model training method, device, storage medium and electronic device. The license plate recognition model training method trains the license plate recognition model by using synthetic sample license plates and real sample license plates, overcomes the difficulty of character-level annotation in model training, and solves the problem of low recognition accuracy of consecutive license plates in traditional methods.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computers, and in particular, to a license plate recognition model training method, device, storage medium and electronic device. Background Art

[0002] With the continuous development of deep learning, especially OCR technology, the applications of text detection and text recognition in natural scenes are increasing. License plate recognition, as a representative technology in this field, has also been widely used.

[0003] The training data for license plate recognition is often annotated at the text box level. In addition to artificially synthesized datasets, there are almost no datasets with character-level data annotation. The main reason for this is that character-level data annotation is not only difficult, but also has low annotation accuracy.

[0004] For the above reasons, in the license plate recognition task, it is often necessary to specially design the image features of the text so that each word in the license plate can be accurately recognized even when the text-level data is annotated. Among them, CTC decoding is the most commonly used decoding method.

[0005] CTC decoding is to send the license plate image with an image size of 32×100 into the deep learning network, and finally obtain the deep features of size 25×71 after passing through the CNN convolutional network and the downsampling network. Among them, 25 represents that the deep learning network finally cuts the license plate image into 25 parts horizontally. Because it is impossible to determine the specific position of each character, it can only use this method of sampling more than the number of license plate characters (25>7). Finally, post-processing is performed to ensure that no character is missed. This is also the reason why CTC decoding is not effective for consecutive license plates. Among them, 71 represents the classification probability of each character in the corresponding license plate recognition dictionary.

[0006] The CTC decoding method is to horizontally split the license plate image into longer sequences, and then decode and merge them. This method can meet general license plate recognition requirements, but the recognition accuracy for consecutive license plates is very low. With the rise of new energy vehicles, more and more new energy license plates need to be recognized, and new energy license plates often have consecutive numbers, and the length of new energy license plates is 8 bits, making it more difficult to distinguish between two consecutive characters.

[0007] Therefore, how to solve the above problems is one of the difficulties that needs to be overcome urgently in the field of license plate recognition. Summary of the invention

[0008] In view of this, embodiments of the present invention provide a license plate recognition model training method, device, storage medium and electronic device to at least partially solve the above problems.

[0009] According to a first aspect of an embodiment of the present invention, there is provided a method for training a license plate recognition model, comprising: obtaining a first training data set, wherein each first sample license plate image in the first training data set is composed of single character images spliced ​​in a preset manner;

[0010] The initial network is trained using the first training data set to obtain a first target network model, wherein the process of training the initial network includes performing center regression and feature segmentation on the first sample license plate image;

[0011] Acquire a second training data set, wherein each second sample license plate image in the second training data set is a real license plate image;

[0012] The first target network model is trained using the second training data set to obtain a second target network model, wherein during the training of the first target network model, the network parameters of the second target network model are updated based on the correlation between the real license plate information and the predicted license plate information obtained by the first target network model.

[0013] In some other examples, the initial network is trained using the first training data set to obtain a first target network model, wherein the process of training the initial network includes performing center regression and feature segmentation on the first sample license plate image, including:

[0014] Inputting the first training data set into the initial network;

[0015] Performing center regression on each of the first sample license plate images to obtain the center point of the first sample license plate;

[0016] Based on the center point of the first sample license plate, feature segmentation is performed on the first sample license plate image to obtain a first sample license plate character feature map;

[0017] The first sample license plate character feature map is classified to obtain license plate information, and the network parameters of the first target network model are updated.

[0018] In some other examples, performing center regression on each of the first sample license plate images to obtain the center point of the first sample license plate includes:

[0019] Based on the single character images, confirming the character position information of each single character image;

[0020] Based on the character position information of each single character image, obtaining a corresponding character frame;

[0021] According to the length and width information of the character frame, the center point of each character frame is confirmed, and the center points of all the character frames constitute the center point position of the first sample license plate.

[0022] In some other examples, based on the center point of the first sample license plate, feature segmentation is performed on the first sample license plate image to obtain a first sample license plate character feature map, including:

[0023] Based on the center point of the first sample license plate, a Gaussian distribution map of the center point of the first sample license plate is obtained;

[0024] Down-sampling is performed on the Gaussian distribution map of the center point of the first sample license plate to obtain the character feature map of the first sample license plate.

[0025] In some other examples, the first target network model is trained by the second training data set to obtain a second target network model, wherein, during the training of the first target network model, the network parameters of the second target network model are updated by the association relationship between the real license plate information and the predicted license plate information obtained by the first target network model, including:

[0026] Inputting the second training data set into the first target network model;

[0027] Performing center regression on each of the second sample license plate images through the first target network model to obtain the center point of the second sample license plate;

[0028] Based on the center point of the second sample license plate, the first target network model performs feature segmentation on the second sample license plate image to obtain a second sample license plate character feature map;

[0029] Classifying the second sample license plate character feature map through the first target network model to obtain predicted license plate information;

[0030] Calculating the target accuracy probability based on the real character frame number information and the predicted character frame number information in the predicted license plate information;

[0031] Based on the target accuracy probability, network parameters of the second target network model are updated.

[0032] According to a second aspect of an embodiment of the present invention, there is provided a license plate recognition method, comprising:

[0033] Input the license plate image to be recognized into the target license plate recognition model;

[0034] Through the target license plate recognition model, the center point of the license plate image to be recognized is obtained by performing center regression on the license plate image to be recognized;

[0035] Based on the center point of the license plate image to be identified, the target license plate recognition model is used to perform feature segmentation on the license plate image to be identified, so as to obtain a character feature map of the license plate to be identified;

[0036] The license plate character feature map to be identified is classified to obtain the license plate information.

[0037] According to a third aspect of an embodiment of the present invention, there is provided a license plate recognition model training device, comprising:

[0038] A first acquisition module is used to acquire a first training data set, wherein each first sample license plate image in the first training data set is formed by splicing a single character image in a preset manner;

[0039] A first training module, used to train the initial network using the first training data set to obtain a first target network model, wherein the process of training the initial network includes performing center regression and feature segmentation on the first sample license plate image;

[0040] A second acquisition module is used to acquire a second training data set, wherein each second sample license plate image in the second training data set is a real license plate image;

[0041] The second training module is used to train the first target network model through the second training data set to obtain a second target network model, wherein, during the training of the first target network model, the network parameters of the second target network model are updated through the association relationship between the real license plate information and the predicted license plate information obtained by the first target network model.

[0042] According to a fourth aspect of an embodiment of the present invention, there is provided a license plate recognition device, comprising:

[0043] An input module, used to input the license plate image to be recognized into the target license plate recognition model;

[0044] A center regression module is used to perform center regression on the license plate image to be identified through the target license plate recognition model to obtain the center point of the license plate image to be identified;

[0045] A feature segmentation module is used to perform feature segmentation on the license plate image to be identified based on the center point of the license plate image to be identified by using the target license plate recognition model to obtain a character feature map of the license plate to be identified;

[0046] The classification module is used to classify the license plate character feature map to be identified to obtain the license plate information.

[0047] According to a fifth aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0048] According to a sixth aspect of an embodiment of the present invention, there is further provided an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0049] The license plate recognition model training method provided in the embodiment of the present invention trains the license plate recognition model by using synthetic sample license plates and real sample license plates, thereby overcoming the difficulty of character-level labeling in model training and solving the problem of low recognition accuracy of consecutive license plates in traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art description are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings:

[0051] Figure 1 is a hardware structure block diagram of a mobile terminal of a license plate recognition model training method according to an embodiment of the present invention;

[0052] Figure 2 is a flow chart of a license plate recognition model training method according to an embodiment of the present invention;

[0053] Figure 3 is a schematic diagram of the training process of the first target network model according to an embodiment of the present invention;

[0054] Figure 4 A flowchart of a license plate recognition method according to an embodiment of the present invention;

[0055] Figure 5 is a structural block diagram of a license plate recognition model training device according to an embodiment of the present invention;

[0056] Figure 6 4 is a structural block diagram of a license plate recognition device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in the field based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.

[0058] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0059] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0060] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal of a license plate recognition model training method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0061] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a license plate recognition model training method in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0062] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0063] In order to better solve the problems raised in the above background technology, the present invention discloses a license plate recognition model training method, device, storage medium and electronic device, which will be described in detail one by one in the following embodiments.

[0064] See also Figure 2 , Figure 2 A flowchart of a license plate recognition model training method provided according to an embodiment of this specification is shown, which specifically includes the following steps:

[0065] S202, obtaining a first training data set, wherein each first sample license plate image in the first training data set is formed by splicing a single character image in a preset manner;

[0066] According to an embodiment of the present invention, a first training data set is obtained, wherein each first sample license plate image in the first training data set is formed by splicing a single character image in a preset manner, and specifically, each first sample license plate image in the first training data set is obtained by obtaining different single character images, performing horizontal splicing, and unifying the size of the single character images to obtain a synthesized first sample license plate image. By this method of obtaining sample license plate data, a large number of first sample license plate images with different contents can be obtained.

[0067] According to an embodiment of the present invention, the first training data set also includes a first sample data information group corresponding to each first sample license plate image, wherein the first sample data information group includes: character label information, character position information and character frame number information.

[0068] According to an embodiment of the present invention, in the first sample data information group, the character label information is the specific content of the characters contained in the first sample license plate image, the character position information is the specific position of the characters in the entire first sample license plate image, and the character frame number information is the total number of single character frames contained in the first sample license plate image. For example, the character label information is the specific content of the characters contained in the image license plate, and the character label information corresponding to the single image character can be obtained; wherein the character position information is the specific position of the character in the entire first sample license plate image, for example, the character position information (specific position) of "image character K" in the sample license plate image is the second character position from left to right; the character frame number information is the total number of single character frames contained in the first sample license plate image, for example, the total number of single character frames contained in the sample license plate image is 7.

[0069] It is worth mentioning that the first sample data information group included in the first training data set, including character label information, character position information and character frame number information, can enable the network model to subsequently classify the individual characters contained in each license plate image without the need to horizontally decompose the license plate image into longer sequences and then decode and merge them to ensure content coverage. Although traditional CTC decoding can meet general license plate recognition requirements, the accuracy of recognizing consecutive license plates is very low.

[0070] S204, training the initial network using the first training data set to obtain a first target network model, wherein the process of training the initial network includes performing center regression and feature segmentation on the first sample license plate image;

[0071] According to an embodiment of the present invention, the initial network may be a convolutional neural network, and the first target network model is obtained by training the initial network with the first training data set.

[0072] According to an embodiment of the present invention, in step S204, the initial network is trained using the first training data set to obtain a first target network model, wherein the process of training the initial network includes performing center regression and feature segmentation on the first sample license plate image, and further includes:

[0073] S2042, inputting the first training data set into the initial network;

[0074] S2044, performing center regression on each of the first sample license plate images to obtain the center point of the first sample license plate;

[0075] Among them, in step S2044, center regression is performed on each of the first sample license plate images to obtain the center points of the first sample license plates, including:

[0076] S20442, based on the single-character image, confirm the character position information of each single-character image;

[0077] S20444, based on the character position information of each single-character image, obtain the corresponding character bounding box;

[0078] S20446, according to the length and width information of the character bounding box, confirm the center point of each character bounding box, and the center points of all the character bounding boxes form the center points of the first sample license plates.

[0079] According to an embodiment of the present invention, as Figure 3 shown, it can be seen from step S202 that in the first training dataset, for each character image corresponding to each first sample license plate image, there is corresponding character label information, character position information, and the number of character bounding boxes information. In other words, the character-level license plate information annotation in the training data can obtain the specific position of each character in the license plate. For example, in the sample license plate including the "character image Jing", the character position information of "Jing" can be obtained corresponding to the "character image Jing", so as to know the rectangular box (character bounding box) surrounding the character "Jing", and the center point corresponding to this rectangular box can be obtained through the length and width of this rectangular box. The goal of the initial network learning is to obtain the position of this center point, that is, to perform regression on the center point.

[0080] S2046, based on the center points of the first sample license plates, perform feature segmentation on the first sample license plate images to obtain the first sample license plate character feature maps;

[0081] According to an embodiment of the present invention, among them, step S2046, based on the center points of the first sample license plates, perform feature segmentation on the first sample license plates to obtain the first sample license plate character feature maps, including:

[0082] S20462, based on the center points of the first sample license plates, obtain the Gaussian distribution map of the center points of the first sample license plates;

[0083] S20464, perform downsampling processing on the Gaussian distribution map of the center points of the first sample license plates to obtain the first sample license plate character feature maps.

[0084] According to an embodiment of the present invention, as Figure 3 shown, each character image corresponds to a Gaussian distribution map of the character center point. The red area (the dark area in the middle of the grayscale image) represents the highest score, the blue area (the light area in the middle of the grayscale image) has the lowest score, and the score in the middle area between adjacent characters gradually decreases.

[0085] Among them, for the Gaussian distribution of each center point in the Gaussian distribution map of the center point of the first sample license plate, the positions of the feature maps corresponding to each character can be intercepted. After further downsampling processing, a digital sequence with a length of 71 will ultimately be obtained. This sequence respectively corresponds to the probabilities of the 71 characters of "皖沪津渝冀晋蒙辽吉黑苏浙京闽赣鲁豫鄂湘粤桂琼川贵云藏陕甘青宁新警学挂应急ABCDEFGHJKLMNPQRSTUVWXYZ0123456789-". This character probability distribution map is the first sample license plate feature map.

[0086] S2048. Classify the character feature map of the first sample license plate to obtain license plate information and update the network parameters of the first target network model;

[0087] Among them, step S2048, which classifies the character feature map of the first sample license plate to obtain license plate information and updates the network parameters of the first target network model, includes:

[0088] S20482. Based on the character feature map of the first sample license plate, confirm the target character of each single character picture. The target characters of all single character pictures constitute the license plate information;

[0089] S20484. Based on the character label information, the number of character frame information, and the license plate information, calculate the loss value and update the network parameters of the first target network model.

[0090] According to an embodiment of the present invention, because in the first sample license plate feature map, each character corresponds to a digital sequence with a length of 71. This sequence respectively corresponds to the probabilities of the 71 characters of "皖沪津渝冀晋蒙辽吉黑苏浙京闽赣鲁豫鄂湘粤桂琼川贵云藏陕甘青宁新警学挂应急ABCDEFGHJKLMNPQRSTUVWXYZ0123456789-". Respectively take the character with the highest probability as the target character. For example, if the probability of the first character "京" in the license plate is the highest, then the first target character of the license plate is "京". Perform the same operation on the remaining characters in turn to obtain the final license plate number recognition result, that is, the license plate information. Finally, based on the character label information, the number of character frame information, and the license plate information, calculate the loss value, and through the loss value, confirm and update the network parameters of the first target network model.

[0091] S206. Obtain a second training data set, where each second sample license plate picture in the second training data set is a real license plate picture;

[0092] According to an embodiment of the present invention, the second training data set also includes a second sample data information group corresponding to each second sample license plate image, wherein the second sample data information group at least includes: character frame number information.

[0093] It is worth mentioning that in the embodiment provided by the present invention, the first sample license plate image in the first training data set is a synthetic license plate image composed of a single character. The data features of the synthetic license plate image and the real license plate image are similar but also different. The first target network model obtained through training can only provide limited help for license plate image recognition in real scenes. Therefore, it is also necessary to use the real license plate image as a sample license plate image for model training based on the first target network model.

[0094] Among them, unlike the synthesized first sample license plate image, although the position of a single character frame in the real license plate image is unknown, the number of characters contained in the license plate is known. Therefore, the difference between the number of characters predicted by the trained first target network model and the actual number of characters in the sample license plate can be used to verify the accuracy of the prediction, thereby updating the network parameters of the second target network model.

[0095] S208, training the first target network model using the second training data set to obtain a second target network model, wherein, during the training of the first target network model, the network parameters of the second target network model are updated through the association between the real license plate information and the predicted license plate information obtained by the first target network model.

[0096] According to an embodiment of the present invention, step S208 includes:

[0097] S2082, inputting the second training data set into the first target network model;

[0098] S2084, performing center regression on each of the second sample license plate images through the first target network model to obtain the center point of the second sample license plate;

[0099] S2086: Based on the center point of the second sample license plate, the first target network model performs feature segmentation on the second sample license plate image to obtain a second sample license plate character feature map;

[0100] S2088, classifying the second sample license plate character feature map through the first target network model to obtain predicted license plate information;

[0101] S2090, calculating the target accuracy probability based on the real character frame number information and the predicted character frame number information in the predicted license plate information;

[0102] According to an embodiment of the present invention, in step S2090, the formula for calculating the target accuracy probability is:

[0103]

[0104] Specifically, for example, in the sample license plate image, the actual number of license plate characters is 7, the predicted number of license plate characters is 6, and the target accuracy probability is 6 / 7.

[0105] S2092: Update network parameters of the second target network model based on the target accuracy probability.

[0106] It is worth mentioning that in the embodiment provided by the present invention, the target accuracy probability is used as the basis for updating the network parameters of the second target network model. The higher the target accuracy probability is, the more accurate the prediction result is, and the network parameters need to be updated more in this direction; the lower the target accuracy probability is, the inaccurate the result is, and the network parameters need to be updated less in this direction.

[0107] In another embodiment provided by the present invention, in order to improve the accuracy of model training, a certain number (for example, about 1 / 5) of synthetic license plate images can be mixed into real license plate images as sample license plates in the second training data set for model training.

[0108] It is worth mentioning that because the annotated data for license plate recognition is difficult to achieve character-level annotation, improvements can only be made on the algorithm side. For example, the commonly used CTC decoding uses a sampling number greater than the actual number of characters to achieve the purpose of covering all characters to complete the decoding process. The traditional CTC decoding scheme can achieve general license plate recognition, but for license plates with more consecutive numbers and new energy license plates with more closely arranged numbers, the recognition effect is often very poor or even unrecognizable. The method provided in the embodiment of the present invention, which trains and decodes the license plate recognition network, can achieve accurate positioning and classification of each character in the absence of character-level data annotation. It not only overcomes the difficulties of character-level annotation, but also solves the problem of low accuracy in recognizing consecutive license plates in traditional methods.

[0109] Based on the above license plate recognition model training method, Figure 4 As shown, the present invention also provides a license plate recognition method, which uses the second target license plate recognition model trained in the above embodiment as the target license plate recognition model to recognize the license plate, including the following steps:

[0110] S302, inputting the license plate image to be recognized into the target license plate recognition model;

[0111] S304, performing center regression on the to-be-recognized license plate image through the target license plate recognition model to obtain the center point of the to-be-recognized license plate image;

[0112] S306, based on the center point of the license plate image to be recognized, perform feature segmentation on the license plate image to be recognized by using the target license plate recognition model to obtain a character feature map of the license plate to be recognized;

[0113] S308, classifying the to-be-recognized license plate character feature graph to obtain license plate information.

[0114] The license plate recognition method provided by the present invention can realize accurate positioning and classification of each character in the license plate, thereby solving the problem of low recognition accuracy of consecutive license plates in traditional methods.

[0115] The method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, or by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0116] In this embodiment, a license plate recognition model training device is also provided, which is used to implement the corresponding license plate recognition model training methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. In addition, the functional implementation of each module of the data processing device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0117] According to another embodiment of the present invention, Figure 5 , providing a license plate recognition model training device, comprising:

[0118] A first acquisition module 40 is used to acquire a first training data set, wherein each first sample license plate image in the first training data set is formed by splicing a single character image in a preset manner;

[0119] According to an embodiment of the present invention, the first training data set also includes a first sample data information group corresponding to each first sample license plate image, wherein the first sample data information group includes: character label information, character position information and character frame number information.

[0120] A first training module 50 is used to train the initial network using the first training data set to obtain a first target network model, wherein the process of training the initial network includes performing center regression and feature segmentation on the first sample license plate image;

[0121] According to an embodiment of the present invention, the first training module 50 further includes:

[0122] A first data input unit 52, used to input the first training data set into the initial network;

[0123] A first center regression unit 54 is used to perform center regression on each first sample license plate image to obtain a center point of the first sample license plate;

[0124] A first feature segmentation unit 56 is used to perform feature segmentation on the first sample license plate image based on the center point of the first sample license plate to obtain a first sample license plate character feature map;

[0125] The first feature classification unit 58 is used to classify the first sample license plate character feature map to obtain license plate information and update the network parameters of the first target network model.

[0126] According to an embodiment of the present invention, the first central regression unit 54 further includes:

[0127] A position confirmation unit 542, configured to confirm the character position information of each single character image based on the single character image;

[0128] A character frame acquiring unit 544, configured to acquire a corresponding character frame based on the character position information of each single character image;

[0129] The center point confirmation unit 546 is used to confirm the center point of each character frame according to the length and width information of the character frame, and the center points of all the character frames constitute the center point of the first sample license plate.

[0130] According to an embodiment of the present invention, the first feature segmentation unit 56 further includes:

[0131] A Gaussian distribution map acquisition unit 562, configured to obtain a Gaussian distribution map of a first sample license plate center point based on the first sample license plate center point position;

[0132] The sampling processing unit 564 is used to perform down-sampling processing on the Gaussian distribution map of the center point of the first sample license plate to obtain the first sample license plate character feature map.

[0133] According to an embodiment of the present invention, the first feature classification unit 58 further includes:

[0134] A target character confirmation unit 582 is used to confirm the target character of each single character image based on the first sample license plate character feature map, and the target characters of all single character images constitute the license plate information;

[0135] The loss calculation unit 584 is used to calculate the loss value based on the character label information, the character frame number information and the license plate information, and update the network parameters of the first target network model.

[0136] A second acquisition module 60 is used to acquire a second training data set, wherein each second sample license plate image in the second training data set is a real license plate image;

[0137] According to an embodiment of the present invention, the second training data set also includes a second sample data information group corresponding to each second sample license plate image, wherein the second sample data information group at least includes: character frame number information.

[0138] The second training module 70 is used to train the first target network model using the second training data set to obtain a second target network model, wherein during the training of the first target network model, the network parameters of the second target network model are updated through the association between the real license plate information and the predicted license plate information obtained by the first target network model.

[0139] According to an embodiment of the present invention, the second training module 70 further includes:

[0140] A second data input unit 72, used to input the second training data set into the first target network model;

[0141] A second center regression unit 74 is used to perform center regression on each second sample license plate image through the first target network model to obtain a center point of the second sample license plate;

[0142] A second feature segmentation unit 76 is used to perform feature segmentation on the second sample license plate image based on the center point of the second sample license plate using the first target network model to obtain a second sample license plate character feature map;

[0143] A second feature classification unit 77 is used to classify the second sample license plate character feature map through the first target network model to obtain predicted license plate information;

[0144] A probability calculation unit 78, used to calculate the target accuracy probability based on the real character frame number information and the predicted character frame number information in the predicted license plate information;

[0145] According to an embodiment of the present invention, the formula for calculating the target accuracy probability is:

[0146]

[0147] The parameter updating unit 79 is used to update the network parameters of the second target network model based on the target accuracy probability.

[0148] It is worth mentioning that because the annotated data for license plate recognition is difficult to achieve character-level annotation, improvements can only be made on the algorithm side. For example, the commonly used CTC decoding uses a sampling number greater than the actual number of characters to achieve the purpose of covering all characters to complete the decoding process. The traditional CTC decoding scheme can achieve general license plate recognition, but for license plates with more consecutive numbers and new energy license plates with more closely arranged numbers, the recognition effect is often very poor or even unrecognizable. The method provided in the embodiment of the present invention trains and decodes the license plate recognition network, which can achieve accurate positioning and classification of each character in the absence of character-level data annotation. It not only overcomes the difficulties of character-level annotation, but also solves the problem of low accuracy in recognizing consecutive license plates in traditional methods.

[0149] Based on the above license plate recognition model training device, Figure 6 As shown, the present invention also provides a license plate recognition device, comprising:

[0150] An input module 801 is used to input a license plate image to be recognized into a target license plate recognition model;

[0151] The center regression module 802 is used to perform center regression on the to-be-recognized license plate image through the target license plate recognition model to obtain the center point of the to-be-recognized license plate image;

[0152] The feature segmentation module 803 is used to perform feature segmentation on the license plate image to be identified based on the center point of the license plate image to be identified by using the target license plate recognition model to obtain a character feature map of the license plate to be identified;

[0153] The classification module 804 is used to classify the license plate character feature map to be identified to obtain the license plate information.

[0154] The license plate recognition device provided by the present invention can realize accurate positioning and classification of each character in the license plate, thereby solving the problem of low recognition accuracy of consecutive license plates in traditional methods.

[0155] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0156] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0157] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0158] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0159] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0160] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0161] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A license plate recognition model training method, characterized in that: include: Obtain a first training data set, wherein each first sample license plate image in the first training data set is composed of a single character image spliced ​​in a preset manner, and the first sample data information group in the first training data set includes character label information, character position information, and character frame number information, wherein the character frame number information indicates the number of character frames contained in the first sample license plate image; train an initial network using the first training data set to obtain a first target network model, wherein the process of training the initial network includes performing center regression and feature segmentation on the first sample license plate image; Acquire a second training data set, wherein each second sample license plate image in the second training data set is a real license plate image, and the second sample data information group in the second training data set includes at least real character frame number information, and the real character frame number information represents the number of real character frames contained in the second sample license plate image; train the first target network model through the second training data set to obtain a second target network model, wherein, during the training of the first target network model, the network parameters of the second target network model are updated through the association relationship between the real license plate information and the predicted license plate information obtained by the first target network model; The method further includes: performing central regression and feature segmentation on the second sample license plate image through the first target network model to obtain a second sample license plate character feature map, and classifying the second sample license plate character feature map to obtain predicted license plate information; calculating a target accuracy probability based on the number of real character frames and the number of predicted character frames to update the network parameters of the second target network model, the predicted license plate information includes the number of predicted character frames, and the target accuracy probability is determined by the following formula: Target accuracy probability = 1-(abs(number of real character boxes-number of predicted character boxes)) / number of real character boxes.

2. The method according to claim 1, characterized in that The initial network is trained by using the first training data set to obtain a first target network model, wherein the process of training the initial network includes performing center regression and feature segmentation on the first sample license plate image, including: Inputting the first training data set into the initial network; Performing center regression on each of the first sample license plate images to obtain the center point of the first sample license plate; Based on the center point of the first sample license plate, feature segmentation is performed on the first sample license plate image to obtain a first sample license plate character feature map; The first sample license plate character feature map is classified to obtain license plate information, and the network parameters of the first target network model are updated.

3. The method according to claim 2, characterized in that Performing center regression on each of the first sample license plate images to obtain the center point of the first sample license plate includes: Based on the single character images, confirming the character position information of each single character image; Based on the character position information of each single character image, obtaining a corresponding character frame; According to the length and width information of the character frame, the center point of each character frame is confirmed, and the center points of all the character frames constitute the center point position of the first sample license plate.

4. The method according to claim 2, characterized in that: Based on the center point of the first sample license plate, feature segmentation is performed on the first sample license plate image to obtain a first sample license plate character feature map, including: Based on the center point of the first sample license plate, a Gaussian distribution map of the center point of the first sample license plate is obtained; Down-sampling is performed on the Gaussian distribution map of the center point of the first sample license plate to obtain the character feature map of the first sample license plate.

5. A license plate recognition method, characterized in that: include: Input the license plate image to be recognized into the target license plate recognition model; Through the target license plate recognition model, the center point of the license plate image to be recognized is obtained by performing center regression on the license plate image to be recognized; Based on the center point of the license plate image to be identified, the target license plate recognition model is used to perform feature segmentation on the license plate image to be identified, so as to obtain a character feature map of the license plate to be identified; The license plate character feature map to be identified is classified to obtain the license plate information; wherein the target license plate recognition model is trained by the following steps: Obtain a first training data set, wherein each first sample license plate image in the first training data set is composed of a single character image spliced ​​in a preset manner, and the first sample data information group in the first training data set includes character label information, character position information, and character frame number information, wherein the character frame number information indicates the number of character frames contained in the first sample license plate image; train an initial network using the first training data set to obtain a first target network model, wherein the process of training the initial network includes performing center regression and feature segmentation on the first sample license plate image; Acquire a second training data set, wherein each second sample license plate image in the second training data set is a real license plate image, and the second sample data information group in the second training data set at least includes real character frame number information, and the real character frame number information represents the number of real character frames contained in the second sample license plate image; train the first target network model through the second training data set to obtain a target license plate recognition model, wherein, during the training of the first target network model, the network parameters of the target license plate recognition model are updated through the association relationship between the real license plate information and the predicted license plate information obtained by the first target network model; The method further includes: performing central regression and feature segmentation on the second sample license plate image through the first target network model to obtain a second sample license plate character feature map, and classifying the second sample license plate character feature map to obtain predicted license plate information; calculating a target accuracy probability based on the number of real character frames and the number of predicted character frames to update the network parameters of the target license plate recognition model, the predicted license plate information includes the number of predicted character frames, and the target accuracy probability is determined by the following formula: Target accuracy probability = 1-(abs(number of real character boxes-number of predicted character boxes)) / number of real character boxes.

6. A license plate recognition model training device, characterized in that: include: A first acquisition module is used to acquire a first training data set, wherein each first sample license plate image in the first training data set is composed of a single character image spliced ​​in a preset manner, and the first sample data information group in the first training data set includes character label information, character position information and character frame number information, and the character frame number information indicates the number of character frames contained in the first sample license plate image; A first training module, used to train the initial network using the first training data set to obtain a first target network model, wherein the process of training the initial network includes performing center regression and feature segmentation on the first sample license plate image; A second acquisition module is used to acquire a second training data set, wherein each second sample license plate image in the second training data set is a real license plate image, and the second sample data information group in the second training data set at least includes real character frame number information, and the real character frame number information indicates the number of real character frames contained in the second sample license plate image; A second training module is used to train the first target network model using the second training data set to obtain a second target network model, wherein, during the training of the first target network model, the network parameters of the second target network model are updated according to the association relationship between the real license plate information and the predicted license plate information obtained by the first target network model; The device is also used to: perform central regression and feature segmentation on the second sample license plate image through the first target network model to obtain a second sample license plate character feature map, and classify the second sample license plate character feature map to obtain predicted license plate information; calculate the target accuracy probability based on the number of real character frames and the number of predicted character frames to update the network parameters of the second target network model, the predicted license plate information includes the number of predicted character frames, and the target accuracy probability is determined by the following formula: Target accuracy probability = 1-(abs(number of real character boxes-number of predicted character boxes)) / number of real character boxes.

7. A license plate recognition device, characterized in that: include: An input module, used to input the license plate image to be recognized into the target license plate recognition model; A center regression module is used to perform center regression on the license plate image to be identified through the target license plate recognition model to obtain the center point of the license plate image to be identified; A feature segmentation module is used to perform feature segmentation on the license plate image to be identified based on the center point of the license plate image to be identified by using the target license plate recognition model to obtain a character feature map of the license plate to be identified; The classification module is used to classify the license plate character feature map to be identified to obtain the license plate information; wherein the device is used to train the target license plate recognition model through the following steps: Obtain a first training data set, wherein each first sample license plate image in the first training data set is composed of a single character image spliced ​​in a preset manner, and the first sample data information group in the first training data set includes character label information, character position information, and character frame number information, wherein the character frame number information indicates the number of character frames contained in the first sample license plate image; train an initial network using the first training data set to obtain a first target network model, wherein the process of training the initial network includes performing center regression and feature segmentation on the first sample license plate image; Acquire a second training data set, wherein each second sample license plate image in the second training data set is a real license plate image, and the second sample data information group in the second training data set at least includes real character frame number information, and the real character frame number information represents the number of real character frames contained in the second sample license plate image; train the first target network model through the second training data set to obtain a target license plate recognition model, wherein, during the training of the first target network model, the network parameters of the target license plate recognition model are updated through the association relationship between the real license plate information and the predicted license plate information obtained by the first target network model; The device is also used to: perform central regression and feature segmentation on the second sample license plate image through the first target network model to obtain a second sample license plate character feature map, and classify the second sample license plate character feature map to obtain predicted license plate information; calculate the target accuracy probability based on the number of real character frames and the number of predicted character frames to update the network parameters of the target license plate recognition model, the predicted license plate information includes the number of predicted character frames, and the target accuracy probability is determined by the following formula: Target accuracy probability = 1-(abs(number of real character boxes-number of predicted character boxes)) / number of real character boxes.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 5 when executed.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 5.

Images