License plate recognition method and device

Abstract

The present disclosure provides a license plate recognition method and apparatus, comprising: obtaining a license plate image to be recognized; inputting the license plate image to be recognized into a license plate detection model to obtain position information of a license plate region in the license plate image to be recognized, output by the license plate detection model; obtaining an image of the license plate region in the license plate image to be recognized based on the position information; inputting the image of the license plate region into a license plate recognition model to obtain a license plate recognition prediction result output by the license plate recognition model; wherein the license plate recognition prediction result is output in the form of a character combination of a preset number of digits. The license plate character prediction result is achieved by performing image detection on the license plate image to be recognized by the license plate detection model to obtain an image corresponding to the license plate region, and then performing image recognition on the license plate region image including the license plate characters through the license plate recognition model, thereby achieving direct end-to-end recognition of the license plate character result through image processing, thereby improving recognition accuracy.

Description

Technical Field

[0001] The present disclosure relates to the field of image recognition technology, and in particular to a license plate recognition method and device. Background Art

[0002] License plate recognition is currently a widely used technology. Whether it is public transportation monitoring or community security monitoring and management, cameras and management systems for license plate recognition are generally installed, which greatly facilitates the rapid identification of license plates.

[0003] Existing license plate recognition methods typically use a segmentation-then-recognition approach, first detecting the position of individual characters, then recognizing each character, and finally generating a license plate recognition result. Because errors accumulate during each individual character recognition phase, this often results in reduced accuracy for the final full-plate recognition. Summary of the Invention

[0004] The present disclosure provides a license plate recognition method and device to solve the problem that the existing technology generally adopts the method of first segmenting and then recognizing, which accumulates errors in each sub-stage and leads to a low overall recognition rate. It realizes direct license plate recognition in an end-to-end manner and improves the recognition rate.

[0005] The present disclosure provides a license plate recognition method, comprising:

[0006] Get the license plate image to be recognized;

[0007] Inputting the license plate image to be identified into a license plate detection model, and obtaining position information of a license plate region in the license plate image to be identified output by the license plate detection model; wherein the license plate detection model is trained based on a first sample image, a target detection object in the first sample image, and a position information label of the target detection object;

[0008] Acquire an image of the license plate area in the license plate image to be identified according to the position information;

[0009] The image of the license plate area is input into a license plate recognition model to obtain a license plate recognition prediction result output by the license plate recognition model; wherein the license plate recognition prediction result is output in the form of a character combination of a preset number of digits; wherein the license plate recognition model is obtained based on training of a second sample image and character labels of the second sample image.

[0010] According to a license plate recognition method provided by the present disclosure, inputting the license plate image to be recognized into a license plate detection model and obtaining the position information of the license plate area in the license plate image to be recognized output by the license plate detection model specifically includes:

[0011] The license plate image to be identified is input into the license plate detection model to obtain the position information of the rectangular area corresponding to the license plate area in the license plate image to be identified output by the license plate detection model, wherein the position information includes: the coordinates of the center point of the rectangular area and the size data of the rectangular area.

[0012] According to a license plate recognition method provided by the present disclosure, the image of the license plate area in the license plate image to be recognized is obtained based on the position information, specifically including: based on the position information of the rectangular area in the license plate image to be recognized, the image corresponding to the rectangular area in the license plate image to be recognized is cut out to obtain the image of the license plate area.

[0013] According to a license plate recognition method provided by the present disclosure, the image of the license plate area in the license plate image to be recognized is obtained according to the position information, and then the method further includes: positionally transforming the license plate area image according to the position information of the license plate area image to obtain a position-corrected license plate area image.

[0014] According to a license plate recognition method provided by the present disclosure, the position of the license plate area image is transformed according to its position information to obtain a position-corrected license plate area image, specifically comprising:

[0015] Inputting the license plate area image into the spatial transformation network model to obtain the position transformation parameters output by the spatial transformation network model;

[0016] The license plate area image is position-transformed according to the position transformation parameters to obtain a position-corrected license plate area image.

[0017] According to a license plate recognition method provided by the present disclosure, inputting the image of the license plate area into a license plate recognition model to obtain the license plate recognition prediction result output by the license plate recognition model specifically includes: inputting the position-corrected license plate area image into the license plate recognition model to obtain the license plate recognition prediction result output by the license plate recognition model.

[0018] According to a license plate recognition method provided by the present disclosure, the training process of the license plate recognition model specifically includes: randomly masking each character in the second sample image according to a preset probability, and modifying the corresponding character label to strengthen the training of the license plate recognition model.

[0019] The present disclosure also provides a license plate recognition device, comprising:

[0020] An image acquisition unit, used to acquire an image of a license plate to be identified;

[0021] a license plate region position acquisition unit, configured to input the license plate image to be identified into a license plate detection model, and obtain position information of the license plate region in the license plate image to be identified output by the license plate detection model; wherein the license plate detection model is trained based on a first sample image, a target detection object in the first sample image, and a position information label of the target detection object;

[0022] a license plate area image acquisition unit, configured to acquire an image of the license plate area in the license plate image to be identified according to the position information;

[0023] A license plate recognition unit is used to input the image of the license plate area into a license plate recognition model to obtain a license plate recognition prediction result output by the license plate recognition model; wherein the license plate recognition prediction result is output in the form of a character combination of a preset number of digits; wherein the license plate recognition model is obtained based on training of the second sample image and the character label of the second sample image.

[0024] The present disclosure also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-described license plate recognition methods when executing the program.

[0025] The present disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of any of the above-mentioned license plate recognition methods are implemented.

[0026] The license plate recognition method and device provided by the present disclosure obtain an image of the corresponding license plate area by subjecting the image of the license plate to be recognized to image detection through a license plate detection model, and then obtain an image of the license plate area including the license plate characters through the license plate recognition model to perform image recognition to achieve a prediction result of the license plate characters, thereby realizing direct end-to-end recognition of the license plate characters through images, thereby improving the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the present disclosure or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0028] Figure 1 It is a flowchart of the license plate recognition method provided by the present disclosure;

[0029] Figure 2 yes Figure 1 A flow chart of the image position transformation process after step 130;

[0030] Figure 3 It is a structural diagram of the license plate recognition device provided by the present disclosure;

[0031] Figure 4 It is a structural diagram of the electronic device provided by the present disclosure. DETAILED DESCRIPTION

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the embodiments of the present disclosure.

[0033] License plate recognition (LPR) is a challenging and important task in urban traffic management, video surveillance, vehicle identification, and parking management. The problem is complicated by image clarity, lighting conditions, weather conditions, image deformation, and the variability of license plate characters. A robust LPR system should be able to handle these diverse environments without sacrificing accuracy.

[0034] The following combination Figure 1-Figure 2 Describe the license plate recognition method of the embodiment of the present disclosure, such as Figure 1 As shown, the embodiment of the present disclosure provides a license plate recognition method, comprising:

[0035] Step 110: Acquire the license plate image to be recognized;

[0036] The license plate recognition method of the disclosed embodiment is applicable to character recognition of double-row license plates, and is also applicable to character recognition of single-row license plates. Therefore, the image in this step can be an image of a single-row license plate to be recognized, or an image of a double-row license plate to be recognized.

[0037] Step 120: Inputting the license plate image to be recognized into a license plate detection model to obtain position information of the license plate region in the license plate image to be recognized output by the license plate detection model; wherein the license plate detection model is trained based on a first sample image, a target detection object in the first sample image, and a position information label of the target detection object;

[0038] In the embodiment of the present disclosure, inputting the license plate image to be identified into a license plate detection model to obtain the position information of the license plate area in the license plate image to be identified output by the license plate detection model specifically includes:

[0039] The license plate image to be identified is input into the license plate detection model to obtain the position information of the rectangular area corresponding to the license plate area in the license plate image to be identified output by the license plate detection model, wherein the position information includes: the coordinates of the center point of the rectangular area and the size data of the rectangular area.

[0040] Specifically, in the disclosed embodiment, the license plate detection model involved in this step is a trained model used to extract features from the license plate image to be identified and obtain the character area in the image. For example, the license plate detection model can use the trained Yolo5m network to directly detect the license plate and obtain the rectangular position information of the license plate: (center_x, center_y, width, height). The Yolo network is a fast and compact open source object detection model and a deep learning target detection architecture based on convolutional neural networks. It is a single network trained end-to-end that can be used to predict the target category and bounding box. The Yolo network is fast and can process images in real time.

[0041] The Yolo network mainly consists of three parts:

[0042] Backbone: A convolutional neural network that aggregates and forms image features at different image granularity levels;

[0043] Neck: A series of network layers that mix and combine image features and pass them to the prediction layer (Head);

[0044] Head: Predicts image features, generates bounding boxes and predicts categories.

[0045] In some embodiments, for example, the resolution of the license plate image to be recognized is 1080p, and the resolution of the image is compressed to 640×640 before being input into the license plate detection model for processing.

[0046] Step 130: Acquire an image of the license plate area in the license plate image to be recognized according to the position information;

[0047] In the disclosed embodiment, obtaining an image of the license plate region in the license plate image to be recognized based on the position information specifically includes: based on the position information of the rectangular region in the license plate image to be recognized, extracting an image corresponding to the rectangular region in the license plate image to be recognized to obtain an image of the license plate region. Specifically, based on the result output in step 120, the license plate region is directly cut out for subsequent operations.

[0048] In the disclosed embodiment, the Yolov5m network used in the license plate detection model can crop the detected image of the corresponding position information area to obtain an image of the license plate area.

[0049] During the training stage of the license plate detection model, for example, images of 1,000 license plates are used as the first sample image training set, and the corresponding license plate position of each first sample image is marked as a label. The license plate detection model is trained, and the iteration is stopped when the set number of iterations is completed or the model loss function reaches a preset value. The model trained at this time is used as the trained license plate detection model.

[0050] Image augmentation involves creating new training samples from existing training data. Since it's impossible to capture an image for every real-world scenario, existing training data needs to be adapted to generalize to other situations, making the model adaptable to a wider range of situations. The Yolov5m network employed in this disclosure utilizes diverse and advanced data augmentation techniques, which are key to maximizing dataset utilization and achieving performance breakthroughs in the object detection framework. Through a series of image augmentation techniques, model performance can be improved without increasing inference latency.

[0051] In the disclosed embodiment, the process of obtaining an image of the license plate area in the license plate image to be identified based on the position information further includes: performing a positional transformation on the license plate area image based on the positional information to obtain a position-corrected license plate area image. In the disclosed embodiment, because the license plate area output in step 120 is positional information for a rectangular area, rather than for a conventional quadrilateral area, the license plate area image cut out from the rectangular area generally includes a certain amount of background. Therefore, particularly when the license plate image is captured at a relatively large angle, significant rotational or projective deformation may occur, necessitating correction of the license plate area image.

[0052] In the disclosed embodiment, the process of performing image position transformation after obtaining the image of the license plate area after step 130 is an optional process step, but the license plate area image after position correction can obtain the best input image of the license plate recognition model.

[0053] Specifically, the license plate area image is transformed according to its position to obtain a position-corrected license plate area image, such as Figure 2 Shown, including:

[0054] Step 210: Inputting the license plate area image into the spatial transformation network model to obtain the position transformation parameters output by the spatial transformation network model;

[0055] Specifically, the spatial transformation network model used in the embodiments of the present disclosure is the STNet network model, that is, the spatial transformation network.

[0056] The STNet network model consists of two convolutional layers and two fully connected layers. The convolutional layers are connected in sequence to the pooling layer and the ReLU activation function. The fully connected layers are connected to the ReLU activation function. After the license plate area image is input into the STNet network model, 6 angle parameters are output.

[0057] Step 220: Transform the license plate area image according to the position transformation parameters to obtain a position-corrected license plate area image.

[0058] Specifically, the position of the license plate area image is adjusted according to the angle parameters obtained in step 210 to obtain an adjusted corrected image.

[0059] In the disclosed embodiments, during the STNet network model training process, no loss is calculated and no explicit supervision is performed. The network parameters are automatically optimized under the global loss. Generally, a license plate image corrected by STNet will output a full, positive license plate image. In the disclosed embodiments, the resolution of the image input to and output from the STNet network model is both 192×64.

[0060] Step 140: Input the image of the license plate area into a license plate recognition model to obtain a license plate recognition prediction result output by the license plate recognition model; wherein the license plate recognition prediction result is output in the form of a character combination of a preset number of digits; wherein the license plate recognition model is obtained based on training of the second sample image and the character label of the second sample image.

[0061] In the disclosed embodiment, inputting the image of the license plate area into the license plate recognition model to obtain the license plate recognition prediction result output by the license plate recognition model specifically includes: inputting the position-corrected license plate area image into the license plate recognition model to obtain the license plate recognition prediction result output by the license plate recognition model.

[0062] Specifically, the rectified license plate area image is input into the license plate recognition model, and then after multiple layers of convolution, a 7-character license plate recognition prediction result is directly output. Because the length of the license plate is 7 characters regardless of whether it is a single or double row license plate, with the exception of special license plates (such as the armed police license plate WJ), the license plate recognition model outputs a 7-character result in the disclosed embodiment.

[0063] Specifically, in the embodiment of the present disclosure, the architecture of the license plate recognition model mainly includes the following connected in sequence: input, convolution layer, maximum pooling layer, Small basic block layer, maximum pooling layer, two Small basic block layers, maximum pooling layer, Dropout layer, two convolution layers, Dropout layer, convolution layer, and output.

[0064] Specifically, the second sample image of 192×64 pixels RGB is used for the license plate recognition model, and the training parameters are set as:

[0065] First convolutional layer: #64 3×3 stride 2;

[0066] First max pooling layer: #64 3×3 stride 1;

[0067] First Small basic block layer: #128 3×3 stride 1;

[0068] Second max pooling layer: #64 3x3 stride(2,1);

[0069] Second Small basic block layer: #256 3x3 stride 1;

[0070] Third Small basic block layer: #256 3x3 stride 1;

[0071] Third max pooling layer: #64 3x3 stride 2;

[0072] First Dropout layer: 0.5ratio;

[0073] Second convolutional layer: #64 5x1 stride 1;

[0074] Third convolutional layer: #256 1x5 stride 1;

[0075] Second Dropout layer: 0.5ratio;

[0076] Fourth convolutional layer: #class_number 1x7 stride 1.

[0077] In this disclosed embodiment, the resolution of the image input to the license plate recognition model is 192×64, and padding is used in each layer of the license plate recognition model to ensure that the input size changes stably before and after each layer. In this disclosed embodiment, through the rational design of the backbone network and the connected convolutional layers between the input and output, the output maintains a width dimension of 1. The convolutional layer parameters control the characters corresponding to different channels at different positions, thus effectively solving the license plate recognition problem.

[0078] In the disclosed embodiment, the images in the training sample set during the training process of the license plate recognition model are grayscale and size-normalized images. In some embodiments, an enhanced training method is adopted during the training process of the license plate recognition model, specifically including: randomly masking each character in the second sample image according to a preset probability, and modifying the corresponding character labels to perform enhanced training on the license plate recognition model.

[0079] Specifically, during the training of the license plate recognition model, the training sample set includes both training samples of license plate images and training samples of single-row license plate images, and noise and disturbance information are added to make it as close to the actual acquisition results as possible. During the training process, a single character in the training sample is masked with a preset probability (for example, a probability of 0.2), and its corresponding label is updated at the same time, thereby updating the training model parameters. This enhancement method can improve the accuracy of character position detection. In real scenarios, since the position of a single character is not calibrated, fine-tuning can be performed using the normal training method.

[0080] The license plate recognition method of the disclosed embodiment does not pre-segment the license plate characters, making it an end-to-end license plate recognition method with high accuracy and strong real-time algorithm performance. The convolutional neural network used in the license plate recognition model of the disclosed embodiment has demonstrated effectiveness and superiority in computer vision tasks such as image classification, object detection, and semantic segmentation.

[0081] In the early general license plate (LP) recognition work, the processing flow includes two stages: character segmentation and character classification:

[0082] Character segmentation usually uses different handcrafted algorithms that combine projection, connection, and contour-based image components. It takes a binary image or intermediate representation as input, so the character segmentation quality is highly affected by the noise, resolution, blur, or deformation of the input image.

[0083] Character classification usually uses optical character recognition (OCR) methods for LP character sets.

[0084] Because character segmentation precedes classification, the quality of end-to-end recognition depends largely on the quality of the segmentation method used. To address the character segmentation problem, an embodiment of the present invention provides an end-to-end convolutional neural network (CNN)-based solution that takes the entire LP image as input and produces an output character sequence.

[0085] The segmentation-free model is based on variable-length sequence decoding driven by a temporally connected classification (CTC) loss. It uses handcrafted LBP features built on binarized images as input to a CNN to produce character class probabilities. This is applied to all input image positions via a sliding window method, whose output is used as input to an LSTM-based decoder. Because the decoder output and target character sequence lengths are different, CTC requires no pre-segmentation for end-to-end training.

[0086] In the existing license plate recognition work, the general processing flow includes two stages: character segmentation and character classification;

[0087] Since character segmentation is performed before classification, the end-to-end recognition quality depends largely on the quality of the applied segmentation method. Errors accumulate in each sub-stage of character segmentation, resulting in a low overall recognition rate.

[0088] The license plate recognition device provided by the embodiment of the present disclosure is described below. The license plate recognition device described below and the license plate recognition method described above can be referenced to each other.

[0089] like Figure 3 As shown, an embodiment of the present disclosure provides a license plate recognition device, comprising:

[0090] An image acquisition unit 310 is used to acquire an image of a license plate to be recognized;

[0091] The license plate recognition device of the disclosed embodiment is suitable for character recognition of double-row license plates, and is also suitable for character recognition of single-row license plates. Therefore, the image acquired by the image acquisition unit 310 can be an image of a single-row license plate to be recognized, or an image of a double-row license plate to be recognized.

[0092] The license plate region position acquisition unit 320 is configured to input the license plate image to be recognized into a license plate detection model to obtain the position information of the license plate region in the license plate image to be recognized output by the license plate detection model; wherein the license plate detection model is trained based on a first sample image, a target detection object in the first sample image, and a position information label of the target detection object;

[0093] In the embodiment of the present disclosure, the license plate area position acquisition unit 320 inputs the license plate image to be identified into the license plate detection model, and obtains the position information of the rectangular area corresponding to the license plate area in the license plate image to be identified output by the license plate detection model, wherein the position information includes: the coordinates of the center point of the rectangular area and the size data of the rectangular area.

[0094] Specifically, in the embodiment of the present disclosure, the license plate detection model applied by the license plate area position acquisition unit 320 is a trained model for performing feature extraction on the license plate image to be identified and obtaining the character area in the image. For example, the license plate detection model can use the trained YOLOv5m network to directly perform license plate detection and obtain the license plate rectangular position information: (center_x, center_y, width, height).

[0095] In some embodiments, for example, if the resolution of the license plate image to be identified is 1080p, the license plate area position acquisition unit 320 compresses the resolution of the license plate image to be identified to 640×640 before inputting it into the license plate detection model for processing.

[0096] The license plate area image acquisition unit 330 is used to acquire an image of the license plate area in the license plate image to be recognized according to the position information;

[0097] In the disclosed embodiment, the license plate region image acquisition unit 330 extracts the image corresponding to the rectangular region in the license plate image to be recognized based on the position information of the rectangular region in the license plate image to be recognized, thereby obtaining an image of the license plate region. Specifically, based on the result output by the license plate region position acquisition unit 320, the license plate region is directly cut out for subsequent operations.

[0098] In the embodiment of the present disclosure, the license plate recognition device further includes an image correction unit for performing position transformation on the license plate area image according to its position information to obtain a position-corrected license plate area image.

[0099] In the disclosed embodiment, the license plate area position acquisition unit 320 outputs the position information of a rectangular area, rather than a conventional quadrilateral area. Therefore, under these conditions, the license plate area image cut out from the rectangular area generally includes a certain amount of background. Therefore, especially when the license plate image is captured at a large angle, there will be significant rotational or projective distortion, necessitating correction of the license plate area image by the image correction unit.

[0100] In the embodiment of the present disclosure, the image correction unit specifically includes:

[0101] A transformation parameter acquisition subunit is used to input the license plate area image into the spatial transformation network model to obtain the position transformation parameters output by the spatial transformation network model;

[0102] Specifically, in the disclosed embodiments, the image correction unit uses an STNet network model, also known as a spatial transform network. The STNet model consists of two convolutional layers and two fully connected layers. The convolutional layers are followed by a pooling layer and a ReLU activation function, and the fully connected layers are followed by a ReLU activation function. The STNet model inputs the license plate area image and outputs six angle parameters.

[0103] The position transformation subunit is used to transform the license plate area image according to the position transformation parameters to obtain a position-corrected license plate area image.

[0104] Specifically, the position of the license plate area image is adjusted according to the angle parameters obtained by the transformation parameter acquisition subunit to obtain an adjusted corrected image.

[0105] In the disclosed embodiment, the STNet network model used in the image correction unit does not calculate loss during training, and does not perform explicit supervision. The network parameters are automatically optimized under global loss. Generally, a license plate image corrected by STNet will output a full, positive license plate image. In the disclosed embodiment, the resolution of the image input to the STNet network model and the image output from the STNet network model are both 192×64.

[0106] The license plate recognition unit 340 is used to input the image of the license plate area into the license plate recognition model to obtain the license plate recognition prediction result output by the license plate recognition model; wherein, the license plate recognition prediction result is output in the form of a character combination of a preset number of digits; wherein, the license plate recognition model is obtained based on the second sample image and the character label training of the second sample image.

[0107] In the disclosed embodiment, the license plate recognition unit 340 inputs the position-corrected license plate area image into the license plate recognition model to obtain a license plate recognition prediction result output by the license plate recognition model.

[0108] Specifically, the rectified license plate area image is input into the license plate recognition model, and then after multiple layers of convolution, a 7-character license plate recognition prediction result is directly output. Because the length of the license plate is 7 characters regardless of whether it is a single or double row license plate, with the exception of special license plates (such as the armed police license plate WJ), the license plate recognition model outputs a 7-character result in the disclosed embodiment.

[0109] Specifically, in the embodiment of the present disclosure, the architecture of the license plate recognition model mainly includes the following connected in sequence: input, convolution layer, maximum pooling layer, Small basic block layer, maximum pooling layer, two Small basic block layers, maximum pooling layer, Dropout layer, two convolution layers, Dropout layer, convolution layer, and output.

[0110] In this disclosed embodiment, the resolution of the image input to the license plate recognition model is 192×64, and padding is used in each layer of the license plate recognition model to ensure that the input size changes stably before and after each layer. In this disclosed embodiment, through the rational design of the backbone network and the connected convolutional layers between the input and output, the output maintains a width dimension of 1. The convolutional layer parameters control the characters corresponding to different channels at different positions, thus effectively solving the license plate recognition problem.

[0111] In the disclosed embodiment, the training process of the license plate recognition model specifically includes: randomly masking each character in the second sample image according to a preset probability, and modifying the corresponding character label to strengthen the training of the license plate recognition model.

[0112] Specifically, during the training process of the license plate recognition model, the training sample set includes both license plate images and single-row license plate images, and incorporates noise and perturbation information to ensure that the images are as close to the actual data as possible. During the training process, individual characters in the training samples are masked with a preset probability (for example, 0.2), and their corresponding labels are updated, thereby updating the training model parameters. This enhancement method can improve the accuracy of character position detection.

[0113] Figure 4 An example of a physical structure diagram of an electronic device is shown below. Figure 4As shown, the electronic device may include: a processor (processor) 410, a communication interface (Communications Interface) 420, a memory (memory) 430 and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call the logic instructions in the memory 430 to execute the license plate recognition method, which includes: obtaining a license plate image to be recognized; inputting the license plate image to be recognized into a license plate detection model to obtain the position information of the license plate area in the license plate image to be recognized output by the license plate detection model; wherein the license plate detection model is obtained based on training of a first sample image, a target detection object in the first sample image, and a position information label of the target detection object; obtaining an image of the license plate area in the license plate image to be recognized according to the position information; inputting the image of the license plate area into the license plate recognition model to obtain a license plate recognition prediction result output by the license plate recognition model; wherein the license plate recognition prediction result is output in the form of a character combination of a preset number of digits; wherein the license plate recognition model is obtained based on training of a second sample image and the character label of the second sample image.

[0114] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the embodiment of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0115] On the other hand, the present disclosure also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the license plate recognition method provided by the above-mentioned methods, the method including: obtaining a license plate image to be identified; inputting the license plate image to be identified into a license plate detection model to obtain the position information of the license plate area in the license plate image to be identified output by the license plate detection model; wherein, the license plate detection model is obtained based on training of a first sample image, a target detection object in the first sample image, and a position information label of the target detection object; obtaining an image of the license plate area in the license plate image to be identified according to the position information; inputting the image of the license plate area into the license plate recognition model to obtain a license plate recognition prediction result output by the license plate recognition model; wherein, the license plate recognition prediction result is output in the form of a character combination of a preset number of digits; wherein, the license plate recognition model is obtained based on training of a second sample image and a character label of the second sample image.

[0116] On the other hand, the present disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the above-mentioned license plate recognition methods, the methods comprising: obtaining a license plate image to be recognized; inputting the license plate image to be recognized into a license plate detection model to obtain position information of the license plate area in the license plate image to be recognized output by the license plate detection model; wherein, the license plate detection model is obtained based on training of a first sample image, a target detection object in the first sample image, and a position information label of the target detection object; obtaining an image of the license plate area in the license plate image to be recognized according to the position information; inputting the image of the license plate area into the license plate recognition model to obtain a license plate recognition prediction result output by the license plate recognition model; wherein, the license plate recognition prediction result is output in the form of a character combination of a preset number of digits; wherein, the license plate recognition model is obtained based on training of a second sample image and character labels of the second sample image.

[0117] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0118] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present disclosure.

Claims

1. A license plate recognition method, characterized in that: include: Get the license plate image to be recognized; Inputting the license plate image to be identified into a license plate detection model, and obtaining position information of a license plate region in the license plate image to be identified output by the license plate detection model; wherein the license plate detection model is trained based on a first sample image, a target detection object in the first sample image, and a position information label of the target detection object; Acquire an image of the license plate area in the license plate image to be identified according to the position information; Inputting the image of the license plate area into a license plate recognition model to obtain a license plate recognition prediction result output by the license plate recognition model; wherein the license plate recognition prediction result is output in the form of a character combination of a preset number of digits; wherein the license plate recognition model is obtained based on training of a second sample image and the character labels of the second sample image; The training process of the license plate recognition model specifically includes: randomly covering each character in the second sample image according to a preset probability, and modifying the corresponding character label to strengthen the training of the license plate recognition model.

2. The license plate recognition method according to claim 1, characterized in that: Inputting the license plate image to be identified into a license plate detection model to obtain position information of a license plate area in the license plate image to be identified output by the license plate detection model specifically includes: The license plate image to be identified is input into the license plate detection model to obtain the position information of the rectangular area corresponding to the license plate area in the license plate image to be identified output by the license plate detection model, wherein the position information includes: the coordinates of the center point of the rectangular area and the size data of the rectangular area.

3. The license plate recognition method according to claim 2, characterized in that: The acquiring of the image of the license plate area in the license plate image to be identified according to the position information specifically includes: based on the position information of the rectangular area in the license plate image to be identified, cutting out the image corresponding to the rectangular area in the license plate image to be identified to obtain the image of the license plate area.

4. The license plate recognition method according to claim 3, characterized in that: The method of acquiring an image of the license plate area in the license plate image to be identified according to the position information further includes: performing position transformation on the license plate area image according to the position information to obtain a position-corrected license plate area image.

5. The license plate recognition method according to claim 4, characterized in that: The position transformation of the license plate area image according to its position information to obtain a position-corrected license plate area image specifically includes: Inputting the license plate area image into the spatial transformation network model to obtain the position transformation parameters output by the spatial transformation network model; The license plate area image is position-transformed according to the position transformation parameters to obtain a position-corrected license plate area image.

6. The license plate recognition method according to claim 4, characterized in that: The step of inputting the image of the license plate area into a license plate recognition model to obtain a license plate recognition prediction result output by the license plate recognition model specifically includes: inputting a position-corrected license plate area image into a license plate recognition model to obtain a license plate recognition prediction result output by the license plate recognition model.

7. A license plate recognition device, characterized in that: include: An image acquisition unit, used to acquire an image of a license plate to be identified; a license plate region position acquisition unit, configured to input the license plate image to be identified into a license plate detection model, and obtain position information of the license plate region in the license plate image to be identified output by the license plate detection model; wherein the license plate detection model is trained based on a first sample image, a target detection object in the first sample image, and a position information label of the target detection object; a license plate area image acquisition unit, configured to acquire an image of the license plate area in the license plate image to be identified according to the position information; a license plate recognition unit, configured to input the image of the license plate area into a license plate recognition model, and obtain a license plate recognition prediction result output by the license plate recognition model; wherein the license plate recognition prediction result is output in the form of a character combination of a preset number of digits; wherein the license plate recognition model is obtained based on the second sample image and the character label of the second sample image; The training process of the license plate recognition model specifically includes: randomly covering each character in the second sample image according to a preset probability, and modifying the corresponding character label to strengthen the training of the license plate recognition model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the license plate recognition method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the license plate recognition method according to any one of claims 1 to 6 are implemented.

Images