License plate recognition system and method

Abstract

The invention provides a license plate recognition system and method. The system comprises a license plate positioning module, a text reconstruction module and a character recognition module. The license plate positioning module is used for carrying out license plate detection on video image data in a monitoring area and sending a license plate detection result to the text reconstruction module; the text reconstruction module is used for performing deblurring processing on a blurred license plate image in the license plate detection result based on the license plate detection result, ensuring that characters of the license plate image can be accurately recognized after deblurring processing, forming a processed license plate detection result, and rebuilding the license plate detection result. The processed license plate detection result is sent to the character recognition module; and the character recognition module is used for outputting a license plate recognition result based on the processed license plate detection result. And a license plate positioning algorithm based on the yov8 is proposed to be combined with a transform algorithm to realize text reconstruction of the fuzzy license plate, and the accuracy of license plate recognition is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of license plate recognition, and in particular relates to a license plate recognition system. Background Art

[0002] License plate recognition is an important task in intelligent traffic management systems. However, challenges in achieving accurate recognition remain due to motion blur from fast-moving vehicles. This blur makes license plates difficult to identify, which is particularly problematic in high-speed vehicles or low-light conditions. These problems are exacerbated at night or in bad weather, resulting in considerable motion blur in captured images. In real-world scenarios, license plate images affected by motion blur often exhibit complex degradations, including noise, low resolution, and ghosting effects. In the case of poor image quality, the accuracy and robustness of the algorithm are reduced. Therefore, it is of great theoretical significance and application value to study license plate recognition algorithms that can adapt to complex scenarios, are robust, and have high accuracy.

[0003] The recognition effect of blurred license plates in the prior art is not good enough. Summary of the invention

[0004] In order to overcome the problem that the recognition effect of blurred license plates in the above-mentioned prior art is not good enough, the present invention proposes a license plate recognition system, including: a license plate positioning module, a text reconstruction module, and a character recognition module;

[0005] The license plate positioning module is used to detect the license plate of the video image data in the monitoring area and send the result of the license plate detection to the text reconstruction module;

[0006] The text reconstruction module is used to perform a deblurring process on the blurred license plate image in the license plate detection result based on the license plate detection result, ensure that the characters of the license plate image can be accurately recognized after the deblurring process, form a processed license plate detection result, and send the processed license plate detection result to the character recognition module;

[0007] The character recognition module is used to output a license plate recognition result based on the processed license plate detection result.

[0008] Preferably, the license plate positioning module is also used to: realize the positioning detection of the license plate based on the deep learning target detection algorithm yolov8, first obtain the image sequence of the license plate, input the image sequence into the target detection algorithm yolov8, and obtain the category and position information of the detected license plate.

[0009] Preferably, the types of license plates include green plates, yellow plates, blue plates, white plates, and black plates.

[0010] Preferably, the text reconstruction module is specifically used to restore the blurred license plate image to a clear license plate image.

[0011] Preferably, the text reconstruction module comprises: a feature extraction module, a SWT-C module and an output module;

[0012] The feature extraction module is used to extract shallow feature information of the blurred license plate image; the shallow feature information includes: the original residual block and the dense residual block of the license plate image, and the original residual block and the dense residual block are combined to form the features of the license plate image; the SWT-C module and the output module are used to combine the convolutional neural network (CNN) with the attention conversion network (Transformer) SWT-C layer to assist the blurred license plate image to restore the edges and details, and then use the pyramid convolution network to output the processed clear image.

[0013] Preferably, the feature extraction module is also used to first design a convolution layer with a convolution kernel size of 1×1 when outputting a processed clear image using a pyramid convolution network, perform channel compression on the obtained license plate image with restored edges and details to form a compressed image, and concentrate the compressed image scattered on the channel on a specific channel to form a compressed image located in the specific channel; then, use the pyramid convolution network to perform convolutions of different scales on the compressed image located in the specific channel to obtain feature information of multiple scales in the blurred license plate image corresponding to the compressed image of the specific channel, thereby obtaining a clear image.

[0014] Preferably, the character recognition module is also used to avoid segmenting the characters in the processed license plate detection result.

[0015] Preferably, the character recognition module is also used to send the license plate recognition result to the OCR text detection and recognition module;

[0016] It also includes an OCR text detection and recognition module, which is used to receive the license plate recognition result, detect the text box position in the license plate image of the license plate recognition result, correct the text box position for the crooked text box, and finally output the license plate characters in the text box.

[0017] The present invention also provides a license plate recognition method, comprising:

[0018] The license plate positioning module in the license plate recognition system is used to detect the license plate of the video image data in the monitoring area, and the result of the license plate detection is sent to the text reconstruction module;

[0019] The text reconstruction module performs deblurring processing on the blurred license plate image in the license plate detection result based on the license plate detection result, ensures that the characters of the license plate image can be accurately recognized after the deblurring processing, forms a processed license plate detection result, and sends the processed license plate detection result to the character recognition module;

[0020] The character recognition module outputs a license plate recognition result based on the processed license plate detection result.

[0021] Compared with the closest prior art, the present invention has the following beneficial effects:

[0022] The system of the present invention comprises a license plate positioning module, a text reconstruction module and a character recognition module; the license plate positioning module is used to perform license plate detection on video image data in a monitoring area and send the result of the license plate detection to the text reconstruction module; the text reconstruction module is used to deblur the blurred license plate image in the result of the license plate detection based on the result of the license plate detection, ensure that the characters of the license plate image can be accurately recognized after the deblurring process, form a processed license plate detection result, and send the processed license plate detection result to the character recognition module; the character recognition module is used to output the license plate recognition result based on the processed license plate detection result, and proposes a license plate positioning algorithm based on yolov8 combined with an attention conversion network (Transformer) algorithm to realize text reconstruction of blurred license plates, thereby improving the accuracy of license plate recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 An architectural diagram of a license plate recognition system provided by the present invention;

[0024] Figure 2 A flow chart of a license plate recognition method provided by the present invention;

[0025] Figure 3 The flowchart in Example 1 provided in the present application;

[0026] Figure 4 The process in Example 1 provided for the present invention Figure 2 . DETAILED DESCRIPTION

[0027] The specific implementation methods of the present invention are further described in detail below with reference to the accompanying drawings.

[0028] Example 1

[0029] Based on the license plate positioning method based on deep learning, the convolutional neural network (CNN) and the attention transformation network (Transformer) algorithm are used to realize license plate character text reconstruction, solve the problem of incorrect character recognition caused by image blur, and further improve the algorithm accuracy.

[0030] The method described in the present invention has the following process: (1) training a vehicle and license plate detection model; (2) detecting a target online on the camera side to obtain a license plate detection result; (3) deblurring the license plate detection result to achieve text reconstruction; (4) performing character recognition on the processed license plate detection result to obtain a final result of license plate recognition.

[0031] (1) License Plate Positioning Module

[0032] The deep learning-based target detection algorithm yolov8 can achieve fast detection while maintaining high-precision detection. Therefore, the algorithm is used to realize license plate positioning detection. First, the image sequence in the application scenario is collected to train the license plate detection model. The video image data in the monitoring scenario is used to perform online multi-category target detection to obtain the category and location information of the detection target. Among them, the detection target categories include green cards, yellow cards, blue cards, white cards, black cards, etc.

[0033] (2) Text reconstruction module

[0034] Due to the rapid movement of the vehicle, the image captured by the camera may be motion blurred, and this blurred license plate will make the characters difficult to recognize. Therefore, this paper proposes an image deblurring method to restore the blurred image to a clear image, so as to facilitate more accurate observation and recognition of the license plate image.

[0035] The present invention combines the advantages of convolutional neural networks and (Transformer) attention conversion networks, making full use of local features while also capturing the ability of long-term dependencies between local features. The model can restore and reconstruct blurred images and has applicability to images of different scales. The dense residual connection structure can continue to obtain richer feature representations of the image, improve the learning ability of the model, and help restore image details. This paper effectively alleviates the problem of feature sparsity by combining dense residuals with the original residual structure.

[0036] The network structure design of the model is mainly divided into four modules, namely the feature extraction module, the residual connection module, the SWT-C module and the output module. The shallow feature information is extracted through the feature extraction network. This paper combines the original residual block with the dense residual block to fuse the features to obtain richer features. The introduction of Swin Transformer as an inference module helps to highlight important integrated features, while suppressing redundant features and making better use of discriminative features. In order to enhance the model's ability to capture the interdependencies between long features, this paper uses the SWT-C layer that combines the convolutional neural network (CNN) with the attention transformation network (Transformer), which can use local features and long-term dependencies at the same time to assist in restoring more edges and details from blurred images. Finally, the pyramid convolutional network is used to output the processed clear image.

[0037] Swin Transformer has shown excellent performance in the field of vision, but the fully connected layer (Multi-Layer Perceptron, MLP) used inside it destroys the local correlation of image features. The SWT-C module replaces the fully connected layer in the original structure with a convolutional layer (Conv). By combining the convolutional neural network (CNN) with the attention conversion network (Transformer), the SWT-C layer draws on the advantages of both and has the ability to fully utilize local features while also capturing the long-term dependencies between local features.

[0038] In the design of the output block, this paper first designs a convolution layer with a convolution kernel size of 1×1, and performs channel compression on the obtained feature array to concentrate the feature information scattered on the channel on a specific channel; secondly, this paper uses a pyramid convolutional neural network to perform convolutions on the image at different scales, so that feature information of multiple scales in the original image can be obtained at the same time, making the model training more efficient and obtaining a clear image.

[0039] (3) Character Recognition Module

[0040] There are two main methods for license plate character recognition based on deep learning: character segmentation-based and character segmentation-free recognition methods. Traditional character segmentation-based methods have problems such as inaccurate segmentation, poor recognition accuracy, and insufficient algorithm robustness in complex license plate recognition environments and fuzzy license plates. The character segmentation-free method does not require pre-segmentation of characters, effectively reducing the time for segmentation processing.

[0041] This paper uses an end-to-end license plate recognition deep learning network. PP-OCR is a lightweight optical character recognition algorithm consisting of a text detector, a detection box corrector, and a text recognizer. First, the text detector of PP-OCRv2 is used to detect the position of the text box in the input image, then the detection box corrector is used to correct the text box, and finally the text recognizer is used to output the license plate characters in the text box.

[0042] The system of the present invention mainly includes three modules, namely, a license plate positioning module, a text reconstruction module, and a character recognition module. Among them, the license plate positioning module mainly performs license plate detection on the video image data under the monitoring area. The text reconstruction module mainly performs further deblurring processing on the result of license plate detection to ensure that the image characters can be accurately recognized and improve the accuracy of character recognition. The character recognition module mainly performs further character recognition based on the processed license plate detection result and outputs the license plate recognition result.

[0043] The main innovations are as follows: (1) The present invention can be applied to multiple scenarios, such as intelligent transportation, smart cities and other scenarios; (2) The present invention uses the yolov8 target detection algorithm to realize license plate detection and improve the detection accuracy of the target; (3) The present invention defuzzifies the license plate detection results and reconstructs the text to facilitate accurate recognition and improve the accuracy of license plate character recognition; (4) The present invention uses a network structure of dense residual blocks to extract features, which is conducive to obtaining a rich representation of features and improving the learning ability of the model; (5) The present invention uses the SWT-C structure that combines a convolutional neural network (CNN) with an attention transformation network (Transformer) to enhance the model's ability to capture the interdependence between long features, so as to facilitate image recovery of more detailed information.

[0044] Those skilled in the art will appreciate that the embodiments of the present invention application can be provided as methods, systems, or computer program products. Therefore, the present invention application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present invention application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0045] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention application rather than to limit its protection scope. Although the present invention application is described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that after reading the present invention application, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the application, but these changes, modifications or equivalent substitutions are all within the protection scope of the claims to be approved.

Claims

1. A license plate recognition system, characterized in that: include: License plate positioning module, text reconstruction module, character recognition module; The license plate positioning module is used to detect the license plate of the video image data in the monitoring area and send the result of the license plate detection to the text reconstruction module; The text reconstruction module is used to perform a deblurring process on the blurred license plate image in the license plate detection result based on the license plate detection result, ensure that the characters of the license plate image can be accurately recognized after the deblurring process, form a processed license plate detection result, and send the processed license plate detection result to the character recognition module; The character recognition module is used to output a license plate recognition result based on the processed license plate detection result.

2. The system according to claim 1, characterized in that The license plate positioning module is also used to: realize the positioning detection of the license plate based on the deep learning target detection algorithm yolov8, first obtain the image sequence of the license plate, input the image sequence into the target detection algorithm yolov8, and obtain the category and position information of the detected license plate.

3. The system according to claim 1, characterized in that The text reconstruction module includes: a feature extraction module, an SWT-C module and an output module; The feature extraction module is used to extract shallow feature information of the blurred license plate image; the shallow feature information includes: the original residual block and the dense residual block of the license plate image, and the original residual block and the dense residual block are combined to form the features of the license plate image; the SWT-C module and the output module are used to combine the convolutional neural network with the attention conversion network. The SWT-C layer assists the blurred license plate image to restore the edges and details, and then uses the pyramid convolution network to output the processed clear image.

4. The system according to claim 1, characterized in that The feature extraction module is also used for, when the processed clear image is outputted by the pyramid convolution network, first designing a convolution layer with a convolution kernel size of 1×1, performing channel compression on the obtained license plate image with restored edges and details to form a compressed image, so that the compressed images scattered on the channels are concentrated on a specific channel to form a compressed image located in the specific channel; Afterwards, the pyramid convolution network is used to perform convolutions of different scales on the compressed image located in the specific channel to obtain feature information of multiple scales in the blurred license plate image corresponding to the compressed image of the specific channel, thereby obtaining a clear image.

5. The system according to claim 1, characterized in that The character recognition module is also used to avoid segmenting the characters in the processed license plate detection result.

6. The system according to claim 1, characterized in that The character recognition module is also used to send the license plate recognition result to the OCR text detection and recognition module; it also includes an OCR text detection and recognition module, which is used to receive the license plate recognition result, detect the text box position in the license plate image of the license plate recognition result, correct the text box position for the skewed text box, and finally, output the license plate characters in the text box.

7. The system according to claim 2, characterized in that The types of license plates include green plates, yellow plates, blue plates, white plates and black plates.

8. The system according to claim 1 or 3, characterized in that: The text reconstruction module is specifically used to restore the blurred license plate image to a clear license plate image.

9. A license plate recognition method, characterized in that: include: The license plate positioning module in the license plate recognition system is used to detect the license plate of the video image data in the monitoring area, and the result of the license plate detection is sent to the text reconstruction module; The text reconstruction module performs deblurring processing on the blurred license plate image in the license plate detection result based on the license plate detection result, ensures that the characters of the license plate image can be accurately recognized after the deblurring processing, forms a processed license plate detection result, and sends the processed license plate detection result to the character recognition module; The character recognition module outputs a license plate recognition result based on the processed license plate detection result.

Images