A license plate recognition method
By employing reparameterized edge enhancement processing and multi-angle edge information extraction, combined with the Lprnet model, the problems of blurry and noisy license plate images captured by drone cameras were solved, thus improving the accuracy of license plate recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGZHOU CHENGZHI INTELLIGENT MACHINE TECH CO LTD
- Filing Date
- 2026-03-12
- Publication Date
- 2026-06-09
AI Technical Summary
License plate images captured by drone cameras are often blurry and noisy due to their high field of view and frequent changes in lighting angle, resulting in low accuracy of existing recognition algorithms.
We employ reparameterized edge enhancement to extract various edge information at 0°, 30°, 45°, 90°, and 135°. We then combine this with the Lprnet model for license plate recognition. Through Conv3×3 convolution and residual connections, we use CLC loss for training to extract shallow, mid, deep, and global features.
It effectively overcomes the problems of blurriness, noise, and complex lighting in drone-captured images, thus improving the accuracy of license plate recognition.
Smart Images
Figure CN122176684A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a license plate recognition method. Background Technology
[0002] With rapid urbanization and an increasing number of vehicles, violations such as illegal parking are also on the rise. Detecting and recognizing license plate images to gather evidence of these violations has become a crucial aspect of traffic management. Currently, license plate images are primarily captured and detected from fixed angles using ground-based surveillance cameras. However, these cameras are relatively fixed in location and expensive to install. Therefore, with the development of drone technology, drone cameras are now being used to capture, detect, and recognize license plate images.
[0003] However, due to the high field of view of drones, zoom shooting is required, resulting in blurred license plates and a lot of noise. In addition, the angle of view is not fixed as the drone moves, and the lighting angle often changes, which leads to the low accuracy of existing recognition algorithms when recognizing license plate images taken by drones. Summary of the Invention
[0004] Based on this, the purpose of the present invention is to improve the accuracy of license plate recognition and to provide a license plate recognition method.
[0005] A license plate recognition method includes S0: capturing a license plate image;
[0006] S1: The license plate image is preprocessed sequentially to obtain a corrected image; the preprocessing includes license plate region detection and correction. S2: Perform edge enhancement processing on the corrected image to obtain an edge-enhanced image; S3: Extract the edge information from the edge enhancement image to obtain an edge feature image; the edge information includes edge information at 0°, 30°, 45°, 90° and 135°; S4: Extract the shallow features, middle features, deep features and global features of the edge feature image and perform recognition to obtain the recognition result.
[0007] Compared with existing technologies, this invention effectively overcomes the problems of blurriness, high noise, and complex lighting in drone-captured images by performing reparameterized edge enhancement and extracting multiple edge information at 0°, 30°, 45°, 90°, and 135°, thereby improving recognition accuracy.
[0008] Furthermore, the edge enhancement process performs n consecutive Conv3×3 convolutions on the corrected image, and then performs residual concatenation between the convolution results and the corrected image to obtain the edge enhancement image, where n is greater than 1.
[0009] Further, in step S3, when extracting the edge information, the edge information and second-order spatial derivatives of 0°, 30°, 45°, 90° and 135° are extracted respectively, and the edge information and second-order spatial derivatives are concatenated.
[0010] Further, in step S3, the edge enhancement image is subjected to Conv3×3 convolution with a kernel of... The convolution extracts the edge information at the 45° angle; By performing Conv3×3 convolution and kernel on the edge enhancement image... The convolution extracts the edge information at 135°.
[0011] Furthermore, during edge enhancement processing in step S2, a combination of... Losses The loss is expressed as: Represented as: , in, , , in, This represents the number of pixels in the feature map. For the first feature map before processing The pixel value of each pixel. The first feature map in the processed feature map corresponds to the first feature map in the unprocessed feature map. The pixel value of each pixel; This represents the number of pixels in the feature map. C is the number of channels, W is the image width, and H is the image height; To process the feature map before training the model, the first... Each feature activation value, To process the first feature map after training the model Each feature activation value.
[0012] Furthermore, in step S4, when extracting the shallow features, mid-level features, deep features, and global features of the edge feature image and performing recognition, CLC loss is used for training.
[0013] Based on the same inventive concept, the present invention also provides a license plate recognition device, including a camera, which performs the following steps: S0: capturing a license plate image; a preprocessing unit, which performs the following steps: S1: sequentially preprocessing the license plate image to obtain a corrected image; the preprocessing includes license plate region detection and correction; an edge feature extraction unit, which performs the following steps: S3: extracting edge information from the edge enhancement image to obtain an edge feature image; the edge information includes edge information at 0°, 30°, 45°, 90° and 135°; and a recognition unit, which performs the following steps: S4: extracting shallow features, mid-level features, deep features and global features from the edge feature image and performing recognition to obtain a recognition result.
[0014] Compared with existing technologies, this invention effectively overcomes the problems of blurriness, high noise, and complex lighting in drone-captured images by performing reparameterized edge enhancement and extracting multiple edge information at 0°, 30°, 45°, 90°, and 135°, thereby improving recognition accuracy.
[0015] Furthermore, the edge feature extraction unit includes an information extraction module and a stitching module. The information extraction module comprises seven branches: the first branch performs Conv3×3 convolution; the second branch performs Conv1×1 and Conv3×3 convolutions sequentially; the third branch performs Conv1×1 convolution and Sobel processing along the 0° x-direction sequentially; the fourth branch performs Conv1×1 convolution and Sobel processing along the 90° y-direction sequentially; the fifth branch performs Conv1×1 convolution and Laplace processing to obtain the second-order spatial derivative; and the sixth branch has a first convolutional layer performing Conv3×3 convolution and a second Sobel layer with an edge information extraction kernel of [missing information]. The edge information of the edge enhancement image at 45° is extracted; the first layer of the seventh branch is a convolutional layer, performing Conv3×3 convolution; the second layer is a Sobel layer, whose edge information extraction kernel is... The edge information of the edge enhancement image at 135° is extracted; the stitching module stitches the results of the seven branches in the information extraction module to obtain the edge feature image.
[0016] Furthermore, the structure of the Lprnet model consists of a shallow feature module, a mid-level feature extraction module, a deep feature extraction module, a context-aware module, and a concatenation module, in that order. The shallow feature extraction module takes the edge feature image as input to obtain shallow features, including convolutional layers and activation layers.
[0017] The mid-layer feature extraction module processes the edge feature image as input to obtain mid-layer features, which are, in sequence, a convolutional layer, an activation layer, a max pooling layer, an SBB convolutional block, and another activation layer. The deep feature extraction module processes the mid-layer features as input to obtain deep features, which are in the following order: a max pooling layer, an SBB convolutional block, an activation layer, another SBB convolutional block, and another activation layer.
[0018] The context-aware module processes the deep features as input to obtain global features, which are in the following order: max pooling layer, convolutional layer, activation layer, second convolutional layer, and activation layer.
[0019] The stitching module includes an average pooling layer, which performs average pooling on the shallow features, mid-level features, and deep features respectively; a stitching layer, which stitches the average pooled shallow features, mid-level features, and deep features with the context features; and a convolutional layer, which convolves the stitching result of the stitching layer to obtain the recognition result.
[0020] The SBB convolutional block performs three consecutive convolutions and ReLU activations on its input image, and then performs a convolution on the result of the last ReLU activation. Each activation layer performs Batch Normalization (BN) and ReLU activation.
[0021] Based on the same inventive concept, the present invention also provides an electronic device, comprising: a processor; a memory for storing a computer program executed by the processor; wherein, when the processor executes the computer program, it implements the license plate recognition method described above.
[0022] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description
[0023] Figure 1 This is a structural diagram of the identification device of the present invention; Figure 2 Here is a flowchart of the identification method; Figure 3 This is a structural diagram of the identification unit; Figure 4 This is a structural diagram of the sbb convolutional block. Detailed Implementation
[0024] To address the limitations of current license plate recognition algorithms in meeting the requirements for license plate recognition from the perspective of drones, the inventors designed a new license plate recognition algorithm that detects and corrects license plate images. This algorithm enhances the edges of the license plate image, extracts edge features, and then performs recognition. By combining edge information from 0°, 30°, 45°, 90°, and 135°, the algorithm accurately identifies the license plate.
[0025] Specifically, please refer to Figure 1 , Figure 1This is a structural diagram of the recognition device of the present invention. The license plate recognition device of the present invention includes a camera M0, a preprocessing unit M1, an edge enhancement preprocessing unit M2, an edge feature extraction unit M3, and a recognition unit M4. It executes the recognition method of the present invention to photograph and recognize vehicles, specifically including: Please see Figure 2 , Figure 2 This is a flowchart of the recognition method. The camera M0 performs step S0: capturing a license plate image.
[0026] The preprocessing unit M1 performs step S1: sequentially preprocessing the license plate image to obtain a corrected image. The preprocessing includes license plate region detection and correction; the detection involves detecting, cropping, and enlarging the license plate region in the license plate image; the correction involves rotating the license plate region to a uniform angle, obtaining a corrected image with consistent size and angle.
[0027] The edge enhancement preprocessing unit M2 performs step S2: edge enhancement processing on the corrected image to obtain an edge-enhanced image.
[0028] The edge enhancement processing unit (EEPM) comprises n consecutive convolutional layers and a residual connection layer. Each convolutional layer performs a Conv3×3 convolution on its input image. The residual connection layer performs a residual concatenation between the convolution result of the last convolutional layer and the input image of the EEPM. Therefore, when performing edge enhancement processing, the edge enhancement processing module performs n consecutive Conv3×3 convolutions on the corrected image and performs a residual concatenation between the convolution result and the corrected image to obtain the edge-enhanced image, where n is greater than 1.
[0029] The edge feature extraction unit M3 performs step S3: extracting the edge information of the edge enhancement image to obtain an edge feature image. The edge feature extraction unit M3 is an EECB unit, which is designed based on the reparameterized ECB unit. When extracting the edge information of the edge enhancement image, the edge feature extraction unit M3 simultaneously extracts the edge information at 0°, 30°, 45°, 90°, and 135°. To address the impact of angular light changes on the edge information in the license plate image obtained from the UAV's perspective, the extraction of 45° and 135° edge information is further enhanced. The edge feature extraction unit M3 includes an information extraction module and a splicing module. The information extraction module comprises seven branches, with the first to fifth branches identical to the existing ECB unit. The first branch performs Conv3×3 convolution; the second branch performs Conv1×1 and Conv3×3 convolutions sequentially; the third branch performs Conv1×1 convolution and Sobel processing along the 0° x-direction sequentially; the fourth branch performs Conv1×1 convolution and Sobel processing along the 90° y-direction sequentially; the fifth branch performs Conv1×1 convolution and Laplace processing to obtain the second spatial derivative; the sixth branch has a first convolutional layer performing Conv3×3 convolution and a second Sobel layer with an edge information extraction kernel of [missing information]. The edge information of the edge enhancement image at 45° is extracted; the first layer of the seventh branch is a convolutional layer, performing Conv3×3 convolution; the second layer is a Sobel layer, whose edge information extraction kernel is... The edge information at 135° is extracted from the edge enhancement image. The stitching module stitches the results of the seven branches in the information extraction module to obtain the edge feature image, the expression of which is: , in, This is the result of the Conv3x3 convolution in the first branch. This is the convolution result of the second branch. This is the result of processing the third branch. This is the result of processing the fourth branch. This is the result of processing the fifth branch. The result of processing the sixth branch. This is the processing result of the seventh branch. The EECB unit is reparameterized into a conv-3x3 convolution to ensure that it can be deployed on edge devices later, while extracting edge information from all directions.
[0030] Please see Figure 3 , Figure 3This is a structural diagram of the recognition unit. The recognition unit M4 performs step S4: extracting shallow features, mid-level features, deep features, and global features from the edge feature image and performing recognition to obtain a recognition result. In this embodiment, the Lprnet model is used to classify and recognize the feature information of the edge feature image. The structure of the Lprnet model consists of a shallow feature module, a mid-level feature extraction module, a deep feature extraction module, a context-aware module, and a stitching module.
[0031] The shallow feature extraction module takes the edge feature image as input to obtain shallow features, including convolutional layers and activation layers.
[0032] The mid-layer feature extraction module processes the edge feature image as input to obtain mid-layer features, which are, in sequence, a convolutional layer, an activation layer, a max pooling layer, an SBB convolutional block, and another activation layer. The deep feature extraction module processes the mid-layer features as input to obtain deep features, which are in the following order: a max pooling layer, an SBB convolutional block, an activation layer, another SBB convolutional block, and another activation layer.
[0033] The context-aware module processes the deep features as input to obtain global features, which are in the following order: max pooling layer, convolutional layer, activation layer, second convolutional layer, and activation layer.
[0034] The stitching module includes an average pooling layer, which performs average pooling on the shallow features, mid-level features, and deep features respectively; a stitching layer, which stitches the average pooled shallow features, mid-level features, and deep features with the context features; and a convolutional layer, which convolves the stitching result of the stitching layer to obtain the recognition result.
[0035] Please see Figure 4 , Figure 4 This is a structural diagram of the sbb convolutional block. The sbb convolutional block performs three consecutive convolutions and ReLU activations on its input image, and then performs a convolution on the result of the last ReLU activation. Each activation layer undergoes Batch Normalization (BN) and ReLU activation processing.
[0036] During the training process, the edge enhancement processing unit M2, i.e., the EEPM unit, is first trained individually, using a combination of... Losses Training is performed using the loss function, through the aforementioned The loss ensures consistency between input and output, and the Lperceptual loss unifies the output distribution. During training, the image processed by the EEPM unit is input into the training model to obtain feature maps, and the loss of the feature maps before and after processing by the training model is compared. The loss function of the EEPM is expressed as: , in, , , Where n is the number of pixels in the feature map. For the first feature map before processing The pixel value of each pixel. The first feature map in the processed feature map corresponds to the first feature map in the unprocessed feature map. The pixel value of each pixel; This represents the number of pixels in the feature map. C is the number of channels, W is the image width, and H is the image height; To process the feature map before training the model, the first... Each feature activation value, To process the first feature map after training the model Each feature activation value.
[0037] After the EEPM model is trained, it is frozen and then connected to the Lprnet model for training. The Lprnet model uses CLC loss.
[0038] In this embodiment, the Lperceptual is trained using a pre-trained VGG16 model as the training model.
[0039] Compared with existing technologies, this invention effectively overcomes the problems of blurriness, high noise, and complex lighting in drone-captured images by performing reparameterized edge enhancement and extracting multiple edge information at 0°, 30°, 45°, 90°, and 135°, thereby improving recognition accuracy.
[0040] This application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.
[0041] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. The singular forms “a,” “the,” and “the” used in the embodiments and claims of this application are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that, unless otherwise stated, “a plurality” means two or more; the terms “first,” “second,” “third,” etc., are used only to distinguish and not to describe a particular order or sequence, nor should they be construed as indicating or implying relative importance. The term “and / or” as used herein refers to and includes any or all possible combinations of one or more associated listed items. When the above description relates to drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. In the description of this application, those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.
[0042] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and the present invention also intends to include these modifications and variations.
Claims
1. A license plate recognition method, characterized in that, include: S0: Capture license plate image; S1: The license plate image is preprocessed sequentially to obtain a corrected image; the preprocessing includes license plate region detection and correction. S2: Perform edge enhancement processing on the corrected image to obtain an edge-enhanced image; S3: Extract the edge information from the edge enhancement image to obtain an edge feature image; the edge information includes edge information at 0°, 30°, 45°, 90° and 135°; S4: Extract the shallow features, middle features, deep features and global features of the edge feature image and perform recognition to obtain the recognition result.
2. The license plate recognition method according to claim 1, characterized in that, The edge enhancement process involves performing n consecutive Conv3×3 convolutions on the corrected image, and then performing a residual concatenation between the convolution result and the corrected image to obtain the edge enhancement image, where n is greater than 1.
3. The license plate recognition method according to claim 1, characterized in that, In step S3, when extracting the edge information, the edge information and second-order spatial derivatives of 0°, 30°, 45°, 90° and 135° are extracted respectively, and the edge information and second-order spatial derivatives are concatenated.
4. The license plate recognition method according to claim 3, characterized in that, In step S3, the edge enhancement image is subjected to Conv3×3 convolution with a kernel of... The convolution process extracts the edge information at the 45° angle; By performing Conv3×3 convolution and kernel on the edge enhancement image... The convolution extracts the edge information at 135°.
5. The license plate recognition method according to claim 1, characterized in that, When performing edge enhancement processing in step S2, a combination of Losses The loss is expressed as: Represented as: ， in, , ， in, This represents the number of pixels in the feature map. For the first feature map before processing The pixel value of each pixel. The first feature map in the processed feature map corresponds to the first feature map in the unprocessed feature map. The pixel value of each pixel; This represents the number of pixels in the feature map. , For the number of channels, Image width, Image height; To process the pre-feature map, the first step in the training model... Each feature activation value, To process the first feature map after training the model Each feature activation value.
6. The license plate recognition method according to claim 2, characterized in that, When extracting shallow features, mid-level features, deep features, and global features from the edge feature image in step S4 and performing recognition, CLC loss is used for training.
7. A license plate recognition device, characterized in that, include: The camera performs step S0: capturing an image of the license plate; The preprocessing unit performs step S1: sequentially preprocessing the license plate image to obtain a corrected image; the preprocessing includes license plate region detection and correction. The edge feature extraction unit performs step S3: extracting the edge information of the edge enhancement image to obtain an edge feature image; the edge information includes edge information of 0°, 30°, 45°, 90° and 135°; The recognition unit performs step S4: extracting shallow features, mid-level features, deep features, and global features of the edge feature image and performing recognition to obtain the recognition result.
8. The license plate recognition device according to claim 7, characterized in that, The edge feature extraction unit includes an information extraction module and a stitching module. The information extraction module comprises seven branches: the first branch performs Conv3×3 convolution; the second branch performs Conv1×1 and Conv3×3 convolutions sequentially; the third branch performs Conv1×1 convolution and Sobel processing along the 0° x-direction sequentially; the fourth branch performs Conv1×1 convolution and Sobel processing along the 90° y-direction sequentially; the fifth branch performs Conv1×1 convolution and Laplace processing to obtain the second spatial derivative; and the sixth branch has a first convolutional layer performing Conv3×3 convolution and a second Sobel layer with an edge information extraction kernel of [missing information]. The edge information of the edge enhancement image at 45° is extracted; the first layer of the seventh branch is a convolutional layer, performing Conv3×3 convolution; the second layer is a Sobel layer, whose edge information extraction kernel is... The edge information of the edge enhancement image at 135° is extracted; the stitching module stitches the results of the seven branches in the information extraction module to obtain the edge feature image.
9. The license plate recognition device according to claim 8, characterized in that, The structure of the Lprnet model consists of a shallow feature module, a mid-level feature extraction module, a deep feature extraction module, a context-aware module, and a splicing module. The shallow feature extraction module takes the edge feature image as input to obtain shallow features, including convolutional layers and activation layers; The mid-layer feature extraction module processes the edge feature image as input to obtain mid-layer features, which are, in sequence, a convolutional layer, an activation layer, a max pooling layer, an SBB convolutional block, and another activation layer. The deep feature extraction module processes the mid-layer features as input to obtain deep features, which are in the following order: a max pooling layer, an SBB convolutional block, an activation layer, another SBB convolutional block, and another activation layer. The context-aware module processes the deep features as input to obtain global features, which are in the following order: max pooling layer, convolutional layer, activation layer, second convolutional layer, and activation layer. The splicing module includes an average pooling layer, which performs average pooling on the shallow features, mid-level features, and deep features respectively; and a splicing layer, which splices the average pooled shallow features, mid-level features, and deep features with the context features. A convolutional layer is used to convolve the stitching result of the stitching layer to obtain the recognition result; The sbb convolutional block performs three consecutive convolutions and ReLU activations on its input image, and performs a convolution on the result of the last ReLU activation; each activation layer performs BN and ReLU activation processing.
10. An electronic device, characterized in that, processor; A memory for storing computer programs executed by the processor; The processor executes the computer program to implement the license plate recognition method according to any one of claims 1-7.