License Plate Detection and Recognition Methods and Devices

Abstract

This invention relates to the field of urban governance, specifically providing a method and apparatus for license plate detection and recognition, comprising the following steps: S1, license plate target detection based on multi-scale convolutional attention using MSCA_YOLOv12; S2, license plate text recognition based on multi-scale edge enhancement using MSEE_LPRNet; S3, end-to-end automated processing of license plate detection and recognition. Compared with existing technologies, this invention effectively solves the core problems that have long existed in the field of license plate recognition, such as poor multi-scale adaptability and insufficient edge feature extraction, providing efficient and reliable technical support for applications such as intelligent traffic management and security monitoring, and possessing broad commercial application prospects.

Description

Technical Field

[0001] This invention relates to the field of urban governance, specifically providing a method and apparatus for license plate detection and recognition. Background Technology

[0002] With the rapid development of intelligent transportation systems and vehicle management needs, automatic license plate recognition technology has become one of the core technologies in modern traffic management, electronic police systems, and intelligent security. Traditional license plate recognition methods mainly rely on algorithms such as edge detection, color segmentation, and template matching in computer vision technology. These methods can achieve certain results under ideal lighting conditions and with standard license plates. However, in practical applications, license plate recognition faces many challenges: on the one hand, complex and changing natural environments (such as strong light, backlight, and rainy weather) can lead to a decrease in license plate image quality; on the other hand, motion blur caused by vehicle movement, tilted shooting angles, and problems such as dirt or occlusion of the license plate itself can significantly affect the recognition accuracy of traditional algorithms.

[0003] In recent years, deep learning technology has made significant progress in object detection and character recognition, with license plate recognition methods based on convolutional neural networks gradually becoming mainstream. The YOLO series of algorithms, due to their high detection performance, are widely used in license plate detection tasks, while dedicated networks such as LPRNet perform excellently in license plate character recognition. However, existing technologies still have significant shortcomings: in the detection stage, traditional YOLO algorithms are ineffective at detecting small-sized license plates and densely arranged license plates, easily resulting in missed detections and false detections; in the recognition stage, existing methods have poor robustness in recognizing low-resolution, blurred, and deformed characters.

[0004] It is particularly noteworthy that most current systems process detection and recognition as two separate modules, resulting in redundant feature extraction, low computational efficiency, and difficulty in meeting the needs of application scenarios with high real-time requirements. Summary of the Invention

[0005] This invention addresses the shortcomings of the prior art by providing a highly practical method for license plate detection and recognition.

[0006] A further technical objective of this invention is to provide a reasonably designed, safe, and applicable license plate detection and recognition device.

[0007] The technical solution adopted by this invention to solve its technical problem is:

[0008] The license plate detection and recognition method has the following steps:

[0009] S1. License plate target detection based on multi-scale convolutional attention in MSCA_YOLOv12;

[0010] S2. License plate text recognition based on MSEE_LPRNet with multi-scale edge enhancement;

[0011] S3, end-to-end automated processing of license plate detection and recognition.

[0012] Furthermore, in step S1, a multi-scale convolutional attention module is introduced in the feature extraction stage of YOLOv12. The multi-scale convolutional attention module includes multi-branch depthwise separable convolution, 1×1 convolutional layer and feature weighted fusion.

[0013] Furthermore, the multi-branch depthwise separable convolution contains multiple parallel branches, each of which approximates a large kernel convolution through 1×k and k×1 depthwise separable convolutions to capture multi-scale contextual features in a lightweight manner.

[0014] Let the input feature tensor be F, and its shape be (batch, channel, height, width). Then the depthwise separable convolution operation of the i-th branch can be expressed as:

[0015]

[0016] Where, k i represents the convolution kernel of the i-th branch. When i = 0, the original expression is an identity connection, which preserves the original features. DepthwiseConv represents the depthwise separable convolution of the i-th branch, which is used to extract multi-scale features.

[0017] Furthermore, the 1×1 convolutional layer is used to fuse multi-branch features and generate an attention weight map. Channel dependencies are modeled through channel shuffling, and F is set as follows: i Let ' be the feature transformed from the feature of the i-th branch. Then the operation of the 1×1 convolutional layer is represented as:

[0018]

[0019] Among them, Conv 1×1 This represents a 1×1 convolutional layer used to fuse multi-branch features and generate an attention weight map. Scale i This represents the feature transformation of the i-th branch.

[0020] Furthermore, the feature weighted fusion involves multiplying the attention weight map element-wise with the input features to enhance key features and suppress background noise.

[0021] In license plate detection tasks, multi-branch structures can adapt to multi-resolution variations caused by shooting distance and angle, capturing multi-scale features from local edges of small license plates to the global contours of larger license plates. Meanwhile, the attention mechanism can dynamically distinguish license plates from complex backgrounds. Its mathematical formula can be described as follows:

[0022]

[0023] Among them, F out Att represents the enhanced features, and Att represents the attention weights. This represents element-wise multiplication, a method that fuses attention weights with input features to output enhanced features.

[0024] Furthermore, in step S2, a multi-scale edge enhancement module is embedded into the LPRNet basic network model, proposing the MSEE_LPRNet license plate text recognition method based on multi-scale edge enhancement.

[0025] The multi-scale edge enhancement module adopts a parallel multi-branch structure, with each branch configured with dilated convolution kernels and orientation-sensitive filter banks with different dilation rates, achieving multi-granularity edge response through the following formula:

[0026]

[0027] in, This represents the dilated convolution operation in the i-th branch. σ represents the edge enhancement weight matrix in the corresponding direction, and σ represents the ReLU activation function.

[0028] Furthermore, the extracted multi-scale edge feature maps are upsampled and downsampled to a uniform size, then channel-wise concatenated to form a feature pyramid. The bottom layer is directly connected to the feature map output by the backbone network, preserving high-frequency detail information. The middle layer is upsampled and channel-wise concatenated with the bottom layer. Combined with 3×3 dilated convolutions, the resolution and receptive field are balanced. The formula is expressed as:

[0029]

[0030] The higher-level approach introduces a self-attention mechanism to dynamically weight multi-scale edge features:

[0031]

[0032] Where Q, K, and V represent the query, key, and value feature matrices, respectively.

[0033] Furthermore, an edge-aware loss is introduced during model training to assign higher weight to the classification error of character edge regions. The formula is as follows:

[0034]

[0035] Here, Edge Regions represents the edge region mask, generated by the Canny edge detection operator, and y represents the ground truth label. The edge-aware loss represents the model's predicted probability, forcing the model to focus on edge regions and improving its ability to distinguish similar characters on license plates.

[0036] Furthermore, step S3 includes:

[0037] S3-1, Image preprocessing;

[0038] First, input images with a resolution ≥ 640×480 are standardized, including mean-variance normalization with a mean of [0.485, 0.456, 0.406] and a variance of [0.229, 0.224, 0.225] and adaptive histogram equalization to enhance image contrast in low-light scenes. Then, a multi-scale scaling strategy with the short side scaled to 320, 640, and 1280 pixels is used to generate a multi-scale image pyramid to cover the resolution differences between far and near license plates.

[0039] The core grayscale mapping formula for adaptive histogram equalization is as follows:

[0040]

[0041] Among them, C i,j (k) represents sub-block B i,j The cumulative distribution function, L represents the minimum value of the cumulative distribution function within the sub-block, and L represents the total number of gray levels.

[0042] S3-2, MSCA_YOLOv12 License Plate Target Detection;

[0043] Secondly, multi-scale feature extraction is achieved by embedding a multi-scale convolutional attention module to extract features at three scales in parallel within the YOLOv12 Backbone: small scale (52×52), medium scale (26×26), and large scale (13×13), corresponding to the detection requirements of small license plates, standard license plates, and large license plates, respectively. Feature fusion and the detection head achieve cross-layer feature interaction through BiFPN in the Neck layer, while the Head layer adopts a dynamic label allocation strategy to output the license plate bounding box coordinates and confidence scores.

[0044] S3-3, Feature clipping and normalization of regions of interest;

[0045] Subsequently, based on the detection results, the license plate region of interest is cropped from the original image, normalized to a fixed size through affine transformation, matched to the input requirements of MSEE_LPRNet, and a perspective transformation correction module is introduced to perform geometric correction on the tilted license plate to generate a frontal view character image.

[0046] The core formula for affine transformation is as follows:

[0047]

[0048] Where (x,y) and (x',y′) represent the pixel coordinates of the license plate ROI in the original image and the transformed image, respectively, and the target size is 94×24 pixels. 11 a 12 a 21 a 22 All represent linear transformation parameters, used to model rotation, scaling, and shearing. x t y This represents the translation parameter, used to adjust the position of the target area;

[0049] The core formula for perspective transformation correction is as follows:

[0050]

[0051] Among them, h 11 h 12 h 21 h 22 Control rotation, scaling, and shearing, h 13 h 23 Control translation, h 31 h 32 h 33 To control perspective distortion, (u,v,w) represents homogeneous coordinates, which are converted to two-dimensional coordinates (x′,y′) by dividing by the third dimension w.

[0052] S3-4, MSEE_LPRNet license plate text recognition;

[0053] Multi-scale edge enhancement extracts edge features with dilation rates of 1, 3, and 5 by embedding a multi-branch dilated convolution module, and then fuses them to generate an enhanced feature map containing fine strokes and global contours; sequence decoding and post-processing utilize CTC loss to achieve segmentation-free end-to-end training, generate character sequences through beam search, and combine license plate format verification to remove invalid results;

[0054] S3-5, Results Fusion and Output;

[0055] The detection box coordinates, recognized characters, and confidence scores are encapsulated into structured results and output to traffic management platforms or edge computing nodes via TCP or IP protocols for real-time early warning and historical data retrieval.

[0056] A license plate detection and recognition device, comprising: at least one memory and at least one processor;

[0057] The at least one memory is used to store a machine-readable program;

[0058] The at least one processor is used to call the machine-readable program to execute the license plate detection and recognition method.

[0059] Compared with the prior art, the license plate detection and recognition method and apparatus of the present invention have the following outstanding advantages:

[0060] The proposed license plate detection and recognition method based on improved YOLOv12 and LPRNet achieves significant breakthroughs in detection accuracy, recognition robustness, and system efficiency through a multi-scale convolutional attention mechanism (MSCA_YOLOv12) and a multi-scale edge enhancement module (MSEE_LPRNet). Experimental results show that this method has higher adaptability, stability, and real-time performance than traditional schemes.

[0061] It effectively solves the core problems that have long existed in the field of license plate recognition, such as poor multi-scale adaptability and insufficient edge feature extraction, and provides efficient and reliable technical support for applications such as intelligent traffic management and security monitoring, and has broad commercial application prospects. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a diagram of the license plate target detection architecture in a license plate detection and recognition method called MSCA_YOLOv12.

[0064] Figure 2 This is an architecture diagram of a multi-scale convolutional attention module in a license plate detection and recognition method.

[0065] Figure 3 This is a diagram of the MSEE_LPRNet license plate text recognition architecture based on multi-scale edge enhancement in a license plate detection and recognition method.

[0066] Figure 4 This is a schematic diagram of a multi-scale edge enhancement module architecture in a license plate detection and recognition method.

[0067] Figure 5 This is a flowchart of an end-to-end automated license plate detection and recognition system in a license plate detection and recognition method. Detailed Implementation

[0068] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to specific embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] The following is a preferred embodiment:

[0070] The license plate detection and recognition method in this embodiment includes the following steps:

[0071] S1. License plate target detection based on multi-scale convolutional attention in MSCA_YOLOv12;

[0072] As the latest evolution of the YOLO series of object detection algorithms, the YOLOv12 network model is designed around an attention mechanism, breaking through the structural limitations of the traditional YOLO series, which is mainly based on convolutional neural networks. It significantly improves detection accuracy while maintaining real-time performance. YOLOv12 adopts the classic Backbone-Neck-Head three-stage architecture. Specifically, the Backbone introduces a Area Attention module. 2 This system effectively captures both local and global features of license plates. The Neck (feature fusion network) employs a Bidirectional Feature Pyramid Network (BiFPN) structure to enhance the ability to represent license plate features at different scales and angles. The Head (detection head) uses a lightweight, deep separable convolutional design, combined with a dynamic label assignment strategy, to predict the location and category of the license plate.

[0073] The existing YOLOv12 has two limitations: firstly, it lacks multi-scale feature modeling, and secondly, its native A... 2 Although the module improves efficiency through region division, it relies solely on a single-scale attention mechanism, which is insufficiently adaptable to multi-resolution variations in license plates caused by shooting distance and angle (such as small license plates with pixels ≤16×64, and aspect ratio distortion of tilted license plates), making it prone to missed detections. Secondly, its ability to suppress complex backgrounds is limited. Traditional attention mechanisms rely on single-scale features to distinguish between license plates and backgrounds (such as vehicle body patterns and complex environmental textures). In occlusion (such as leaves partially obscuring the license plate) or low-light scenes, it is easy to mistakenly include background areas in the detection box.

[0074] To address the aforementioned issues, this method introduces a multi-scale convolutional attention mechanism into the YOLOv12 base network model, proposing an MSCA_YOLOv12 license plate target detection method based on multi-scale convolutional attention. The architecture of the MSCA_YOLOv12 license plate target detection method based on multi-scale convolutional attention is as follows: Figure 1 As shown.

[0075] Specifically, this method introduces a multi-scale convolutional attention module in the feature extraction stage of YOLOv12. The architecture of the multi-scale convolutional attention module is as follows: Figure 2 As shown. This module mainly consists of three parts:

[0076] (1) Multi-branch depthwise separable convolution;

[0077] This part contains multiple parallel branches (such as strip convolutions with kernel sizes of 7, 11, and 21). Each branch approximates a large kernel convolution through 1×k and k×1 depthwise separable convolutions, capturing multi-scale contextual features in a lightweight manner. Let the input feature tensor be F, with shape (batch, channel, height, width), then the depthwise separable convolution operation of the i-th branch can be expressed as:

[0078]

[0079] Where, k i represents the convolution kernel of the i-th branch. When i = 0, the original expression is an identity connection, preserving the original features. DepthwiseConv represents the depthwise separable convolution of the i-th branch, used to extract multi-scale features.

[0080] (2) 1×1 convolutional layer;

[0081] This section is used to fuse multi-branch features and generate an attention weight map, modeling inter-channel dependencies through channel shuffling. Let F... i Let ' be the feature transformed (e.g., normalized, activated function, etc.) of the i-th branch. Then the operation of a 1×1 convolutional layer can be represented as:

[0082]

[0083] Among them, Conv 1×1 This represents a 1×1 convolutional layer used to fuse multi-branch features and generate an attention weight map. Scale i This represents the feature transformation (such as normalization, activation function, etc.) of the i-th branch.

[0084] (3) Feature-weighted fusion;

[0085] This part multiplies the attention weight map element-wise with the input features to enhance key features and suppress background noise. In license plate detection, the multi-branch structure can adapt to multi-resolution variations caused by shooting distance and angle, capturing multi-scale features from local edges of small license plates to the global contours of larger license plates. The attention mechanism can dynamically distinguish license plates from complex backgrounds, improving robustness to complex scenes such as occlusion and tilt. Its mathematical formula can be described as:

[0086]

[0087] Among them, F out Att represents the enhanced features, and Att represents the attention weights. This represents element-wise multiplication. This method fuses attention weights with input features to output enhanced features.

[0088] In summary, the MSCA_YOLOv12 license plate target detection method based on multi-scale convolutional attention introduces a multi-branch structure into the YOLOv12 framework, utilizing convolutional kernels of different scales to achieve multi-level modeling of license plate features. Small-sized convolutional kernels extract local detail features of the license plate, such as character strokes and spacing; large-sized convolutional kernels acquire overall structural features of the license plate, such as its shape and edges. This method enhances the feature representation capability of the YOLOv12 target detection model for the license plate region through weighted fusion of multi-scale features via an attention mechanism, providing more accurate and robust target region features for subsequent license plate character recognition.

[0089] S2. License plate text recognition based on MSEE_LPRNet with multi-scale edge enhancement;

[0090] The License Plate Recognition Network (LPRNet) is an end-to-end license plate recognition network that does not require pre-segmentation of characters. Its core advantages lie in its lightweight design and real-time performance. The lightweight convolutional backbone network reduces computation through depthwise separable convolutions and a multi-branch structure, while simultaneously extracting multi-scale spatial features. Connectionist Temporal Classification (CTC) loss and sequence decoding decode the feature sequence output by the backbone network into a character probability distribution, generating the final license plate character sequence through greedy search or beam search.

[0091] Despite LPRNet's significant achievements in license plate recognition, its performance remains insufficient when dealing with low-resolution, blurred characters, uneven lighting, and complex backgrounds. This is mainly because the traditional LPRNet feature extraction module relies on a single-scale convolutional kernel, lacking effective utilization of multi-scale edge information. It struggles to capture detailed features and distinguish characters from the background, thus limiting the model's generalization ability across license plate images of different sizes and qualities.

[0092] To address the aforementioned issues, this method embeds a multi-scale edge enhancement module into the LPRNet base network model, proposing the MSEE_LPRNet license plate text recognition method based on multi-scale edge enhancement. The architecture of the MSEE_LPRNet license plate text recognition method based on multi-scale edge enhancement is as follows: Figure 3 As shown.

[0093] Specifically, this method embeds a multi-scale edge enhancement module into the lightweight convolutional backbone of LPRNet. This module extracts edge features at different scales through a multi-branch structure and combines them with a feature pyramid to achieve multi-level modeling. The architecture of the multi-scale edge enhancement module is as follows: Figure 4 As shown, this module employs a parallel multi-branch structure, with each branch configured with dilated convolutional kernels of different dilation rates (e.g., 3×3, 5×5, 7×7) and a direction-sensitive filter bank. Multi-granularity edge responses are achieved through the following formula:

[0094]

[0095] in, This represents the dilated convolution operation in the i-th branch. The weight matrix represents the edge enhancement weights in the corresponding direction, and σ represents the ReLU activation function. This design simulates the multi-scale edge perception mechanism of human vision through sparse convolutional kernel combinations, effectively capturing different morphological features of character strokes (such as straight edges, curves, corners, etc.).

[0096] Subsequently, the extracted multi-scale edge feature maps are upsampled and downsampled to a uniform size, then concatenated channel by channel to form a feature pyramid. The bottom layer directly connects to the feature map output from the backbone network (94×24 resolution), preserving high-frequency detail information (such as character stroke edges). The middle layer is upsampled and concatenated with the bottom layer at the channel level, combined with 3×3 dilated convolutions to balance resolution and receptive field. The formula is expressed as:

[0097]

[0098] The higher-level approach introduces a self-attention mechanism to dynamically weight multi-scale edge features:

[0099]

[0100] Here, Q, K, and V represent the query, key, and value feature matrices, respectively. This mechanism enables the model to adaptively focus on key edge regions and suppress background noise.

[0101] Finally, an edge-aware loss is introduced during model training to give higher weight to the classification error of character edge regions. The formula is as follows:

[0102]

[0103] Here, Edge Regions represents the edge region mask, generated by the Canny edge detection operator. y represents the ground truth label. This represents the model's predicted probability. Edge-aware loss forces the model to focus on edge regions, improving its ability to distinguish similar characters on license plates (such as "O / 0" and "B / 8").

[0104] In summary, the MSEE_LPRNet license plate character recognition method based on multi-scale edge enhancement achieves multi-level modeling and enhancement of license plate character edge features by embedding a multi-branch, multi-scale edge enhancement module into the LPRNet backbone network and combining feature pyramids and self-attention mechanisms. This module extracts multi-scale edge information using dilated convolutional kernels with different dilation rates, fuses cross-layer details and semantics using feature pyramids, dynamically suppresses background noise through self-attention, and introduces edge-aware loss to force the model to focus on character edge regions. This method effectively solves the core problems of insufficient edge feature extraction and poor multi-scale adaptability in traditional methods, providing a reliable technical solution for high-precision applications of end-to-end license plate recognition systems in intelligent transportation, security monitoring, and other fields.

[0105] S3, end-to-end automated processing of license plate detection and recognition;

[0106] This invention constructs an end-to-end license plate detection and recognition system by deeply integrating the MSCA_YOLOv12 license plate target detection module and the MSEE_LPRNet license plate text recognition module, achieving automated processing from image input to character sequence output. This solution significantly improves overall efficiency and accuracy by optimizing the feature transfer path and reducing redundant computation. The end-to-end automated license plate detection and recognition system flow is as follows: Figure 5 As shown.

[0107] include:

[0108] S3-1, Image preprocessing;

[0109] First, input images with a resolution ≥ 640×480 are standardized, including mean-variance normalization with a mean of [0.485, 0.456, 0.406] and a variance of [0.229, 0.224, 0.225], and adaptive histogram equalization (AHE) to enhance image contrast in low-light scenes. A multi-scale scaling strategy, scaling the shorter side to 320, 640, and 1280 pixels, is then used to generate a multi-scale image pyramid, covering the resolution differences between near and far license plates. The core grayscale mapping formula for adaptive histogram equalization is as follows:

[0110]

[0111] Among them, C i,j (k) represents sub-block B i,j The cumulative distribution function, This represents the minimum cumulative distribution function within the sub-block, and L represents the total number of gray levels (usually 256). This formula improves the detail discernibility of low-light license plate images through local contrast enhancement and noise suppression.

[0112] S3-2, MSCA_YOLOv12 License Plate Target Detection;

[0113] Secondly, multi-scale feature extraction is achieved by embedding multi-scale convolutional attention modules (such as...). Figure 2 As shown, features at three scales—small (52×52), medium (26×26), and large (13×13)—are extracted in parallel within the YOLOv12 Backbone, corresponding to the detection requirements of small license plates (≤16×64 pixels), standard license plates (≈32×128 pixels), and large license plates (≥64×256 pixels), respectively. Feature fusion and the detection head achieve cross-layer feature interaction through BiFPN in the Neck layer, while the Head layer adopts a dynamic label allocation strategy (such as SimOTA) to output the license plate bounding box coordinates and confidence scores.

[0114] S3-3, Feature clipping and normalization of regions of interest;

[0115] Subsequently, based on the detection results, the Region of Interest (ROI) of the license plate is cropped from the original image, normalized to a fixed size (e.g., 94×24 pixels) through affine transformation, matched to the input requirements of MSEE_LPRNet, and a perspective transformation correction module is introduced to perform geometric correction on the tilted license plate (within ±30° of rotation angle) to generate a frontal view character image.

[0116] The core formula for affine transformation is as follows:

[0117]

[0118] Among them, (x, y) and (x′, y′) respectively represent the pixel coordinates of the license plate ROI in the original image and the transformed image, and the target size is usually 94×24 pixels. a 11 、a 12 、a 21 、a 22 all represent linear transformation parameters, which are used to model geometric operations such as rotation, scaling, and shearing. t x 、t y represent translation parameters, which are used to adjust the position of the target area.

[0119] The core formula for perspective transformation correction is as follows:

[0120]

[0121] Among them, h 11 、h 12 、h 21 、h 22 control rotation, scaling, and shearing, h 13 、h 23 control translation, h 31 、h 32 、h 33 control perspective distortion (such as skew correction). (u, v, w) represents homogeneous coordinates, which are converted to two-dimensional coordinates (x′, y′) by dividing by the third dimension w.

[0122] S3-4, License plate text recognition of MSEE_LPRNet;

[0123] Multi-scale edge enhancement extracts edge features with dilation rates of 1, 3, and 5 by embedding a multi-branch dilated convolution module (as shown in Figure 4 ), and generates an enhanced feature map that contains fine strokes (such as the slash of "7") and global contours (such as the rectangular border of the license plate) after fusion; sequence decoding and post-processing use the CTC loss to achieve end-to-end training without segmentation, generate a character sequence through beam search (BeamSearch, BS), and eliminate invalid results by combining license plate format verification (such as the provincial abbreviation + letter + number structure of "Zhe A·D8K23").

[0124] S3-5, Result fusion and output;

[0125] Package the detection box coordinates, recognized characters, and confidence level (≥0.9) into a structured result, and output it to the traffic management platform or edge computing node through the TCP / IP protocol, supporting real-time warning (such as recognition of cloned license plates) and historical data retrieval.

[0126] The proposed license plate detection and recognition method based on improved YOLOv12 and LPRNet was systematically validated on the China City Parking Dataset (CCPD). The experiment used 78,598 images (including complex scenes with different lighting, occlusion, and tilt), divided into a training set (62,878 images), a validation set (7,860 images), and a test set (7,860 images) in an 8:1:1 ratio. Data augmentation (horizontal flip, Gaussian blur, and brightness adjustment) was applied to improve the model's generalization ability. Experimental results show that this method significantly outperforms traditional methods in terms of detection accuracy, recognition robustness, and real-time performance. Specific benefits are as follows:

[0127] ① Improved license plate detection performance

[0128] The results of the comparative experiment on license plate target detection are shown in Table 1. In terms of detection accuracy, the MSCA_YOLOv12 model achieved an average accuracy (mAP@0.5) of 98.7% on the test set, a 2.5 percentage point improvement over the native YOLOv12 (96.2%), significantly reducing false negatives (from 3.1% to 0.8%) and false positives (from 2.5% to 0.6%). Regarding multi-scale adaptability, for license plates of different sizes (small license plates ≤ 16×64 pixels, medium license plates ≈ 32×128 pixels, and large license plates ≥ 64×256 pixels), the detection accuracy of MSCA_YOLOv12 reached 97.3%, 99.1%, and 98.5%, respectively, a significant improvement over the traditional YOLOv12 (92.4% for small license plates), validating the effectiveness of the multi-scale convolutional attention mechanism.

[0129] Table 1 Comparative Experiment Results of License Plate Target Detection

[0130]

[0131] ② Improved license plate character recognition performance

[0132] The results of the comparative experiments on license plate character recognition are shown in Table 2. In terms of recognition accuracy, MSEE_LPRNet achieved a character-level accuracy of 99.2% and a license plate-level accuracy of 98.6% on the test set, a significant improvement over the native LPRNet (96.8% for character-level and 95.1% for license plate-level). Regarding adaptability to low-quality images, this method achieved recognition accuracies of 97.8% and 96.5% for blurred characters (Gaussian blur σ = 2.0) and low-light conditions (brightness reduced by 50%), respectively, demonstrating greater robustness compared to traditional methods (an average decrease of 12.3%). In terms of similar character discrimination, this method, through edge-aware loss optimization, reduced the misrecognition rate of easily confused character pairs (such as "0 / O" and "8 / B") from 4.7% to 1.2%.

[0133] Table 2. Comparison Experiment Results of License Plate Character Recognition

[0134]

[0135] ③ End-to-end system efficiency optimization

[0136] The results of the end-to-end system efficiency comparison experiment are shown in Table 3. In terms of real-time performance, this method only requires 28ms (35.7FPS) to process a single frame image (640×480 resolution) end-to-end on an NVIDIA Jetson AGX Xavier edge device, meeting the real-time requirements. This represents a 46.2% improvement in efficiency compared to the traditional two-stage scheme (independent processing of detection and recognition, with an average processing time of 52ms). Regarding computational resource consumption, this method has 12.5M model parameters and 8.7G of floating-point operations (FLOPs), which is 31.7% less than that of schemes with equivalent precision (such as Faster R-CNN+LPRNet, with 18.3M parameters).

[0137] Table 3. Experimental Results of End-to-End System Efficiency Comparison

[0138]

[0139] In summary, the license plate detection and recognition method proposed in this invention, based on improved YOLOv12 and LPRNet, achieves significant breakthroughs in detection accuracy, recognition robustness, and system efficiency through a multi-scale convolutional attention mechanism (MSCA_YOLOv12) and a multi-scale edge enhancement module (MSEE_LPRNet). Experimental results show that this method has higher adaptability, stability, and real-time performance than traditional schemes. Its innovative design effectively solves the core problems that have long existed in the field of license plate recognition, such as poor multi-scale adaptability and insufficient edge feature extraction, providing efficient and reliable technical support for applications such as intelligent traffic management and security monitoring, and possessing broad commercial application prospects.

[0140] Based on the above method, the license plate detection and recognition device in this embodiment includes: at least one memory and at least one processor;

[0141] The at least one memory is used to store a machine-readable program;

[0142] The at least one processor is used to call the machine-readable program to execute the license plate detection and recognition method.

[0143] The above-described specific embodiments are merely specific examples of the present invention. The patent protection scope of the present invention includes, but is not limited to, the above-described specific embodiments. Any technical solution that conforms to the above-described specific embodiments of the present invention and any appropriate changes or substitutions made by those skilled in the art should fall within the patent protection scope of the present invention.

[0144] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for detecting and recognizing license plates, characterized in that, The specific steps include: S1. Image preprocessing specifically includes; First, input images with a resolution ≥ 640×480 are standardized, including mean-variance normalization with a mean of [0.485, 0.456, 0.406] and a variance of [0.229, 0.224, 0.225] and adaptive histogram equalization to enhance image contrast in low-light scenes. Then, a multi-scale scaling strategy with the short side scaled to 320, 640, and 1280 pixels is used to generate a multi-scale image pyramid to cover the resolution differences between far and near license plates. The formula for adaptive histogram equalization grayscale mapping is as follows: ； in, Representative sub-block The cumulative distribution function, This represents the minimum value of the cumulative distribution function within the sub-block. Represents the total number of gray levels; S2 and MSCA_YOLOv12 license plate target detection specifically include: Secondly, multi-scale feature extraction is achieved by embedding a multi-scale convolutional attention module to extract small-scale, medium-scale, and large-scale features in parallel in the YOLOv12 backbone, corresponding to the detection requirements of small license plates, standard license plates, and large license plates, respectively. Feature fusion and detection head are achieved through BiFPN in the Neck layer to realize cross-layer feature interaction, and the Head layer adopts a dynamic label allocation strategy to output the license plate bounding box coordinates and confidence scores. S3. Interest region feature clipping and normalization specifically includes: Subsequently, based on the detection results, the license plate region of interest is cropped from the original image, normalized to a fixed size through affine transformation, matched to the input requirements of MSEE_LPRNet, and a perspective transformation correction module is introduced to perform geometric correction on the tilted license plate to generate a frontal view character image. The formula for affine transformation is as follows: ； in, , These represent the pixel coordinates of the license plate ROI in the original image and the transformed image, respectively. , , , These all represent linear transformation parameters, used to model rotation, scaling, and shearing. , This represents the translation parameter, used to adjust the position of the target area; The formula for perspective transformation correction is as follows: ； in, , , , Control rotation, scaling, and shearing. , Controlling translation, , , Controlling perspective distortion, Representing homogeneous coordinates, obtained by dividing by the third dimension. Convert to two-dimensional coordinates ; S4 and MSEE_LPRNet license plate text recognition specifically include: Multi-scale edge enhancement extracts edge features with dilation rates of 1, 3, and 5 by embedding a multi-branch dilated convolution module. A multi-scale edge enhancement module is embedded into the LPRNet basic network model, and a license plate text recognition method based on multi-scale edge enhancement, MSEE_LPRNet, is proposed. The multi-scale edge enhancement module adopts a parallel multi-branch structure, with each branch configured with dilated convolution kernels and orientation-sensitive filter banks with different dilation rates, achieving multi-granularity edge response through the following formula: ； in, Representing the i Dilated convolution operations in each branch σ represents the edge enhancement weight matrix in the corresponding direction, and σ represents the ReLU activation function. After fusion, an enhanced feature map containing fine strokes and global contours is generated; sequence decoding and post-processing utilize CTC loss to achieve segmentation-free end-to-end training, generate character sequences through beam search, and combine license plate format verification to remove invalid results; An edge-aware loss is introduced during model training to assign higher weight to the classification error of character edge regions. The formula is as follows: ； in, This represents the edge region mask, generated by the Canny edge detection operator. y Represents the true label, ŷ The edge-aware loss represents the model's predicted probability, which forces the model to focus on edge regions and improves its ability to distinguish similar characters on license plates. S5. Result fusion and output specifically include: The detection box coordinates, recognized characters, and confidence scores are encapsulated into structured results and output to traffic management platforms or edge computing nodes via TCP or IP protocols for real-time early warning and historical data retrieval.

2. The license plate detection and recognition method according to claim 1, characterized in that, In step S1, a multi-scale convolutional attention module is introduced in the feature extraction stage of YOLOv12. The multi-scale convolutional attention module includes multi-branch depthwise separable convolution, 1×1 convolutional layer and feature weighted fusion.

3. The license plate detection and recognition method according to claim 2, characterized in that, The multi-branch depthwise separable convolution contains multiple parallel branches, each of which approximates a large kernel convolution through 1×k and k×1 depthwise separable convolutions to capture multi-scale contextual features in a lightweight manner. set up The input feature tensor is F Then the depthwise separable convolution operation of the i-th branch is expressed as: ； in, Represents the first i The convolution kernel of each branch, when i When = 0, the original expression is an identity connection, preserving the original characteristics. Represents the first i Depth-separable convolutions with multiple branches are used to extract multi-scale features.

4. The license plate detection and recognition method according to claim 3, characterized in that, The 1×1 convolutional layer is used to fuse multi-branch features and generate an attention weight map. For the first i The features transformed from each branch are then represented as follows: ； in, Represents attention weight, This represents a 1×1 convolutional layer used to fuse multi-branch features and generate an attention weight map. Represents the first i Feature transformation of each branch.

5. The license plate detection and recognition method according to claim 4, characterized in that, The feature weighted fusion involves multiplying the attention weight map element-wise with the input features to enhance key features and suppress background noise. In license plate detection tasks, multi-branch structures can adapt to multi-resolution variations caused by shooting distance and angle, capturing multi-scale features from local edges of small license plates to global contours of large license plates. Meanwhile, the attention mechanism can dynamically distinguish license plates from complex backgrounds. Its mathematical formula can be described as follows: ； in, Represents the enhanced features, Represents attention weight, This represents element-wise multiplication, a method that fuses attention weights with input features to output enhanced features.

6. A license plate detection and recognition device, characterized in that, include: At least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to invoke the machine-readable program to perform the method according to any one of claims 1 to 5.

Images