License plate recognition method and device and electronic equipment

Abstract

The invention discloses a license plate recognition method and device and electronic equipment. The method comprises the steps that a target image is acquired, a license plate area corresponding to a license plate to be recognized is determined from the target image, and the target image comprises at least one license plate to be recognized; a definition index corresponding to the license plate area is determined, and the definition index is used for quantitatively representing the richness degree of license plate details in the license plate area; an area type corresponding to the definition index is determined, and the area type is used for reflecting the complexity at least needing to be met by an identification network for identifying the license plate area; and performing character recognition on the license plate area by adopting a recognition network corresponding to the area type to obtain character information of the license plate to be recognized. The technical problem of low recognition efficiency caused by adopting a unified recognition network to recognize the license plate image in the prior art is solved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a method, apparatus, and electronic device for license plate recognition. Background Technology

[0002] License plate recognition technology is widely used in road traffic management, smart parking lots, security monitoring and other fields. However, the license plate recognition technology uses a unified recognition network to recognize images. In order to adapt to different environmental conditions, the recognition network is often complex in structure, resulting in low license plate recognition efficiency.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a method, apparatus, and electronic device for license plate recognition, which at least solves the technical problem of low recognition efficiency caused by the use of a unified recognition network to recognize license plate images in related technologies.

[0005] According to one aspect of the embodiments of this application, a method for license plate recognition is provided, comprising: acquiring a target image and determining a license plate region corresponding to a license plate to be recognized from the target image, wherein the target image contains at least one license plate to be recognized; determining a sharpness index corresponding to the license plate region, wherein the sharpness index is used to quantify the richness of license plate details in the license plate region; determining a region type corresponding to the sharpness index, wherein the region type is used to reflect the minimum complexity that a recognition network for recognizing the license plate region needs to satisfy; and performing character recognition on the license plate region using a recognition network corresponding to the region type to obtain character information of the license plate to be recognized.

[0006] In some embodiments of this application, a recognition network corresponding to the region type is used to perform character recognition on the license plate region, including: when the region type is a first type, a first neural network is used to perform character recognition on the license plate region; when the region type is a second type, a second neural network is used to perform character recognition on the license plate region, wherein the structural complexity of the second neural network is greater than the result complexity of the first neural network.

[0007] In some embodiments of this application, after using a second neural network to perform character recognition on the license plate area, the method further includes: obtaining an initial recognition result of the second neural network, wherein the initial recognition result includes a character sequence probability distribution of the license plate to be recognized; decoding the character sequence probability distribution to obtain an initial character sequence; and using a language model to correct the initial character sequence to obtain a target character sequence, wherein the language model is trained based on historical license plate character sequences, and the target character sequence is used to determine the character information of the license plate to be recognized.

[0008] In some embodiments of this application, a language model is used to correct an initial character sequence to obtain a target character sequence, including: comparing the initial character sequence with license plate rules using a first language model to obtain a comparison result, and correcting the initial character sequence based on the comparison result to obtain the target character sequence, wherein the first language model is a model based on license plate format rules; or, using a second language model to determine a first probability of each character appearing in the initial character sequence, and fusing the first probability with a second probability corresponding to each character in the character sequence probability distribution to obtain a first fusion result; decoding the first fusion result to obtain the target character sequence, wherein the second language model is a model based on statistical regularities between license plate characters; or, using a third language model to predict a third probability of each character in the initial character sequence, and fusing the third probability with the second probability to obtain a second fusion result; decoding the second fusion result to obtain the target character sequence, wherein the third language model includes a model based on a bidirectional long short-term memory network.

[0009] In some embodiments of this application, determining the sharpness index corresponding to the license plate area includes: performing edge detection on the license plate area to obtain a detection result, wherein the detection result is used to reflect the local change rate of pixels in the license plate area; determining the pixel value variance of pixels in the license plate area corresponding to the detection result, and determining the sharpness index corresponding to the pixel value variance.

[0010] In some embodiments of this application, determining the license plate region corresponding to the license plate to be identified from the target image includes: using a first network of the recognition model to determine the pixel weight corresponding to each pixel in the target image to obtain a first feature map, wherein the first network is used to enhance the recognition model's focus on the license plate region, and the pixel weight of the license plate region in the first feature map is greater than the pixel weight of other regions; using a second network of the recognition model to adjust the pixel weight in the first feature map to obtain a second feature map, wherein the second network is used to enhance the recognition model's ability to model global information of the license plate region; using the detection head of the recognition model to predict the second feature map to obtain a prediction result, wherein the prediction result includes the position and category of the license plate to be identified; and determining the license plate region corresponding to the prediction result.

[0011] In some embodiments of this application, the loss function of the recognition model during training is determined in the following manner: determining a first loss of the recognition model, wherein the first loss is used to reflect the degree of matching between the predicted bounding box of the license plate region in the training image and the real bounding box; determining a second loss of the recognition model, wherein the second loss is used to reflect the accuracy of the recognition model in predicting non-license plate regions in the training image; determining a third loss of the recognition model, wherein the third loss is used to reflect the accuracy of the recognition model in predicting the categories of different objects in the training image; determining a fourth loss of the recognition model, wherein the fourth loss is used to reflect the sensitivity of the second network to pixel weight adjustments in the training image; and determining a loss function that corresponds to the first loss, the second loss, the third loss, and the fourth loss.

[0012] According to another aspect of the embodiments of this application, a license plate recognition apparatus is also provided, comprising: an acquisition module, configured to acquire a target image and determine a license plate region corresponding to a license plate to be recognized from the target image, wherein the target image contains at least one license plate to be recognized; a first determination module, configured to determine a sharpness index corresponding to the license plate region, wherein the sharpness index is used to quantify the richness of license plate details in the license plate region; a second determination module, configured to determine a region type corresponding to the sharpness index, wherein the region type is used to reflect the minimum complexity that a recognition network for recognizing the license plate region needs to satisfy; and a recognition module, configured to perform character recognition on the license plate region using a recognition network corresponding to the region type to obtain character information of the license plate to be recognized.

[0013] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the above-described method for license plate recognition.

[0014] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-described license plate recognition method by running the computer program.

[0015] According to another aspect of the embodiments of this application, a computer program product is also provided, including computer instructions that, when executed by a processor, implement the above-described method for license plate recognition.

[0016] In this embodiment, an adaptive recognition strategy is adopted. By dynamically analyzing the clarity of the license plate area in the target image, the most suitable recognition network is accurately matched, thereby achieving efficient and accurate license plate character recognition. This solves the technical problem of low recognition efficiency caused by using a unified recognition network to recognize license plate images in related technologies. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a hardware structure block diagram of a computer terminal for a license plate recognition method according to an embodiment of this application;

[0019] Figure 2 This is a flowchart of a license plate recognition method according to an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of the overall process of a license plate recognition method according to an embodiment of this application;

[0021] Figure 4 This is a schematic diagram of the structure of a license plate recognition device according to an embodiment of this application. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] With the increasing number of vehicles, license plate recognition has become increasingly important in areas such as traffic management and smart parking. Traditional license plate detection methods often rely on manual feature extraction and simple models, making it difficult to handle real-world scenarios such as complex backgrounds and varying lighting conditions. Furthermore, although license plate recognition systems combining YOLO (You Only Look Once) and CRNN (Convolutional Recurrent Neural Network) have achieved some success, they still have the following limitations:

[0025] (1) Multi-target and occlusion problem: The traditional YOLO algorithm has insufficient recall rate for detecting dense or partially occluded license plates in complex environments such as rain, fog, and low light.

[0026] (2) Deformation sensitivity: Convolutional networks with fixed receptive fields are difficult to adapt to non-standard license plates such as tilted or curved ones;

[0027] (3) Poor recognition of low-quality images: The standard CRNN model lacks a semantic verification mechanism and has a high misrecognition rate for blurry, tilted, and low-resolution license plate characters;

[0028] (4) Lack of privacy protection: Many related technologies use plaintext to store license plate information, which poses a risk of data leakage;

[0029] (5) Lack of adaptive capability: The license plate recognition system used in related technologies lacks the ability to dynamically adjust the recognition strategy to adapt to different image qualities under different image inputs.

[0030] To address the aforementioned technical problems, this application provides corresponding solutions, which are detailed below.

[0031] The license plate recognition method embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal for implementing a license plate recognition method is shown. Figure 1 As shown, the computer terminal 10 may include one or more processors (shown as 102a, 102b, ..., 102n in the figure) (the processor may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions connected via wired and / or wireless networks. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art will understand that... Figure 1The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0032] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be implemented wholly or partially as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be wholly or partially integrated into any other element in the computer terminal 10. As involved in the embodiments of this application, the data processing circuits serve as processor control (e.g., selection of a variable resistor termination path connected to an interface).

[0033] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the license plate recognition method in this embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned license plate recognition method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0034] The transmission module 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission module 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission module 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0035] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10.

[0036] It should be noted here that, in some optional embodiments, the above... Figure 1The computer terminal shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 1 This is only one instance of a specific particular instance, and is intended to illustrate the types of components that may exist in the aforementioned computer terminal.

[0037] In the above operating environment, this application provides a method embodiment for license plate recognition. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0038] Figure 2 This is a flowchart of a license plate recognition method according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:

[0039] Step S202: Obtain the target image and determine the license plate area corresponding to the license plate to be identified from the target image, wherein the target image contains at least one license plate to be identified.

[0040] In step S202 above, the target image refers to the original image or video frame that needs to be recognized by license plate recognition, including but not limited to various acquisition devices such as urban road monitoring cameras and highway checkpoint cameras, and the license plate area refers to the specific image area in the target image that is recognized as the target license plate.

[0041] After acquiring the target images, annotation tools such as Labelme can be used to generate an initial dataset containing vehicle and license plate information. The dataset supports COCO (Common Objects in Context) or VOC (The Visual Object Classes Challenge) formats to ensure dataset diversity and annotation accuracy. Based on the initial dataset, targeted data augmentation strategies can be added, such as simulating different weather conditions (rain, fog, low light at night), motion blur, etc., using a Generative Adversarial Network (GAN) to generate synthetic data to improve the model's adaptability to complex environments. The formula is expressed as:

[0042] I enhanced =G(I original ,φ)

[0043] Among them, I originalLet G be the original image, φ be the generative adversarial network (GAN), and I be the parameters of the GAN, including but not limited to generator parameters (such as weights and biases of convolutional layers, fully connected layers, activation function layers, etc.), discriminator parameters, and loss function parameters, such as generator loss (e.g., minimizing the discriminator's error in judging the "realism" of the generated image) and discriminator loss (e.g., maximizing the correct classification of real and generated data). enhanced This is the enhanced image.

[0044] In some embodiments of this application, preprocessing of the target image includes: receiving the image to be identified (i.e., the target image) from a sensor or database, and simultaneously introducing a super-resolution reconstruction module to enhance the license plate image, thereby improving the image quality of the low-resolution license plate image and improving the accuracy of subsequent character recognition. For example, residual learning is used through SRResNet (Super-Resolution Residual Network) to improve the training efficiency and reconstruction quality of the network, and the image quality is improved by learning the mapping relationship between low-resolution and high-resolution images.

[0045] Furthermore, adaptive spatial attention networks can be introduced into the recognition model to enhance its ability to detect multiple license plates and tilted license plates. Through the attention mechanism, the model can focus more on the license plate region, improving detection accuracy and robustness. For example, introducing a Transformer module into the backbone network of YOLOv5 leverages its powerful feature extraction capabilities and global information modeling ability to improve the model's detection performance in complex backgrounds or occlusion conditions.

[0046] In some embodiments of this application, the license plate region corresponding to the license plate to be identified can be determined from the target image in the following manner: A first network of the recognition model is used to determine the pixel weight corresponding to each pixel in the target image to obtain a first feature map, wherein the first network is used to enhance the recognition model's focus on the license plate region, and the pixel weight of the license plate region in the first feature map is greater than the pixel weight of other regions; a second network of the recognition model is used to adjust the pixel weights in the first feature map to obtain a second feature map, wherein the second network is used to enhance the recognition model's ability to model global information of the license plate region; the detection head of the recognition model is used to predict the second feature map to obtain a prediction result, wherein the prediction result includes the position and category of the license plate to be identified; and the license plate region corresponding to the prediction result is determined.

[0047] Specifically, the first network enhances the recognition model's focus on the license plate region. By learning the saliency of different regions, it assigns a weight to each pixel, making the license plate region more prominent in the feature map and helping the model to locate the license plate more accurately. The second network enhances the recognition model's ability to model global information about the license plate region, that is, to understand the contextual relationship of the license plate in the image. In complex image environments, the second network helps the model better understand the relationship between the license plate and other objects, improving the robustness of detection.

[0048] For example, an adaptive spatial attention network (the first network) can be used to process the target image, assigning a weight to each pixel that reflects its importance in detecting license plates. In practice, the weights can be dynamically adjusted based on pixel location and image content, with higher weights for the license plate region. In complex backgrounds and under occlusion conditions, license plates may be unclear or partially missing, and traditional object detection models may fail to effectively locate them. By introducing an adaptive spatial attention network, the license plate region can be highlighted, accurately identifying the license plate location even under partial occlusion or complex backgrounds.

[0049] Faced with challenges such as license plates being tilted, curved, or having variable positions in images, traditional convolutional neural networks may not be able to accurately model the global structural information of license plates. To address this issue, based on the first feature map obtained after processing by an adaptive spatial attention network, a Transformer module (second network) can be used to further adjust pixel weights to enhance the modeling of global information in the license plate region. This module is based on a self-attention mechanism and can capture long-distance dependencies between pixels, thereby highlighting the global features of the license plate region in the image feature map.

[0050] The loss function of the recognition model during training can be determined as follows: First, determine the first loss of the recognition model, which reflects the degree of matching between the predicted bounding box and the ground truth bounding box of the license plate region in the training image; second, determine the second loss of the recognition model, which reflects the accuracy of the recognition model in predicting non-license plate regions in the training image; third, determine the third loss of the recognition model, which reflects the accuracy of the recognition model in predicting the categories of different objects in the training image; fourth, determine the fourth loss of the recognition model, which reflects the sensitivity of the second network to pixel weight adjustments in the training image; finally, determine the loss function that corresponds to the first, second, third, and fourth losses.

[0051] Specifically, the first loss reflects the error term of the recognition model in matching the predicted bounding box of the license plate region in the training image with the real bounding box. In the license plate detection task, accurately predicting the bounding box of the license plate is crucial, so the first loss is used to quantify the accuracy of the model in bounding box localization. The second loss measures the accuracy of the recognition model in predicting non-license plate regions in the training image. In license plate recognition, reducing false detections of non-license plate regions is equally important. The second loss helps the model learn to ignore non-license plate regions and avoid false alarms. The third loss reflects the accuracy of the recognition model in predicting different object categories in the training image. Although this application mainly focuses on license plates, it may be necessary to distinguish other objects, such as signs and pedestrians, in complex scenes to avoid interference and ensure the purity of license plate recognition. The fourth loss measures the sensitivity of the second network (such as the Transformer module) to the adjustment of pixel weights in the training image. During the optimization process, the fourth loss ensures that the Transformer can appropriately adjust the weights to strengthen the features of the license plate region without over-adjusting and weakening the model's generalization ability.

[0052] In some embodiments of this application, the first loss can be determined by calculating the positional deviation and magnitude difference between the predicted bounding box and the ground truth bounding box, such as IoU (Intersection over Union) loss and L1 / L2 loss. The second loss can use classification loss (such as softmax cross-entropy loss) to measure the model's confidence in predicting non-license plate areas. When the model predicts an area as a license plate, but the area is actually background or other objects, the second loss will penalize this prediction, prompting the model to reduce the false recognition rate. The third loss can also use classification loss to quantify the model's accuracy in recognizing various types of objects. The fourth loss aims to optimize the weight adjustment function of the Transformer module, ensuring that it both highlights the features of the license plate area and maintains sensitivity to global information. For example, this effect can be measured by calculating the difference between the feature maps before and after adjustment.

[0053] The comprehensive loss function can be created by weighting the first, second, third, and fourth losses to ensure that the model is optimized in bounding box prediction, false positives in non-license plate areas, object category prediction, and Transformer weight adjustment. Its calculation formula is as follows:

[0054] L total =L obj +αL noobj +βL class +γL transformer

[0055] Among them, L obj For the loss in the target area (first loss), L noobjFor the loss in the targetless region (second loss), L class For category prediction loss (third loss), L transformer α represents the loss of the Transformer module (the fourth loss), and α, β, and γ are the balancing parameters.

[0056] Step S204: Determine the clarity index corresponding to the license plate area, wherein the clarity index is used to quantify the richness of license plate details in the license plate area.

[0057] In step S204 above, the sharpness index refers to a numerical value that quantifies the richness of detail information in the license plate in the image. It can be related to attributes such as image resolution, contrast, and sharpness. In a license plate recognition system, the sharpness index helps the system judge the quality of the license plate image and determine subsequent processing strategies, such as whether super-resolution reconstruction is needed or which recognition model to use.

[0058] It should be noted that after obtaining the license plate area in the target image based on the recognition model, perspective transformation can be performed on the extracted license plate image to correct the distortion caused by the shooting angle.

[0059] In some embodiments of this application, the sharpness index corresponding to the license plate area can be determined by: performing edge detection on the license plate area to obtain detection results, wherein the detection results are used to reflect the local change rate of pixels in the license plate area; determining the pixel value variance of the pixels in the license plate area corresponding to the detection results, and determining the sharpness index corresponding to the pixel value variance.

[0060] Edge detection is used to identify and enhance abrupt changes in pixel brightness or color in an image. These changes typically correspond to the boundaries or contours of objects. In this embodiment, edge detection is used to highlight character boundaries within the license plate area to more accurately assess the sharpness of the license plate image. For example, the Sobel operator can be used for edge detection, identifying edges by calculating the gradients of the image in two directions (the gradients of the image in the x and y directions). Under complex lighting conditions, the edges of license plate characters may become blurred, making recognition difficult. By using the Sobel operator to perform edge detection on the license plate area, this application can enhance the contours of the license plate characters, clearly capturing character edges even under adverse conditions such as low light or backlight, thus improving the accuracy of subsequent recognition.

[0061] Local rate of change refers to the speed or extent of change in pixel values ​​within the license plate area after edge detection. In license plate images, a higher local rate of change means a clearer image and more distinct character edges.

[0062] Pixel variance is a statistical metric used to measure the difference in pixel values ​​in an image. High variance usually indicates that the image has rich detail and contrast. In license plate recognition systems, calculating the pixel variance of the license plate area can serve as a sharpness indicator, helping the system to evaluate the quality of the license plate image.

[0063] Step S206: Determine the region type corresponding to the clarity index, wherein the region type is used to reflect the complexity that the recognition network for recognizing the license plate region must at least meet.

[0064] In step S206 above, the region type refers to the system classifying license plate regions into different types based on the clarity index. These types reflect the complexity required by the recognition network when recognizing license plates. For example, high-resolution license plate regions may be classified as "simple type," while low-resolution license plate regions may be classified as "complex type."

[0065] For example, a sharpness threshold can be set for classification. That is, a pre-set sharpness threshold is used to judge the sharpness. When the sharpness index of the license plate area is higher than the threshold, it is classified as "simple type", which is suitable for recognition using a faster CNN (Convolutional Neural Network); when the sharpness index is lower than the threshold, it is classified as "complex type", which requires a more complex CRNN for recognition to improve recognition accuracy.

[0066] In complex environments, the clarity of license plate images varies greatly. By setting a clarity threshold, the type of license plate area can be automatically determined, and an appropriate recognition network can be selected for processing. This ensures the accuracy of recognition, saves computing resources, and improves the efficiency and real-time performance of the system.

[0067] For example, a machine learning model (such as support vector machine, random forest, etc.) can be trained to predict the type of license plate area based on sharpness indicators and other factors that may affect recognition (such as angle, degree of occlusion, etc.). By inputting the license plate area data into the model, the system can obtain a more detailed and accurate area type classification, thereby selecting the most suitable recognition network.

[0068] In practical applications, in addition to clarity metrics, the accuracy and efficiency of license plate recognition are also affected by factors such as angle, degree of occlusion, and license plate deformation. By introducing machine learning models to predict region types, multiple factors can be considered comprehensively, the complexity of the license plate region can be judged more accurately, and the most suitable recognition strategy can be selected to improve the robustness and recognition accuracy of the system.

[0069] In some embodiments of this application, the machine learning model can be trained in the following ways:

[0070] (1) Collect a large amount of license plate image data, including license plate images taken under various conditions such as various angles, different degrees of occlusion, and license plate deformation. Each image should contain features such as sharpness indicators (e.g., the pixel value variance calculated in the previous step), license plate angle, occlusion ratio, and the region type marked by experts (simple or complex).

[0071] (2) Feature engineering is performed on the collected data, including data cleaning, feature selection, and feature engineering to improve the model's predictive performance. Data cleaning removes noise and outliers, feature selection ensures that only the features that contribute most to the model's prediction are retained, and feature engineering may include creating interactive features or using dimensionality reduction techniques. For example, for the occlusion ratio, we can discretize it into several intervals, such as "<10% occlusion", "10%-30% occlusion", and ">30% occlusion", to better reflect the impact of the degree of occlusion on the recognition difficulty. In addition, the license plate angle and occlusion ratio can be combined with the clarity index to create new composite features to more comprehensively reflect the complexity of the license plate area.

[0072] (3) Train the model using a machine learning framework (such as Scikit-Learn), such as a support vector machine (SVM) or random forest (RF), using feature data as input and region type as the output target. Select model parameters through cross-validation to optimize prediction performance.

[0073] During the recognition process, the features of the license plate image are input into a trained model to predict its region type. Based on the prediction result, the appropriate recognition network is selected for further processing (e.g., using CNN to recognize simple license plate regions and CRNN to recognize complex license plate regions). Specifically, when the system acquires a new license plate image, it first calculates its features (clarity index, license plate angle, occlusion ratio, etc.), and then inputs them into a trained machine learning model. The model outputs a predicted region type, and the system selects the appropriate recognition network accordingly. For example, a license plate region with low clarity index, high occlusion ratio, and large angle skew may be predicted by the model as "complex type," thus triggering the use of the CRNN recognition network.

[0074] Step S208: Use a recognition network corresponding to the region type to perform character recognition on the license plate region to obtain the character information of the license plate to be recognized.

[0075] In step S208 above, the recognition network refers to a deep learning network used for license plate character recognition, such as CNN (convolutional neural network) and CRNN (convolutional recurrent neural network). CNN is suitable for processing high-resolution license plate images without complex deformations, while CRNN performs better when processing low-resolution, tilted, or partially obscured license plates.

[0076] Character information refers to the sequence of characters on the license plate obtained after processing by the recognition network, i.e., the license plate number.

[0077] Different license plate image conditions place different demands on the recognition network. Simple images can be recognized quickly, while complex images require more refined processing to improve the recognition rate. By selecting the recognition network based on the region type, the most suitable recognition strategy can be intelligently matched to improve recognition efficiency and accuracy. For example, if the machine learning model predicts the license plate region as "simple type," indicating that the image is clear, the angle is normal, and there is little occlusion, then a CNN can be used for character recognition. If it predicts as "complex type," indicating that the image quality is poor, the angle is tilted, or there is severe occlusion, then a CRNN combined with a language model should be used for recognition.

[0078] Character recognition of the license plate area is performed using a recognition network corresponding to the area type. Specifically: when the area type is the first type, a first neural network is used to perform character recognition of the license plate area; when the area type is the second type, a second neural network is used to perform character recognition of the license plate area. The structural complexity of the second neural network is greater than the result complexity of the first neural network.

[0079] The first type of neural network is mainly used for simple types of license plate region recognition, such as CNN (Convolutional Neural Network). Its structure is relatively simple, its processing speed is fast, and it is suitable for clear, non-tilted, and minimally occluded license plate images. The second type of neural network is specifically designed for complex types of license plate regions, such as the improved CRNN (Convolutional Recurrent Neural Network). Its structure is more complex and it can handle low-resolution, tilted, and occluded conditions, thus improving recognition accuracy.

[0080] For example, after obtaining the license plate area, the corresponding license plate image can be re-evaluated for clarity. This prevents a drop in recognition rate caused by failing to meet the high-resolution criteria after super-resolution reconstruction of a low-resolution image. Furthermore, the subsequent processing method can be automatically selected based on the clarity of the license plate image.

[0081]

[0082] Where C is the image complexity, T is the complexity threshold, and R... high For high definition (Region 1 type), R low For low-resolution (second region type) images, in some embodiments of this application, the Laplacian convolution operator can be used to determine the image resolution.

[0083] (1) For high-resolution images R high We use CNN (Convolutional Neural Network) to directly recognize characters.

[0084] (2) For low-resolution images R low The system employs CRNN (Convolutional Recurrent Neural Network) for recognition and utilizes a pre-trained language model to perform post-processing correction on the recognition results, thereby improving recognition accuracy.

[0085] After using the second neural network to perform character recognition on the license plate area, the following steps can be performed: obtain the initial recognition result of the second neural network, wherein the initial recognition result includes the probability distribution of the character sequence of the license plate to be recognized; decode the probability distribution of the character sequence to obtain the initial character sequence; use a language model to correct the initial character sequence to obtain the target character sequence, wherein the language model is trained based on historical license plate character sequences, and the target character sequence is used to determine the character information of the license plate to be recognized.

[0086] The character sequence probability distribution refers to the predicted probability of each character output by the second neural network, forming a sequence that represents the likelihood of recognizing the license plate characters. The initial character sequence is the preliminary license plate character recognition result obtained by decoding the character sequence probability distribution (e.g., using a CTC decoder).

[0087] For example, an improved license plate region image after YOLOv5 localization and perspective correction is input into a CRNN model for character recognition. The CRNN model incorporates a language model for post-processing correction, and combines statistical license plate format or dictionary information to correct the recognition results, thereby improving the accuracy of character recognition. The output of the CRNN model is a sequence probability distribution, which is decoded using Connected Temporal Classification (CTC) to obtain the final character sequence.

[0088]

[0089] Where P is the sequence probability distribution output by the CRNN. This is the decoded character sequence.

[0090] In some embodiments of this application, rule-based, statistical, or deep learning language models can be used to correct the initial character sequence obtained from CTC decoding to improve recognition accuracy. For example, a deep learning-based BiLSTM language model can evaluate the probability of a sequence at the character level. If a sequence does not conform to common license plate formats, the model will give a lower probability score, prompting the system to select a more reasonable sequence. Under complex conditions, such as blurred images, tilted license plates, or occlusions, the recognition result of the second neural network may contain some unreasonable or incorrect characters. By introducing a language model for correction, errors in the recognition result can be corrected according to common license plate formats and character combination rules, thereby improving the accuracy of character recognition.

[0091] The initial character sequence is corrected using a language model to obtain the target character sequence, including: comparing the initial character sequence with license plate rules using a first language model, obtaining the comparison result, and correcting the initial character sequence based on the comparison result to obtain the target character sequence, wherein the first language model is a model based on license plate format rules; or, using a second language model to determine the first probability of each character appearing in the initial character sequence, and fusing the first probability with the second probability corresponding to each character in the character sequence probability distribution to obtain a first fusion result; decoding the first fusion result to obtain the target character sequence, wherein the second language model is a model based on the statistical regularity between license plate characters; or, using a third language model to predict the third probability of each character in the initial character sequence, and fusing the third probability with the second probability to obtain a second fusion result; decoding the second fusion result to obtain the target character sequence, wherein the third language model includes a model based on a bidirectional long short-term memory network.

[0092] The first language model is based on license plate format rules and is used to check and correct sequences to conform to the standard format of license plates in a specific region. The second language model is based on statistical patterns between license plate characters (such as N-gram models) and is used to estimate the probability of character sequences to correct possible recognition errors. The third language model is a deep learning language model based on Bidirectional Long Short-Term Memory (BiLSTM) networks, which can capture long-distance dependencies between characters and improve the reasonableness of the recognition results.

[0093] Specifically, the first language model performs a validity check on the initial character sequence obtained by CTC decoding according to the preset license plate format rules. If the sequence does not conform to the rules, the model will automatically correct it, such as replacing illegal characters or filling in missing characters according to the context.

[0094] The second language model (such as the N-gram model) estimates the probability of each sequence by statistically analyzing the frequency of occurrence of characters in historical license plate data. The second probability output by the CRNN can be fused with the first probability of the N-gram model (e.g., by using a weighted average) to obtain a more accurate first fusion result. This fusion result is then decoded to obtain the target character sequence with the highest probability.

[0095] For example, a Log-linear combination (i.e., log-linear combination) can be used to obtain the first fusion result. Specifically, the second probability output by the CRNN and the first probability output by the N-gram model are converted into logarithmic values to avoid numerical instability problems that may be caused by floating-point multiplication; the logarithmic probabilities of the two models are combined using weighted summation to obtain the logarithmic probability of the first fusion result; the logarithmic probability of the first fusion result is converted back to the original probability form; the first fusion result is decoded to obtain the target character sequence with the highest probability.

[0096] The third language model (such as a BiLSTM-based model) can learn more complex character sequence patterns, capture long-distance dependencies between characters through bidirectional LSTM units, thereby estimating the third probability of the initial character sequence, fusing it with the second probability to obtain the second fusion result, and then decoding to obtain the target character sequence.

[0097] For example, a Softmax fusion strategy can be adopted. In a license plate recognition system, the Softmax function can be used to combine the second probability (from the CRNN) and the third probability (from the third language model based on BiLSTM). Specifically, for each possible character sequence, the prediction probabilities of the CRNN and BiLSTM models are calculated separately; the prediction probabilities of each model are converted into scores and then converted into a probability distribution through the Softmax function. For example, for the sequence "京A12345", there are the second probability from the CRNN and the third probability from the BiLSTM; these scores are converted into a new probability distribution through the Softmax function to reflect the relative confidence of the models. In the Softmax function, when the image clarity is high (such as greater than a preset threshold), the weight of the BiLSTM can be reduced and more reliance can be placed on the output of the CRNN. When the image quality is poor or the license plate character form is irregular, the weight of the BiLSTM is increased to utilize its grasp of long-sequence dependencies to improve the recognition accuracy; a probability maximization decoding algorithm such as BeamSearch is used to find the most likely sequence from the probability distribution after Softmax fusion.

[0098] In some embodiments of the present application, the introduced language model provides the following three solutions:

[0099] (1) Rule-based language model: Specify strong format constraints according to license plate rules in a region or country. For example, the fixed structure of the license plate: For a regular license plate, it is "province Chinese character + letter + 5-digit letter / number", and for a new energy license plate, it is "province Chinese character + letter + D / F + 5-digit letter / number". <从

[0100] (2) Statistic-based language model (such as N-gram model), including: <从

[0101] Training data: Collect historical license plate databases (such as millions of real license plate data).

[0102] Probabilistic modeling: Calculating the conditional probability of character combinations:

[0103]

[0104] Wherein, P(c i |c i-1 ,c i-2 ) is given the first two characters c i-1 ,c i-2 In this case, the next character is c. i conditional probability; count(c i-2 c i-1 c i ) is a sequence c i-2 c i-1 c i The number of times it appears; count(c i-2 c i-1 ) is a sequence c i-2 c i-1 Number of times it appears.

[0105] Decoding optimization: In the CTC decoding stage of CRNN, character probabilities are fused with language model probabilities.

[0106] Score = logP CRNN (y)+λlogP N-gram (y)

[0107] Wherein, logP CRNN (y) represents the logarithmic probability (i.e., the second probability) of the sequence y output by the CRNN model; logP N-gram (y) represents the log probability (i.e., the first probability) of sequence y based on the N-gram language model; λ is a weighting parameter used to balance the contributions of the log probability output by the CRNN model and the log probability output by the N-gram model to the final score; Score is the total score after fusion (i.e., the first fusion result), which combines the outputs of the image recognition model (CRNN) and the language model (N-gram), and is used to evaluate and select the most likely character sequence during the decoding process.

[0108] (3) Deep learning language models (such as BiLSTM / Transformer), including:

[0109] Training data: The training is conducted using a large amount of real license plate data (such as millions of records) to ensure that as many character combinations and license plate formats as possible are covered.

[0110] Model structure: The input characters are converted into fixed-dimensional vector representations. A bidirectional LSTM is used to capture the dependencies between characters in the sequence. The output of the LSTM layer is mapped to the final character category, which is then converted into the probability distribution of the character category.

[0111] Autoregressive probability prediction: The model predicts the probability distribution of characters at each time step, forming a probability chain of sequences. Teacher Forcing is used to optimize this process during training, that is, the real target is used as the input for the next step at each step.

[0112] Combining CRNN with language model scores: In the CTC decoding stage of CRNN, the probability of character recognition and the language model score based on deep learning are fused. Joint decoding can be achieved through the Beam Search algorithm, which comprehensively considers the outputs of the two models and selects the most likely license plate character sequence.

[0113] In some embodiments of this application, after obtaining the character information of the license plate, it can be encrypted before storage to ensure data security. For example, adding a license plate information encryption module or real-time anonymization function to the system can protect user privacy and meet the requirements of regions with strict regulations.

[0114] L encrypted =E(L original ,K)

[0115] Among them, L original The original license plate information is represented by E, the encryption function is represented by K, and L is represented by L. encrypted This is the encrypted license plate information.

[0116] Through steps S202 to S208 above, an adaptive recognition strategy is adopted. By dynamically analyzing the clarity of the license plate area in the target image, the goal of accurately matching the most suitable recognition network is achieved. This results in efficient and accurate license plate character recognition technology, thereby solving the technical problem of low recognition efficiency caused by using a unified recognition network to recognize license plate images in related technologies.

[0117] To facilitate understanding of the above process, some specific embodiments are described below.

[0118] (1) License plate recognition in complex urban road conditions. Application scenarios: Traffic monitoring systems on main urban roads, which have complex situations such as multiple vehicles running in parallel, some license plates being obscured, and frequent changes in lighting conditions (such as tree shade, tunnel entrances and exits).

[0119] 1) Data acquisition and enhancement.

[0120] Vehicle images were collected from the city's main roads at different times and under different weather conditions (sunny, cloudy, and light rain). The labelme annotation tool was used to annotate the license plate locations and information to generate an initial dataset in COCO format.

[0121] Generative Adversarial Networks (GANs) are used to simulate environments such as rain, fog, and low light at night to enhance the initial dataset. For example, for a vehicle image taken on a sunny day, a blurred version under light rain conditions is generated using a GAN, expressed by the formula I. enhanced =G(I original ,φ), where I original The image represents the original sunny day image, G is the trained generative adversarial network, φ represents the network parameters, and I... enhanced An enhanced image of light rain.

[0122] 2) Data preprocessing.

[0123] For the input image to be recognized, if it is low-resolution (e.g., blurred due to being taken from a distance or insufficient lighting), the SRResNet super-resolution reconstruction module is used for processing. This module improves the resolution and clarity of the license plate area by learning the mapping relationship between low-resolution and high-resolution images. For example, it can reconstruct a blurry license plate image with a resolution of 128×64 into a high-resolution image of 256×128, thereby improving the accuracy of subsequent character recognition.

[0124] 3) Improved YOLOv5 detection.

[0125] An adaptive spatial attention network and a Transformer module are introduced into the YOLOv5 backbone network. The adaptive spatial attention network allows the model to focus more on the license plate region and ignore background interference; the Transformer module enhances the modeling ability of global information and improves detection performance in complex backgrounds. The improved objective function is L. total =L obj +αL noobj +βL class +γL transformer L obj For the purpose of loss in the target area, L noobj For loss without a target area, L class For category prediction loss, L transformer For the loss of the Transformer module, α, β, and γ are set to 0.5, 0.3, and 0.2 respectively for balancing.

[0126] The optimized YOLOv5 model is used to locate license plate regions in the input image, for example, in a multi-vehicle parallel scene, to accurately detect the location of partially occluded license plates.

[0127] 4) Perspective correction.

[0128] Perform perspective transformation on the extracted license plate image to correct the deformation caused by the shooting angle. For example, for the license plate image of a vehicle traveling obliquely, it is corrected to a front view through perspective transformation to facilitate subsequent character recognition.

[0129] 5) Adaptive clarity discrimination.

[0130] Calculate the image complexity C using the Laplacian convolution operator, set the complexity threshold T = 100. If C > T, it is determined as a high-clarity image, and CNN is directly used for character recognition; if C ≤ T, it is determined as a low-clarity image, and CRNN combined with a language model is used for recognition.

[0131] For high-clarity images, such as license plate images with improved clarity after being processed by SRResNet, use the CNN model to quickly and accurately recognize characters.

[0132] For low-clarity images, such as blurred license plate images taken in light rain weather, input them into the CRNN model and perform post-processing correction using a rule-based language model. The rule-based language model constrains the recognition results according to the license plate format (province Chinese character + letter + 5 letters / numbers). For example, if the letter is not followed by a letter after the province Chinese character in the recognition result, it is corrected.

[0133] 6) Information encryption and storage.

[0134] Perform encryption processing on the finally recognized license plate information "Beijing A12345", use the AES encryption algorithm, and the encryption function is L encrypted =E(L original ,K), where L original is the original license plate information, E is the AES encryption function, K is the encryption key "1234567890abcdef", and the encrypted information is stored in ciphertext form, such as "U2FsdGVkX19fX05a5d5f5c3b2a9e7a6f3c2b1", to ensure data security.

[0135] (2) License plate recognition in bad weather, application scenario: highway. In case of heavy rain, fog and other bad weather, the license plate may be blocked by rain and fog, and the image clarity is extremely low.

[0136] 1) Data collection and augmentation. <​​​​​​

[0139] For the incoming extremely low - resolution license plate images, use SRResNet for multiple super - resolution reconstructions to restore image details as much as possible.

[0140] 3) Improve YOLOv5 detection.

[0141] The improved YOLOv5 model can still effectively detect partially occluded license plates in bad weather, such as license plates with some areas covered by rain.

[0142] 4) Adaptive clarity discrimination.

[0143] Since the image complexity C is usually much lower than the threshold T in bad weather, it is determined as a low - clarity image, and CRNN combined with a deep - learning language model (BiLSTM) is used for recognition.

[0144] The deep - learning language model is trained with a large amount of real license plate data, converts the input characters into vector representations, captures the forward and backward dependencies of the character sequence through bidirectional LSTM. In the CTC decoding stage of CRNN, the Beam Search algorithm is used to fuse the scores of CRNN and BiLSTM to select the most likely license plate character sequence. For example, for a license plate "Beijing B56789" with some characters blurred due to fog occlusion, the BiLSTM model corrects the misrecognition of "Beijing B5678X" that may occur in CRNN according to the forward and backward dependencies of the character sequence.

[0145] 5) Information encryption and storage.

[0146] Encrypt and store the final recognition result to ensure the security and reliability of the license plate information recognized in bad weather.

[0147] Figure 3 It is a schematic diagram of the overall process of a license plate recognition method according to an embodiment of the present application. As Figure 3 shown, in some embodiments of the present application, the license plate recognition method may include the following steps:

[0148] Step S302: Data collection and enhancement.

[0149] Data collection: Collect image data containing vehicles and their license plates from various sources, such as camera monitoring, public datasets, etc. Pay special attention to license plate images under different environmental conditions (such as rain, fog, low light at night) to cover as many recognition challenges as possible.

[0150] Data enhancement: Improve the robustness and generalization ability of the model by increasing the diversity of images. Use deep - learning techniques, such as generative adversarial networks (GAN), to generate license plate images simulating different weather conditions and different shooting angles. At the same time, ensure that the dataset format is compatible with COCO or VOC for convenient model training.

[0151] Step S304: Data preprocessing.

[0152] A super-resolution reconstruction module, such as SRResNet, is introduced to enhance the low-resolution input license plate image, improving its resolution and detail clarity to facilitate subsequent character recognition. Image preprocessing techniques, such as grayscale conversion, binarization, and edge detection, are used to eliminate noise and enhance contrast, ensuring that the features of the license plate area are more prominent.

[0153] Step S306: Improved YOLOv5 detection.

[0154] Adaptive Spatial Attention Network and Transformer module are introduced into the backbone network of YOLOv5 to enhance the model's ability to capture local features and understand global information. The Adaptive Spatial Attention Network focuses on the license plate area and reduces background interference, while the Transformer module enhances the model's detection performance in complex environments, such as rainy or foggy weather or partial occlusion.

[0155] Furthermore, adaptive spatial attention and Transformer loss can be combined with the traditional YOLOv5 loss function to optimize the model.

[0156] Step S308: Perspective correction.

[0157] A perspective transformation algorithm is used to correct the detected license plate area, resolving issues such as tilted or bent license plates. For example, by calculating the four corner points of the license plate, the correct perspective relationship is found in the image coordinate system, and then a geometric transformation is performed to correct the license plate into a front view.

[0158] Step S310: Adaptive sharpness determination.

[0159] Using the Laplacian operator or image sharpness assessment algorithm, the sharpness of the license plate image is automatically determined to meet the high-definition threshold based on its quality and detail.

[0160] Step S312: CNN recognition in high definition.

[0161] When the license plate is determined to be of high resolution, a CNN model is used directly for character recognition. The CNN uses convolutional and pooling layers to extract image features, and then uses fully connected layers to classify the features and recognize each character on the license plate.

[0162] Step S314: Optimize CRNN recognition under low resolution.

[0163] The low-resolution license plate image is processed by the optimized CRNN. The optimization measures include, but are not limited to, increasing data augmentation and adjusting the model architecture to better adapt to the recognition of blurred image features.

[0164] Step S316: Correct using a language model.

[0165] Language models (such as N-gram models, BiLSTM-based models) are used to verify and correct the preliminary recognition results. For example, based on statistical rules or sequence patterns, the language model can adjust the recognition probability of the character sequence to ensure the rationality of the final result. For the recognition results of the optimized CRNN, the language model corrects possible misidentifications by evaluating the sequence probability. For example, it corrects "Beijing A1234X" to "Beijing A12345".

[0166] Step S318: Encrypt and store information.

[0167] The recognized license plate information is encrypted before storage. A symmetric encryption algorithm such as AES is used to ensure data security and protect user privacy. The encryption process includes converting the license plate information into ciphertext and uploading it to a database or cloud storage service. The encryption key is securely managed by the system to prevent unauthorized access.

[0168] This application combines the improved YOLOv5 object detection algorithm and the CRNN model to achieve precise positioning and character recognition of the license plate area. Especially when dealing with low-quality images, it can significantly improve the recognition accuracy and solve the limitations of traditional methods in practical scenarios such as complex backgrounds and lighting changes. Moreover, an adaptive clarity discrimination mechanism is designed, which can automatically select an appropriate processing path (such as multiple recognition networks like CNN or optimized CRNN) according to the clarity of the license plate image to adapt to input images of different qualities. In addition, this application can also use a language model combined with statistical license plate formats or dictionary information to post-process and correct the preliminary recognition results, further correcting possible errors, ensuring that the final output license plate information is both logical and in line with the actual situation, thereby improving the accuracy of the recognition results. Finally, in the last stage of information processing, the recognized license plate information is encrypted and stored to protect personal privacy and data security and prevent information leakage, which is particularly important for meeting the requirements of regions with strict regulations and provides an additional data protection layer.

[0169] Figure 4 It is a structural diagram of a license plate recognition device according to an embodiment of the present application, as Figure 4 shown. The device includes:

[0170] An acquisition module 402, configured to acquire a target image and determine a license plate area corresponding to the license plate to be recognized from the target image, where the target image contains at least one license plate to be recognized;

[0171] The first determining module 404 is used to determine the clarity index corresponding to the license plate area, wherein the clarity index is used to quantify the richness of the license plate details in the license plate area;

[0172] The second determining module 406 is used to determine the region type corresponding to the clarity index, wherein the region type is used to reflect the complexity that the recognition network for recognizing the license plate region must at least meet.

[0173] The recognition module 408 is used to perform character recognition on the license plate area using a recognition network corresponding to the area type, so as to obtain the character information of the license plate to be recognized.

[0174] It should be noted that, Figure 4 The license plate recognition device shown is used to perform... Figure 2 The license plate recognition method shown, therefore Figure 2 The relevant explanations in the license plate recognition method also apply to Figure 4 The license plate recognition device shown will not be described in detail here.

[0175] This application also provides an electronic device, which includes a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the steps of the license plate recognition method implemented in various embodiments of this application.

[0176] For example, the processor performs the following functions by executing program instructions stored in memory: acquiring a target image and determining the license plate region corresponding to the license plate to be identified from the target image, wherein the target image contains at least one license plate to be identified; determining a sharpness index corresponding to the license plate region, wherein the sharpness index is used to quantify the richness of license plate details in the license plate region; determining a region type corresponding to the sharpness index, wherein the region type is used to reflect the minimum complexity that the recognition network for recognizing the license plate region must satisfy; and performing character recognition on the license plate region using a recognition network corresponding to the region type to obtain the character information of the license plate to be identified.

[0177] This application also provides a non-volatile storage medium including a stored computer program, wherein the device containing the non-volatile storage medium executes the steps of the license plate recognition method in various embodiments of this application by running the computer program.

[0178] This application also provides a computer program product, including computer instructions that, when executed by a processor, implement the steps of the license plate recognition method in various embodiments of this application.

[0179] This application also provides a computer program that, when executed by a processor, implements the steps of the license plate recognition method in various embodiments of this application.

[0180] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0181] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0182] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0183] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0184] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0185] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0186] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for license plate recognition, characterized in that, include: Acquire a target image and determine the license plate region corresponding to the license plate to be identified from the target image, wherein the target image contains at least one of the license plates to be identified; Determine a clarity index corresponding to the license plate area, wherein the clarity index is used to quantify the richness of license plate details in the license plate area; Determine the region type corresponding to the clarity index, wherein the region type is used to reflect the complexity that the recognition network for recognizing the license plate region must at least meet; The license plate area is used to perform character recognition using a recognition network corresponding to the area type, thereby obtaining the character information of the license plate to be recognized.

2. The method according to claim 1, characterized in that, Character recognition is performed on the license plate area using a recognition network corresponding to the area type, including: When the region type is the first type, a first neural network is used to perform character recognition on the license plate region; When the region type is the second type, a second neural network is used to perform character recognition on the license plate region, wherein the structural complexity of the second neural network is greater than the result complexity of the first neural network shown.

3. The method according to claim 2, characterized in that, After performing character recognition on the license plate area using a second neural network, the method further includes: Obtain the initial recognition result of the second neural network, wherein the initial recognition result includes the probability distribution of the character sequence of the license plate to be recognized; The probability distribution of the character sequence is decoded to obtain the initial character sequence; The initial character sequence is corrected using a language model to obtain a target character sequence, wherein the language model is trained based on historical license plate character sequences, and the target character sequence is used to determine the character information of the license plate to be identified.

4. The method according to claim 3, characterized in that, The initial character sequence is corrected using a language model to obtain the target character sequence, including: The initial character sequence is compared with the license plate rules using a first language model to obtain a comparison result. Based on the comparison result, the initial character sequence is corrected to obtain the target character sequence. Here, the first language model is a model based on license plate format rules; or... A second language model is used to determine the first probability of each character appearing in the initial character sequence, and the first probability is fused with the second probability corresponding to each character in the initial character sequence in the probability distribution of the character sequence to obtain a first fusion result; the first fusion result is decoded to obtain the target character sequence, wherein the second language model is a model based on the statistical regularity between license plate characters; or... A third language model is used to predict the third probability of each character in the initial character sequence, and the third probability is fused with the second probability to obtain a second fusion result; the second fusion result is decoded to obtain the target character sequence, wherein the third language model includes a model based on a bidirectional long short-term memory network.

5. The method according to claim 1, characterized in that, Determining the clarity index corresponding to the license plate area includes: Edge detection is performed on the license plate area to obtain detection results, wherein the detection results are used to reflect the local change rate of pixels in the license plate area; Determine the pixel value variance of the pixels in the license plate area corresponding to the detection result, and determine the sharpness index corresponding to the pixel value variance.

6. The method according to claim 1, characterized in that, Determining the license plate region corresponding to the license plate to be identified from the target image includes: The first network of the recognition model is used to determine the pixel weight corresponding to each pixel in the target image to obtain a first feature map. The first network is used to enhance the recognition model's focus on the license plate area. The pixel weight of the license plate area in the first feature map is greater than the pixel weight of other areas. The second network of the recognition model is used to adjust the pixel weights in the first feature map to obtain a second feature map, wherein the second network is used to enhance the recognition model's ability to model global information of the license plate area; The detection head of the recognition model is used to predict the second feature map to obtain a prediction result, wherein the prediction result includes the position and category of the license plate to be identified; Determine the license plate region corresponding to the prediction result.

7. The method according to claim 6, characterized in that, The loss function of the recognition model during training is determined in the following way: A first loss of the recognition model is determined, wherein the first loss is used to reflect the degree of matching between the predicted bounding box and the real bounding box of the license plate region in the training image by the recognition model; Determine a second loss for the recognition model, wherein the second loss reflects the accuracy with which the recognition model predicts non-license plate regions in the training image; A third loss is determined for the recognition model, wherein the third loss is used to reflect the accuracy of the recognition model in predicting the categories of different objects in the training images; A fourth loss is determined for the recognition model, wherein the fourth loss is used to reflect the sensitivity of the second network to pixel weight adjustments of the training image; Determine the loss function that corresponds to the first loss, the second loss, the third loss, and the fourth loss.

8. A license plate recognition device, characterized in that, include: An acquisition module is used to acquire a target image and determine the license plate region corresponding to the license plate to be identified from the target image, wherein the target image contains at least one of the license plates to be identified; The first determining module is used to determine the clarity index corresponding to the license plate area, wherein the clarity index is used to quantify the richness of license plate details in the license plate area; The second determining module is used to determine the region type corresponding to the clarity index, wherein the region type is used to reflect the complexity that the recognition network for recognizing the license plate region must at least meet. The recognition module is used to perform character recognition on the license plate area using a recognition network corresponding to the area type, so as to obtain the character information of the license plate to be recognized.

9. An electronic device, characterized in that, include: A memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the license plate recognition method according to any one of claims 1 to 7.

10. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored computer program, wherein the device containing the non-volatile storage medium executes the license plate recognition method according to any one of claims 1 to 7 by running the computer program.

11. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the license plate recognition method according to any one of claims 1 to 7.

Images