A method for improving accurate positioning of license plates

Abstract

The application provides a method for improving accurate positioning of license plates, applying an OCR detection model to detect a license plate, using a heat map as a network output result to detect the license plate, and determining license plate positioning through a license plate key point model, quantizing the model to low bits to realize license plate detection and license plate positioning. The method starts from data acquisition, loads the data to the license plate detection model, further judges whether the license plate can be detected, ends the detection if the license plate cannot be detected, and enters the license plate key point model if the license plate can be detected, corrects the license plate through perspective transformation and outputs, and finally ends the detection. The method makes the license plate detection and license plate positioning more accurate and reduces the quantization difficulty.

Description

Technical Field

[0001] This invention relates to the field of intelligent image processing technology, and in particular to a method for improving the accuracy of license plate positioning. Background Technology

[0002] With the development of computer technology and the widespread application of computer vision principles, the use of computer image processing technology for real-time target detection and tracking has become increasingly popular. Dynamic real-time tracking and localization of targets are used in intelligent transportation systems, intelligent monitoring systems, and military target detection, and it also has wide-ranging applications in locating surgical instruments for medical navigation surgery. The task of target detection is to identify all targets of interest in an image and determine their position and size; it is one of the core problems in the field of machine vision. Due to the different appearances, shapes, and poses of various objects, coupled with interference from factors such as lighting and occlusion during imaging, target detection has always been one of the most challenging problems in the field of machine vision.

[0003] In existing technologies, license plate recognition systems monitor vehicles on the road and automatically extract and process vehicle license plate information. When a vehicle enters the capture area of ​​the license plate recognition system, it triggers the integrated license plate recognition device to capture an image of the vehicle and automatically identify the license plate number. The vehicle detector mainly serves as a trigger; only after being triggered will the integrated license plate recognition device be activated for monitoring and capture, avoiding the need for the integrated license plate recognition device to be constantly on. Currently, license plate detection systems are mostly used at barrier gates. This application is for roadside parking scenarios. In this scenario, the camera is located on the roadside, and the acquired images show significant license plate distortion.

[0004] Currently, traditional license plate detection directly regresses four coordinate points, which is quite difficult and results in inaccurate regression points. Furthermore, the regression coordinate points are not easy to quantize (models have evolved from floating-point to 8-bit, 4-bit, and 2-bit), and low-bit models will have reduced accuracy, leading to inaccurate regression points. Directly regressing coordinates further increases the difficulty and makes quantization challenging. Inaccurate regression points will negatively impact the performance of license plate recognition.

[0005] In addition, the following are commonly used terms in the prior art:

[0006] 1. OCR (Optical Character Recognition) refers to the process by which electronic devices (such as scanners or digital cameras) examine characters printed on paper, determine their shapes by detecting dark and light patterns, and then translate the shapes into computer text using character recognition methods.

[0007] 2. OCR detection: Detects the position of the characters. Summary of the Invention

[0008] To address the issues of inaccurate regression points and difficulty in quantification in traditional license plate detection systems, this application aims to provide a method for accurately locating license plates in a license plate detection system, reducing the difficulty of quantification, and thereby training license plate detection to improve accuracy.

[0009] Specifically, the present invention provides a method for improving the accuracy of license plate positioning. The method involves using an OCR detection model to detect the license plate, using a heatmap as the network output result to detect the license plate, determining the license plate positioning through a license plate key point model, and quantizing the model to low bits to achieve license plate detection and positioning.

[0010] The method begins by acquiring data, loading the data into the license plate detection model, and further determining whether the license plate can be detected. If the license plate cannot be detected, the detection ends; if the license plate can be detected, the method proceeds to the license plate key point model, performs perspective transformation to correct the license plate, and outputs the result, finally ending the detection.

[0011] The method further includes the following steps:

[0012] S1, License Plate Data Generation:

[0013] The license plate is marked with its four corner points. Data enhancement is performed using perspective transformation, simultaneously rotating the plate on the x, y, and z axes to achieve the effect captured by a roadside camera. This involves placing the camera at one of the four corners of a roadside parking space, with the camera position roughly at or equal to the height of the license plate. This results in a larger angle of the license plate captured by the camera. Figure 1 As shown;

[0014] S2, Training the license plate detection model:

[0015] This application uses a segmentation method from OCR detection to detect license plates: First, a DBNet network (using ResNet-18+FPN as the backbone) outputs a heatmap of the text (license plate) segmentation results (i.e., a probability map, where each pixel represents the probability of being a positive sample). Then, a preset threshold (0.3) is used to convert the segmentation result map into a binary image. Finally, the outline of the license plate is found and the license plate is bounded. The process is as follows: Figure 2 As shown;

[0016] In OCR detection, contour finding is a function that already exists; here, the `cv2.findContours` function is used. ResNet-18 is a classic network, and FPN is a network architecture. Figure 2 This can be reflected in the middle;

[0017] S3, License Plate Key Point Training:

[0018] After the license plate is detected, a license plate key point model needs to be added. The model uses an 8-layer CNN convolutional network to output the four corner points of the license plate. The four corner points are located, and then perspective transformation is performed on the image using these four points. The perspective transformation can use existing functions; here, the cv2.warpPerspective function is used to correct the license plate, making it easier to recognize.

[0019] The model trained in step S2 is adapted to license plates under roadside cameras, license plates that are heavily exposed at night, license plates that are dimly lit, and license plates that are tilted.

[0020] The detection model used in the method is the DBNet framework applied to OCR detection, and the model result is a heatmap.

[0021] The license plate detection method described is used in roadside scenarios where the camera acquiring image data is positioned low, resulting in significant distortion of the license plate shape.

[0022] The method may further include:

[0023] S4, Model Quantization:

[0024] The license plate detection model is quantized to 4 bits without loss of accuracy. Since key points require accurate positioning, and because the key point model is small, it is quantized to 8 bits without loss of accuracy. This quantization can directly use existing quantization methods; here, the quantization platform developed by Beijing Junzheng Inheritance Circuit Co., Ltd. (hereinafter referred to as Beijing Junzheng) is used.

[0025] Therefore, the advantage of this application is:

[0026] This method applies OCR detection principles, using binary images as network output to detect license plates and incorporating a license plate key point model. This results in more accurate license plate detection and localization, while reducing quantization complexity. It achieves high accuracy, quantizing to low bits without loss of precision. Attached Figure Description

[0027] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0028] Figure 1 This is a schematic diagram of the license plate detection results in the roadside license plate scenario of this application.

[0029] Figure 2 This is a simplified diagram of the overall network of this application, which shows the FPN structure.

[0030] Figure 3 A schematic diagram of the results of key points of the license plate in this application method.

[0031] Figure 4 This is a simplified flowchart of the vehicle inspection process in the method described in this application.

[0032] Figure 5 This is a flowchart of the main steps of the method described in this application. Detailed Implementation

[0033] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0034] The detection model used in this application is a modified version of the DBNet network applied to OCR detection. OCR detection identifies the position of characters and uses a heatmap (binary image) as the network output. This heatmap method is easy to quantize and lossless. License plates are composed of characters, so this method can be used to detect them.

[0035] Specifically, this application uses OCR detection to detect license plates and accurately locates them using a key point model.

[0036] Further including, such as Figures 1-5 As shown, the method of the present invention relates to a method for improving the accuracy of license plate positioning, the method comprising the following steps:

[0037] S1, License Plate Data Generation:

[0038] The license plate is marked with its four corner points. Since this license plate detection application is located on the roadside, and the camera is positioned low and near the road, the shape of the license plate is significantly distorted. Furthermore, to avoid overexposure, the colors in the image captured by the camera also vary considerably. Therefore, data augmentation needs to be strengthened. Perspective transformation can be used, rotating the image along the x, y, and z axes to achieve the effect seen by a roadside camera, thus enhancing the model's generalization ability. Color augmentation is also necessary to adapt the model to the roadside scene. Data augmentation methods are crucial; they transform limited data into the necessary data, increasing data diversity. Well-prepared data leads to better training results for the same model.

[0039] S2, Training the license plate detection model:

[0040] The license plate detection model is trained using OCR. In the model's output, the license plate area is highlighted in white, while other areas remain black. Post-processing is then used to find the license plate's outline and define it. The trained model exhibits excellent adaptability to license plates detected by roadside cameras, achieving a recall rate of up to 97%. It works well even with heavily exposed, dimly lit, and tilted license plates, although it may not perform well with excessively tilted plates. The results are as follows: Figure 1 As shown;

[0041] S3, License Plate Key Point Training:

[0042] Because license plate detection cannot accurately locate the four corner points of a license plate, which affects the accuracy of license plate recognition, a license plate key point model needs to be added after the license plate is detected to locate the four corner points. This allows for perspective transformation of the image using these four points, correcting the license plate and facilitating license plate recognition. Figure 3 As shown;

[0043] S4, Model Quantization:

[0044] The license plate detection model is quantized to 4 bits without loss of accuracy. Since key points need to be accurately located, and the key point model is very small, quantization to 8 bits is sufficient without loss. The quantized model's runtime is significantly reduced, allowing it to be deployed on a chip.

[0045] like Figure 4 As shown, the process begins by acquiring image data, loading the data into the license plate detection model, and further determining whether the license plate can be detected. If the license plate cannot be detected, the detection ends. If the license plate can be detected, the process proceeds to the license plate key point model, where perspective transformation is performed to correct the license plate and the result is output. Finally, the detection ends.

[0046] Therefore, as Figure 5 As shown, the main implementation steps of this method are as follows:

[0047] S1. License plate data creation;

[0048] S2. Train the license plate detection model;

[0049] S3. Training on key points of license plates.

[0050] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for improving the accuracy of license plate location, characterized in that, The method describes the application of an OCR detection model to detect license plates, using heatmaps as network outputs for license plate detection, and determining license plate location through a license plate key point model. The model is then quantized to low bits to achieve license plate detection and location. The method begins by acquiring data, loading the data into the license plate detection model, and further determining whether the license plate can be detected. If the license plate cannot be detected, the detection process ends. If the license plate can be detected, the process proceeds to the license plate key point model, where perspective transformation is performed to correct the license plate and the result is output. Finally, the detection ends. The method further includes: S1, License Plate Data Creation: The license plate is marked with the four corner points of the license plate. Data enhancement is strengthened by using perspective transformation and rotating the angles simultaneously on the x, y, and z axes to achieve the effect of the roadside camera capturing the license plate. S2, Training the license plate detection model: The license plate is detected using the segmentation method in OCR detection: First, the DBNet network is used, where the backbone uses ResNet-18+FPN, to output the text segmentation result heatmap of the image. The segmentation result map is converted into a binary map using a preset threshold, which is 0.3 here. Finally, the outline of the license plate is found and the license plate is bounded. S3, License Plate Key Point Training: After the license plate is detected, a license plate key point model needs to be added. The model uses an 8-layer CNN convolutional network to output the four corner points of the license plate. The four corner points are located, and then perspective transformation is performed on the image using these four points to correct the license plate and perform license plate recognition.

2. The method for improving the accuracy of license plate positioning according to claim 1, characterized in that, In step S2, the text of the output image is the license plate, and the segmentation result heatmap is a probability map, where each pixel represents the probability of whether it is a positive sample. The ResNet-18 is a classic network, and FPN is a network architecture; in finding the license plate contour, an existing function is used, specifically the cv2.findContours function. The trained license plate detection model is adapted to license plates under roadside cameras, license plates that are heavily exposed at night, license plates that are dimly lit, and license plates that are tilted.

3. The method for improving the accuracy of license plate positioning according to claim 1, characterized in that, The detection model used in the method is the DBNet framework applied to OCR detection.

4. The method for improving the accuracy of license plate positioning according to claim 1, characterized in that, The license plate detection method described is used in roadside scenarios where the camera acquiring image data is positioned low, resulting in significant distortion of the license plate shape.

5. The method for improving the accuracy of license plate positioning according to claim 1, characterized in that, The method may further include: S4, Model Quantization: The license plate detection model is quantized to 4 bits without loss of accuracy; since key points need to be accurately located, and since the key point model is small, it is quantized to 8 bits without loss of accuracy.

Images