A license plate detection and recognition system based on hyperdimensional computing

By improving the YOLOv8 model and combining U2-Net and ultra-dimensional computing technology, the problems of license plate recognition accuracy and speed in complex scenarios are solved, efficient identification of small-target license plates is achieved, and the license plate detection performance is significantly improved.

CN117854053BActive Publication Date: 2024-11-29HUNAN SOFTSENSE TECHNOLOGY CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202311662294.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-05
Publication Date
2024-11-29
Estimated Expiration
2043-12-05

AI Technical Summary

Technical Problem

In complex scenarios, traditional license plate recognition systems are difficult to meet the requirements of accuracy and speed, and cannot effectively identify small target license plates.

Method used

Adopting a license plate recognition system based on improved YOLOv8, the license plate recognition module is improved by improving the loss function, adding feature enhancement module and small object detection head, improving the NMS algorithm, as well as the data processing module based on U2-Net and distortion correction and the license plate recognition module based on ultra-dimensional calculation, to improve the license plate recognition module. Accuracy and speed of license plate detection.

Benefits of technology

It effectively improves the accuracy and speed of license plate detection, especially the ability to identify small targets in complex scenarios, and significantly improves the license plate detection performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117854053B_ABST
    Figure CN117854053B_ABST
Patent Text Reader

Abstract

The present invention discloses a license plate detection and recognition system based on hyperdimensional computing, comprising a license plate detection module based on improved YOLOv8, a data processing module based on U2‑Net and distortion correction, and a license plate recognition module based on hyperdimensional computing, wherein the license plate detection module uses the improved YOLOv8 model to detect license plate photos; further data processing is performed on the detected license plate part picture, and finally the license plate recognition module based on hyperdimensional computing is used to recognize the license plate data after data processing, and the improved YOLOv8 model includes using a loss function based on Repulsion, adding a feature enhancement module, adding a small target detection head, and improving a non-maximum suppression algorithm. The present invention effectively improves the license plate detection performance by improving the YOLOv8 algorithm, and greatly improves the license plate detection speed through the data processing module and the license plate recognition module based on hyperdimensional computing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to small target detection and recognition, and in particular to a license plate detection and recognition system based on hyperdimensional computing. Background Art

[0002] As the number of motor vehicles on the road continues to increase, the traffic environment on the road has gradually become more complicated, and traffic safety has become more important. Therefore, in order to accurately identify motor vehicles on the road and alleviate traffic safety problems, license plate recognition systems have come into being.

[0003] License Plate Recognition System (LPRS) is a widely used technology in the field of machine vision and computer vision in traffic scenarios. Its research began with image processing technology in the 1980s. Due to the rapid development of deep learning technology and artificial neural network technology in recent years, the license plate recognition system has also made great progress. The speed and accuracy of traditional license plate recognition algorithms in specific scenarios can meet daily needs. However, in complex scenarios, traditional license plate recognition systems are difficult to meet the requirements. Therefore, in order to improve the accuracy and speed of license plate recognition systems in complex scenarios, the present invention proposes a license plate recognition system based on improved YOLOv8. Summary of the invention

[0004] Purpose of the invention: In view of the problems existing in the prior art, the present invention provides a license plate recognition system based on improved YOLOv8, which is used to improve the accuracy and speed of license plate recognition in complex scenarios.

[0005] Technical solution: The present invention discloses a license plate detection and recognition system based on hyperdimensional computing, including a license plate data set production module, a license plate detection module based on YOLOv8, a data processing module based on U2-Net and distortion correction, and a license plate recognition module based on hyperdimensional computing;

[0006] The license plate dataset making module includes collecting videos and images and combining them with the CCPD dataset, and making them into a dataset in the YOLO format of YOLOv8;

[0007] The YOLOv8-based license plate detection module is used to detect the photo containing the license plate using the YOLOv8 model to obtain the area containing the license plate;

[0008] The data processing module based on U2-Net and distortion correction is used to process the detected image using U2-net and distortion correction, and convert it into feature information of a one-dimensional structure;

[0009] The license plate recognition module based on hyperdimensional computing is used to recognize the characteristic information of the one-dimensional structure output by the data processing module by using hyperdimensional computing to obtain the license plate information.

[0010] Furthermore, the license plate photo dataset adopts the Chinese city license plate open source dataset CCPD, and only uses part of the data, including CCPD-Rotate and CCPD-Tilt, and adds the collected videos and images to the CCPD dataset to expand its generalization ability for Chinese characters in different provinces.

[0011] Furthermore, the YOLOv8-based license plate detection module also improves the YOLOv8 model. The specific improved YOLOv8 model is: loss function improvement, adding feature enhancement module, adding small target detection head and improving NMS algorithm;

[0012] The loss function is improved to use Repulsion Loss instead of the original IoU loss function;

[0013] The added feature enhancement module is used to combine the original C2f module with the SA attention mechanism to design a new C2f_SA module;

[0014] The addition of a small target detection head is used to increase the detection capability of the license plate for small targets;

[0015] The improved NMS algorithm is used to improve the original non-maximum suppression algorithm NMS to GIoU_NMS, so as to avoid the phenomenon that the score in the detection result suddenly jumps.

[0016] Furthermore, the loss function of the Repulsion improved by the loss function is:

[0017] L Rep =L Attr +α*L RepGT +β*L RepBox

[0018] Among them, L Rep is the loss function of the entire Repulsion, L Attr To attract the loss function, the predicted box is required to be close to the specified target, L RepGT and L RepBox It is a rejection term, which requires the surrounding real target objects to be far away from the prediction box. The coefficients α and β are the weights for balancing the auxiliary loss.

[0019] Furthermore, the feature enhancement module adds the designed new C2f_SA module to the original YOLOv8 model backbone.

[0020] Furthermore, the new C2f_SA module designs a new Bottlenec_SA module internally to replace the original Bottleneck module, in which the SA attention module first groups the channel sizes into multiple sub-features and processes these sub-features in parallel, and then uses a shuffle unit to describe the correlation of spatial and channel temperature features on each sub-feature. Then each sub-feature is aggregated and the "channel shuffle" operator is used to realize the information channels of all different sub-features.

[0021] Furthermore, the small target detection head is added. In the original YOLOv8 model, three detection heads are included, namely 80*80, 40*40, and 20*20 detection heads, and a new 160*160 detection head is added, which is located in front of the 80*80 detection head.

[0022] Furthermore, the improved NMS algorithm (24) adopts the GIoU algorithm, and the calculation formula is:

[0023]

[0024] Where W S is the area of ​​the minimum enclosed region of the two boxes, |W S -U| is the area in the closure region that does not belong to the two boxes.

[0025] Furthermore, the data processing module based on U2-Net and distortion correction includes the following steps: binarization, grayscale conversion, Canny edge detection and one-dimensional feature vector output;

[0026] Binarization processing: The license plate detection result output by the license plate detection module based on YOLOv8 is binarized through the U2-Net model to obtain the binarized license plate;

[0027] Grayscale conversion: The license plate obtained after binarization is converted to grayscale, and Gaussian blur is performed on the grayscale conversion image. Since Gaussian blur is a filter, it can calculate the transformation of each pixel in the image through normal distribution. In N-dimensional space, the formula of normal distribution is:

[0028]

[0029] Where r is the blur radius and σ is the standard deviation of the normal distribution;

[0030] Canny edge detection, using the Canny edge detection algorithm for edge detection, the main steps include: using Gaussian filtering for noise reduction; using Sobel or other operators to calculate the gradient value and gradient direction; for the enlarged edge, it is necessary to filter out the points that are not edges to make the edge width as close to 1 pixel as possible; using a dual threshold algorithm to detect simultaneously connected edges to improve accuracy;

[0031] The sorted() method is used to quickly sort the contour area and extract its four vertices. Then, the transformation matrix is ​​obtained by perspective transformation when the vertices are known. The specific calculation process is as follows:

[0032]

[0033] And the transformed graphics can be obtained;

[0034] One-dimensional feature vector output, the transformed image is compressed into one-dimensional data, and the two-dimensional image data is compressed into one-dimensional vector data. Then, principal component analysis is used to reduce redundant data, and tensor product is used to perform high-bit mapping encoding and output.

[0035] Furthermore, the specific execution process of the license plate recognition module based on hyperdimensional computing is as follows:

[0036] Hyperdimensional computing uses hyperdimensional vectors as the basic operation object, where similarity is a very important indicator to measure the relationship between different hyperdimensional vectors. It is also the basis for decision-making and coding of hyperdimensional computing models. The Hamming distance is used to measure the similarity of hyperdimensional vectors. The calculation formula is:

[0037]

[0038] Where i represents the i-th element in the hyperdimensional vector, d is the dimension of the space, and S and W are two different hyperdimensional vectors.

[0039] Beneficial effects:

[0040] The invention discloses a license plate recognition system based on improved YOLOv8, which relates to small target detection and recognition, and includes a license plate detection module based on improved YOLOv8, a data processing module based on U2-Net and distortion correction, and a license plate recognition module based on hyperdimensional computing, wherein the license plate detection module is used to detect license plate photos using the improved YOLOv8 model; further data processing is performed on the detected license plate part picture, and finally the license plate data after data processing is recognized using the license plate recognition module based on hyperdimensional computing, and the module's ability to attract targets and repel the environment is enhanced by using a loss function based on Repulsion, and a feature enhancement module is designed to enhance the feature extraction capability of images, and the recognition capability of small targets such as license plates is improved by adding a small target detection head, and finally the non-maximum suppression algorithm is improved. The invention effectively improves the license plate detection performance by improving the YOLOv8 algorithm, and greatly improves the license plate detection speed by using the data processing module and the license plate recognition module. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of the license plate recognition method based on improved YOLOv8 of the present invention;

[0042] Figure 2 It is a newly added feature enhancement module of the present invention;

[0043] Figure 3 This is a flowchart of the Shuffle Attention mechanism used in the present invention.

[0044] Figure 4 The license plate detection data results obtained by the improved algorithm on this data set experiment.

[0045] Figure 5 This is the detection result of the YOLOv8 algorithm on the data set.

[0046] Figure 6 This is the detection result of the YOLOv5 algorithm on the data set. DETAILED DESCRIPTION

[0047] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0048] The present invention provides a license plate detection and recognition system based on hyperdimensional computing, such as Figure 1 , including the production of license plate data set 1, a license plate detection module based on improved YOLOv8 2, a data processing module based on U2-Net and distortion correction 3, and a license plate recognition module based on hyperdimensional computing 4.

[0049] Create a license plate data set 1, collect videos and images and combine them with the CCPD data set, and create a data set according to the YOLO format of YOLOv8. In the embodiment of the present invention, the license plate photo data set uses the Chinese city license plate open source data set CCPD released by USTC. The original data set contains a large amount of data, and most of the Chinese characters in the license plate are Anhui. The generalization performance for Chinese characters in other provinces is not strong, so only part of the data is used this time, including CCPD-Rotate and CCPD-Tilt, and the collected videos and images are added to the CCPD data set to expand its generalization ability for Chinese characters in different provinces.

[0050] Based on the improved YOLOv8 license plate detection module 2, the improved YOLOv8 model is used to detect the photo containing the license plate to obtain the area containing the license plate.

[0051] The specific improvements in the improved YOLOv8 network are: loss function improvement 21, adding a feature enhancement module 22, adding a small target detection head 23 and improving the NMS algorithm 24.

[0052] Among them, loss function improvement 21 is used to use Repulsion Loss instead of the original IoU loss function to enhance the algorithm's ability to reject targets other than license plates and its ability to attract license plates.

[0053] Repulsion Loss is used instead of the original IoU loss function to enhance the algorithm's ability to exclude targets other than license plates. The loss function of Repulsion is:

[0054] L Rep =L Attr +α*L RepGT +β*L RepBox

[0055] Where L Rep is the loss function of the entire Repulsion, L Attr To attract the loss function, the predicted box is required to be close to the specified target, L RepGT and L RepBox It is a rejection term, which requires the surrounding real target objects to be far away from the prediction box. The coefficients α and β are the weights for balancing the auxiliary loss.

[0056] The feature enhancement module 22 is added to combine the original C2f module with the SA (Shuffle Attention) attention mechanism to design a new C2f_SA module to enhance the algorithm's ability to extract license plate features. Figure 3Combined with the above, a new C2f_SA(221) module is designed, such as Figure 2 , and added to the original YOLOv8 model backbone.

[0057] The new C2f_SA(221) module has a new Bottlenec_SA module designed inside it to replace the original Bottleneck module. The SA attention module first groups the channel sizes into multiple sub-features and processes these sub-features in parallel. It then uses a shuffle unit to describe the correlation of spatial and channel temperature features on each sub-feature. Each sub-feature is then aggregated and the “channel shuffle” operator is used to implement the information channels of all different sub-features.

[0058] The small target detection head 23 is added to increase the detection capability of the license plate for small targets.

[0059] Add a small target detection head 23. The original network contains three detection heads, namely 80*80 (small target), 40*40 (medium target), and 20*20 (large target) detection heads. In order to enhance the detection ability of this algorithm for small targets, a 160*160 detection head is added this time, which is located in front of the 80*80 detection head.

[0060] Improved NMS algorithm 24 is used to improve the original non-maximum suppression algorithm NMS to GIoU_NMS to avoid the phenomenon that the score in the detection result will suddenly jump. In the original NMS algorithm, its IoU indicator is used to suppress redundant detection frames, but it only considers the overlapping part, and often misjudges the non-overlapping part, resulting in incorrect suppression. This time, the GIoU algorithm is used, which is an improvement of the IoU algorithm. Its main function is to not only focus on the overlapping part of the target frame and the detection frame, but also focus on other non-overlapping parts, which can better reflect the overlap between the two. Its calculation formula is:

[0061]

[0062] Where W S is the area of ​​the minimum enclosed region of the two boxes, |W S -U| is the area in the closure region that does not belong to the two boxes.

[0063] The data processing module 3 based on U2-Net and distortion correction uses U2-net and distortion correction to process the detected image and convert it into feature information of a one-dimensional structure, which includes the following steps: binarization processing 31, grayscale conversion 32, Canny edge detection 33 and one-dimensional feature vector output 34.

[0064] The binarization process 31 is to binarize the license plate detection result output by the license plate detection module through the U2-net model to obtain a binarized license plate.

[0065] Grayscale conversion 32 is to convert the license plate obtained after binarization into grayscale, and perform Gaussian blur processing on the grayscale conversion image. Since Gaussian blur is a filter, it can calculate the transformation of each pixel in the image through normal distribution. In N-dimensional space, the formula of normal distribution is:

[0066]

[0067] Where r is the blur radius and σ is the standard deviation of the normal distribution.

[0068] Canny edge detection 33 uses the Canny edge detection algorithm to perform edge detection. Its main steps include: using Gaussian filtering for noise reduction, which makes the image smooth (blurred), which may also increase the edge width; using Sobel or other operators to calculate the gradient value and gradient direction; for the enlarged edge, it is necessary to filter the points that are not edges to make the edge width as close to 1 pixel as possible; using a dual threshold algorithm to detect simultaneously connected edges to improve accuracy.

[0069] The sorted() method is used to quickly sort the contour area and extract its four vertices. Then, through perspective transformation, the transformation matrix can be obtained when the vertices are known. The specific calculation process is:

[0070]

[0071] And the transformed graphics can be obtained;

[0072] The function of the one-dimensional feature vector output 34 is to perform one-dimensional data compression on the transformed image, compressing it from two-dimensional image data to one-dimensional vector data, and then using principal component analysis to reduce redundant data, and then using tensor product to perform high-bit mapping encoding and output.

[0073] The license plate recognition module 4 based on hyperdimensional computing uses hyperdimensional computing to recognize the characteristic information of the one-dimensional structure output by the data processing module to obtain the license plate information. By using hyperdimensional computing, the high-order mapping code output by the U2-Net and distortion correction data processing module 3 is calculated and recognized.

[0074] Since hyperdimensional computing uses hyperdimensional vectors as its basic operation objects, the similarity is a very important indicator to measure the relationship between different hyperdimensional vectors. It is also the basis for the decision and coding of the hyperdimensional computing model. The Hamming distance is very suitable for encoding binary data such as binary and bipolar in the similarity measurement of hyperdimensional vectors. Its calculation formula is:

[0075]

[0076] Where i represents the i-th element in the hyperdimensional vector, d is the dimension of the space, and S and W are two different hyperdimensional vectors.

[0077] in Figure 4 The license plate detection data results obtained by the improved experiment of the present invention are shown in the first row from left to right, respectively, the anchor box loss function of each round of training, the loss function of classification, the loss when processing data imbalance (Distribution Focal Loss), precision and recall rate; the second row from left to right, respectively, the anchor box loss function of each round of verification, the loss function of classification, the loss when processing data imbalance (Distribution Focal Loss), the average precision with an IOU threshold of 0.5, and the average precision with an IOU threshold of 0.5 to 0.95. After the improvement, the system can achieve 98.7% precision and 98.3% average precision with an IOU threshold of 0.5 when detecting the CCPD data set.

[0078] and Figure 5 The detection result of YOLOv8 algorithm on the data set is only 90%, and when the IOU threshold is 0.5, its average accuracy is only 89.5%. Experimental verification shows that this algorithm can effectively improve the detection performance of YOLOv8 for the license plate data set.

[0079] Figure 6 The detection result of YOLOv5 on this dataset is only 80%, and when the IOU threshold is 0.5, its average accuracy is only 76%.

[0080] The above experimental results show that compared with conventional detection algorithms on the market, this improved algorithm has a significant improvement in detection.

[0081] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A license plate detection and recognition system based on hyperdimensional computing, It is characterized in that It includes a license plate data set preparation module (1), a license plate detection module based on YOLOv8 (2), a data processing module based on U2-Net and distortion correction (3), and a license plate recognition module based on hyperdimensional computing (4); The license plate dataset making module (1) comprises collecting videos and images and combining them with the CCPD dataset, and making them into a dataset according to the YOLO format of YOLOv8; The YOLOv8-based license plate detection module (2) is used to detect a photo containing a license plate using a YOLOv8 model to obtain an area containing the license plate; the YOLOv8-based license plate detection module (2) improves the YOLOv8 model, and the specific improved YOLOv8 model includes: loss function improvement (21), adding a feature enhancement module (22), adding a small target detection head (23) and improving the NMS algorithm (24); The loss function improvement (21) is used to use Repulsion Loss to replace the original IoU loss function; the loss function of Repulsion in the loss function improvement (21) is: L Rep =L Attr +a*L RepGT +β*L RepBox Among them, L Rep is the loss function of the entire Repulsion, L Attr To attract the loss function, the predicted box is required to be close to the specified target, L RepGT and L RepBox It is a repulsion term, requiring the surrounding real target objects to be far away from the prediction box. The coefficients α and β are the weights for balancing the auxiliary loss. The feature enhancement module (22) is used to combine the original C2f module with the Shuffle Attention mechanism to design a new C2f_SA module; the feature enhancement module (22) adds the designed new C2f_SA module to the original YOLOv8 model backbone; the new C2f_SA module internally designs a new Bottlenec_SA module to replace the original Bottleneck module, wherein the SA attention module first groups the channel size into multiple sub-features and processes these sub-features in parallel, and then uses a shuffle unit to describe the correlation of the features in space and channel temperature on each sub-feature, and then each sub-feature is aggregated, and a "channel shuffle" operator is used to realize the information channel of all different sub-features; The added small target detection head (23) is used to increase the detection capability of the license plate for small targets. The original YOLOv8 model includes three detection heads, which are 80*80, 40*40, and 20*20 detection heads, and a new 160*160 detection head is added, which is located in front of the 80*80 detection head; The improved NMS algorithm (24) is used to improve the original non-maximum suppression algorithm NMS into GIoU_NMS to avoid the phenomenon that the score in the detection result suddenly jumps; the GIoU algorithm is used, and the calculation formula is: Where W S is the area of ​​the minimum enclosed region of the two boxes, |W S -U| is the area in the closure region that does not belong to the two boxes; The data processing module (3) based on U2-Net and distortion correction is used to process the detected image using U2-net and distortion correction, and convert it into feature information of a one-dimensional structure; The license plate recognition module (4) based on hyperdimensional computing is used to recognize the characteristic information of the one-dimensional structure output by the data processing module by using hyperdimensional computing to obtain the license plate information; The specific execution process of the license plate recognition module (4) based on hyperdimensional computing is as follows: Hyperdimensional computing uses hyperdimensional vectors as the basic operation object, where similarity is a very important indicator to measure the relationship between different hyperdimensional vectors. It is also the basis for decision-making and coding of hyperdimensional computing models. The Hamming distance is used to measure the similarity of hyperdimensional vectors. The calculation formula is: Where i represents the i-th element in the hyperdimensional vector, d is the dimension of the space, and S and W are two different hyperdimensional vectors.

2. A license plate detection and recognition system based on hyperdimensional computing according to claim 1, It is characterized in that The license plate photo dataset uses the Chinese city license plate open source dataset CCPD, and only uses part of the data, including CCPD-Rotate and CCPD-Tilt, and adds the collected videos and images to the CCPD dataset to expand its generalization ability for Chinese characters in different provinces.

3. A license plate detection and recognition system based on hyperdimensional computing according to claim 1, It is characterized in that The data processing module (3) based on U2-Net and distortion correction includes the following steps: binarization processing, grayscale conversion, Canny edge detection and one-dimensional feature vector output; Binarization processing: the license plate detection result output by the license plate detection module based on YOLOv8 is binarized through the U2-Net model to obtain the binarized license plate; Grayscale conversion: The license plate obtained after binarization is converted to grayscale, and Gaussian blur is performed on the grayscale conversion image. Since Gaussian blur is a filter, it can calculate the transformation of each pixel in the image through normal distribution. In N-dimensional space, the formula of normal distribution is: Where r is the blur radius and σ is the standard deviation of the normal distribution; Canny edge detection, using the Canny edge detection algorithm for edge detection, the main steps include: using Gaussian filtering for noise reduction; using Sobel or other operators to calculate the gradient value and gradient direction; for the enlarged edge, it is necessary to filter out the points that are not edges to make the edge width as close to 1 pixel as possible; using a dual threshold algorithm to detect simultaneously connected edges to improve accuracy; The sorted() method is used to quickly sort the contour area and extract its four vertices. Then, the transformation matrix is ​​obtained by perspective transformation when the vertices are known. The specific calculation process is as follows: And the transformed graphics can be obtained; One-dimensional feature vector output, the transformed image is compressed into one-dimensional data, and the two-dimensional image data is compressed into one-dimensional vector data. Then, principal component analysis is used to reduce redundant data, and tensor product is used to perform high-bit mapping encoding and output.

Citation Information

Patent Citations

  • Vehicle color recognition method and apparatus and electronic device

    CN107480676A

  • License plate detection and recognition system and method in unlimited security scene

    CN112966631A

  • License plate recognition system based on improved yov5 algorithm

    CN116597432A

  • License plate detection and identification method based on Yolov5s network and CRNN

    CN116704487A

  • Object detection in image based on stochastic optimization

    US20210097277A1