License plate detection and identification method and system based on YOLOv8

Abstract

The invention relates to the technical field of image processing, and discloses a license plate detection and recognition method and system based on YOLOv8, and the system comprises an image detection module and a license plate recognition module. The method comprises the following steps: step 1, a high-definition camera shoots vehicle images, and images meeting a screening standard are reserved to form an image data set; step 2, marking a central point ZD of each image, and establishing a two-dimensional coordinate system after the central points ZD of all the images are overlapped and calibrated to be original points; 3, dividing the image in the two-dimensional coordinate system into grids, and calculating the total number B of bounding boxes of all the grids; 4, calculating a confidence score ZX of each bounding box by using a YOLOv8 algorithm, filtering the bounding boxes by using an NMS, and keeping a high-score bounding box for continuous recognition, so that the robustness of an algorithm optimization model is more stable; and step 5, the RNN algorithm model calculates and generates a target character group Mzf, license plate characters are output in combination with a preamble and a subsequent sequence, and the license plate character recognition precision is high.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a license plate detection and recognition method and system based on YOLOv8. Background Art

[0002] License plate detection and recognition is an indispensable part of the intelligent traffic management system. By accurately identifying vehicle license plate information, the intelligent traffic management system provides efficient and automated solutions for traffic management, security monitoring, toll collection systems, parking lot management and other fields. At busy traffic intersections, the license plate recognition system can automatically record passing vehicles, provide data support for traffic flow statistics, illegal vehicle tracking and real-time traffic control, and greatly improve the efficiency and response speed of traffic management. Through license plate recognition technology, traffic violations such as speeding and driving against traffic can be monitored and recorded in real time, which helps to reduce traffic violations and improve road driving safety. At highway toll stations and parking lots, license plate detection and recognition can realize unattended automatic deduction of fees, reduce human resource consumption, and improve vehicle traffic efficiency. In the field of public security monitoring, license plate detection and recognition technology can help the police quickly lock suspected vehicles and provide key clues for case detection. The application of license plate recognition technology in commercial services, such as smart parking lots, self-service gas payment, etc., provides users with a convenient and fast service experience. The YOLOv8 algorithm is one of the important tools in the current target detection field. It can complete the task of object detection through a single forward propagation, which makes it perform well in scenarios that require real-time processing. In the specific implementation of license plate detection and recognition, the YOLOv8 algorithm was first used to detect the location of the license plate in the image. Since the first launch of this series of algorithms, YOLO has won widespread attention and application in the field of computer vision with its fast and accurate detection capabilities.

[0003] At present, traditional license plate detection and recognition methods mostly rely on a single YOLOv8 algorithm model. During the target recognition process, recognition accuracy and computational efficiency cannot be guaranteed at the same time. If higher recognition accuracy is required, the computational efficiency is low and time-consuming. If higher computational efficiency is required in a short time, the recognition accuracy cannot be guaranteed. The recognized license plate characters are prone to disorder, indistinguishable letters and numbers, and erroneous text. In addition, the image content is uncertain in a dynamic environment, and the single YOLOv8 algorithm model is not robust. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the shortcomings of the prior art, the present invention provides a license plate detection and recognition method and system based on YOLOv8, which has the advantages of more stable algorithm optimization model robustness and high license plate character recognition accuracy. It solves the problem that the traditional license plate detection and recognition method relies on a single algorithm model with poor robustness and cannot guarantee both recognition accuracy and computational efficiency.

[0006] (II) Technical solution

[0007] To achieve the above object, the present invention provides the following technical solution: a license plate detection and recognition method based on YOLOv8, comprising the following steps:

[0008] Step 1: The image detection module uses a high-definition camera to capture the front and back images of the vehicle, and then uses the YOLOv8 algorithm to retain the images that meet the screening criteria to form an image dataset;

[0009] Step 2: The image detection module marks the center point ZD of each image in the image data set through the YOLOv8 algorithm, and establishes a two-dimensional coordinate system after calibrating the center points ZD of all images in the image data set to coincide with the origin;

[0010] Step 3: The image detection module divides the image in the two-dimensional coordinate system into S×S grids through the YOLOv8 algorithm, and each grid is predicted to have c bounding boxes, and the total number of bounding boxes B of all grids is calculated;

[0011] Step 4: The license plate recognition module is set with a confidence threshold ZXY. The license plate recognition module calculates the confidence score ZX of each bounding box through the YOLOv8 algorithm, and uses NMS to filter out high-scoring bounding boxes and low-scoring bounding boxes, retaining the high-scoring bounding boxes for continued recognition;

[0012] Step 5: The license plate recognition module is equipped with an RNN algorithm model. The RNN algorithm model calculates and generates the target character group Mzf based on the high-score bounding box, and outputs the license plate characters in the order of the context.

[0013] Preferably, in step 1, the screening criteria require that the size of each image meets 1280×720 pixels, the resolution reaches above 300 dpi, and the content of each image includes the vehicle license plate.

[0014] Preferably, in step 1, the expression of the image data set is {T1, T2, T3, ..., T n}, T1 to T n Corresponding to each image data that meets the screening criteria, the digital numbers 1 to n indicate that there are n images that meet the screening criteria.

[0015] Preferably, in step 2, the image center point ZD is calculated using the following formula:

[0016]

[0017] In the formula, ZD represents the center point of the image, T i K represents the width of the i-th image in the image dataset, T i The height of the i-th image in the G image dataset, It means calculating the center points of n images in the image data set at the same time, and then aligning the center points ZD of multiple images to the same origin, which is the origin of the two-dimensional coordinate system.

[0018] Preferably, in step 3, the calculation formula for the total number of bounding boxes B is as follows:

[0019] B=S×c

[0020] In the formula, S represents the total number of bounding boxes of all grids, S represents the number of grids into which the image is divided, and c represents the hyperparameter of the number of bounding boxes predicted by the YOLOv8 algorithm for each grid.

[0021] Preferably, in step 4, the confidence score ZX is calculated as follows:

[0022]

[0023] In the formula, ZX represents the confidence score of a single bounding box, M represents the recognition target, which includes quadrilateral license plates and characters, and B k represents the kth bounding box among all bounding boxes, Indicates the probability that the kth bounding box contains the recognized target, IOU practical Represents the overlap area ratio between the predicted bounding box and the true target area.

[0024] Preferably, in step 4, if the confidence score ZX of a single bounding box is lower than the confidence threshold ZXY, that is, it is a low-score bounding box, the license plate recognition module stops recognizing the low-score bounding box.

[0025] Preferably, in step 4, if the confidence score ZX of a single bounding box is higher than the confidence threshold ZXY or is included in the range of the confidence threshold ZXY, it is a high-score bounding box, indicating that the bounding box contains multiple recognition targets, and the license plate recognition module continues to recognize the high-score bounding box.

[0026] Preferably, in step 5, the target character group Mzf is calculated as follows:

[0027]

[0028] In the formula, Mzf represents the target character group, T i represents the i-th image in the image dataset, represents the bounding box contained in the grid of the i-th image, It means that the RNN algorithm model recognizes the characters contained in the grid bounding boxes of n images at the same time.

[0029] A license plate detection and recognition system based on YOLOv8, including an image detection module and a license plate recognition module;

[0030] The image detection module collects an image data set by connecting a high-definition camera through a network. The image data set includes all image data with a size of 1280×720 pixels, a resolution of 300 dpi or more, and image content including vehicle license plates. The image detection module marks the center point ZD of each image in the image data set through the YOLOv8 algorithm, and establishes a two-dimensional coordinate system after calibrating the center points ZD of all images in the image data set to coincide with the origin. The image detection module divides the image in the two-dimensional coordinate system into S×S grids through the YOLOv8 algorithm, and each grid is predicted to have c bounding boxes, and the total number of bounding boxes B of all grids is calculated. The image detection module transmits the image data set, the two-dimensional coordinate system and the total number of bounding boxes B to the license plate recognition module through the network;

[0031] The license plate recognition module is provided with a confidence threshold ZXY. The license plate recognition module calculates the confidence score ZX of each bounding box through the YOLOv8 algorithm, and uses NMS to filter out high-score bounding boxes and low-score bounding boxes, retaining the high-score bounding boxes for continued recognition. The license plate recognition module is provided with an RNN algorithm model. The RNN algorithm model calculates and generates a target character group Mzf based on the high-score bounding box, and outputs the license plate characters in combination with the context sequence.

[0032] Compared with the prior art, the present invention provides a license plate detection and recognition method and system based on YOLOv8, which has the following beneficial effects:

[0033] 1. The present invention uses a high-definition camera to shoot the front image and the back image of the vehicle. The image detection module uses the YOLOv8 algorithm to retain the images that meet the screening criteria and then form an image data set. The image detection module uses the YOLOv8 algorithm to mark the center point ZD of each image in the image data set, and calibrates the center points ZD of all images in the image data set to the origin after coinciding and establishing a two-dimensional coordinate system. The center points ZD of multiple images are coinciding and calibrated to be the only origin, which can flexibly deal with original images of any size. The algorithm can accurately focus on the center point ZD of each image, and the batch processing recognition efficiency is higher. The image detection module uses the YOLOv8 algorithm to calibrate the two-dimensional coordinate system The image in is divided into S×S grids, and each grid is predicted to have c bounding boxes, and the total number of bounding boxes B of all grids is calculated. It can efficiently locate and classify multiple objects in the image. The image detection module transmits the image dataset, two-dimensional coordinate system and the total number of bounding boxes B to the license plate recognition module through the network. The license plate recognition module is set with a confidence threshold ZXY. The license plate recognition module calculates the confidence score ZX of each bounding box through the YOLOv8 algorithm, and uses NMS to filter out high-scoring bounding boxes and low-scoring bounding boxes, retains high-scoring bounding boxes for continued recognition, reduces duplicate bounding boxes to improve the detection accuracy, and the algorithm optimizes the model to make it more robust and stable.

[0034] 2. The present invention is provided with an RNN algorithm model through the license plate recognition module. The RNN algorithm model can process text sequences of different lengths. By learning the features in a large amount of data and focusing on the sequence of identifying characters, it helps to improve the accuracy of recognition and can flexibly cope with the challenges of complex image data. The RNN algorithm model calculates and generates the target character group Mzf based on the high-score bounding box, outputs the license plate characters in combination with the sequence of the context, and converts them into text data that can be edited by a computer, so as to avoid the situation where the recognized license plate characters are out of order, letters and numbers cannot be distinguished, or wrong characters are avoided, and the license plate character recognition accuracy is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a step diagram of the method of the present invention;

[0036] Figure 2 It is a schematic diagram of the system flow of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] Since traditional license plate detection and recognition methods mostly rely on a single YOLOv8 algorithm model, in the target recognition process, recognition accuracy and computational efficiency cannot be guaranteed at the same time. If higher recognition accuracy is required, the computational efficiency is low and time-consuming. If higher computational efficiency is required in a short time, the recognition accuracy cannot be guaranteed. The recognized license plate characters are prone to disorder, letters and numbers cannot be distinguished, and there are problems with incorrect text. In addition, the image content in a dynamic environment is uncertain, and the single YOLOv8 algorithm model is not robust. Therefore, a license plate detection and recognition method and system based on YOLOv8 is proposed. Please refer to Figure 1-Figure 2 , the method comprises the following steps:

[0039] Step 1: The image detection module uses a high-definition camera to capture the front and back images of the vehicle, and then uses the YOLOv8 algorithm to retain the images that meet the screening criteria to form an image dataset. The screening criteria require that the size of each image must meet 1280×720 pixels, the resolution must reach 300 dpi or more, and each image must contain the vehicle license plate. The larger the image size, the richer the details, which helps to improve the recognition accuracy. For example, the YOLOv8 algorithm can accurately crop the area containing the license plate in the original image through open source OCR libraries such as PaddleOCR. The expression of the image dataset is {T1, T2, T3, ..., T n}, T1 to T n Corresponding to each image data that meets the screening criteria, the digital numbers 1 to n indicate that there are n images that meet the screening criteria. Data preparation and preprocessing are key steps to ensure the effectiveness of the algorithm. High-quality image data sets are the basis for improving the model's recognition capabilities.

[0040] Step 2: The image detection module marks the center point ZD of each image in the image data set through the YOLOv8 algorithm, and establishes a two-dimensional coordinate system after calibrating the center points ZD of all images in the image data set to coincide with the origin;

[0041] The calculation formula of center point ZD is as follows:

[0042]

[0043] In the formula, ZD represents the center point of the image, T i K represents the width of the i-th image in the image dataset, T i The height of the i-th image in the G image dataset, It means calculating the center points of n images in the image data set at the same time, and then aligning the center points ZD of multiple images to the same origin, which is the origin of the two-dimensional coordinate system. The center points ZD of multiple images are aligned and calibrated to be the only origin. It can flexibly deal with original images of any size. The algorithm can accurately focus on the center point ZD of each image, and batch processing recognition efficiency is higher.

[0044] Step 3: The image detection module divides the image in the two-dimensional coordinate system into S×S grids through the YOLOv8 algorithm, and each grid predicts c bounding boxes, and calculates the total number of bounding boxes B of all grids. The calculation formula is as follows:

[0045] B=S×c

[0046] In the formula, B represents the total number of bounding boxes of all grids, S represents the number of grids into which the image is divided, and selecting an appropriate S value is an important factor in balancing detection performance and running speed, which can be adjusted according to actual conditions. c represents the hyperparameter of the number of bounding boxes predicted by the YOLOv8 algorithm for each grid, which can be adjusted according to actual conditions by manual adjustment, grid search, random search, Bayesian optimization, etc. The YOLOv8 algorithm can efficiently locate and classify multiple objects in an image through grid division and bounding box prediction. Each grid is predicted independently, so that the YOLOv8 algorithm can achieve a fast running speed while maintaining high detection accuracy, which is suitable for real-time target detection tasks;

[0047] Step 4. The license plate recognition module is set with a confidence threshold ZXY. The license plate recognition module calculates the confidence score ZX of each bounding box through the YOLOv8 algorithm, and uses NMS to filter out high-score bounding boxes and low-score bounding boxes, reducing duplicate bounding boxes to improve the accuracy of detection. If the confidence score ZX of a single bounding box is lower than the confidence threshold ZXY, it is a low-score bounding box. The license plate recognition module stops recognizing the low-score bounding box and filters out redundant bounding boxes. If the confidence score ZX of a single bounding box is higher than the confidence threshold ZXY or is included in the confidence threshold ZXY range, it is a high-score bounding box, indicating that the bounding box contains multiple recognition targets. The bounding box that is most likely to accurately locate the target object is retained, and the license plate recognition module continues to recognize the high-score bounding box. The algorithm optimizes the model to be more robust and stable.

[0048] The confidence score ZX is calculated as follows:

[0049]

[0050] In the formula, ZX represents the confidence score of a single bounding box, M represents the recognition target, which includes quadrilateral license plates and characters, and characters include numbers, letters, and Chinese characters. k represents the kth bounding box among all bounding boxes, represents the probability that the kth bounding box contains the recognized target, HOU practical It represents the overlapping area ratio between the predicted bounding box and the real target area. The overlapping area ratio needs to be dynamically calculated according to the actual situation. The confidence score ZX directly reflects the probability of the object existing in the predicted bounding box and the accuracy of the bounding box prediction.

[0051] Step 5. The license plate recognition module is equipped with an RNN algorithm model. The RNN algorithm model can process text sequences of different lengths. By learning the features in a large amount of data and focusing on the order of identifying characters, it helps to improve the accuracy of recognition and can flexibly cope with the challenges of complex image data. The RNN algorithm model calculates and generates the target character group Mzf based on the high-score bounding box, and outputs the license plate characters in combination with the order of the context. The calculation formula is as follows:

[0052]

[0053] In the formula, Mzf represents the target character group, T i represents the i-th image in the image dataset, represents the bounding box contained in the grid of the i-th image, It means that the RNN algorithm model recognizes the characters contained in the grid bounding boxes of n images at the same time. The Chinese vehicle license plate usually consists of seven characters, including a Chinese character, an English letter and a combination of five numbers or letters. Although the number of license plate characters varies depending on the type of vehicle, the combination structure usually remains the same. The RNN algorithm model can convert the characters in the detected license plate image into computer-editable text data, avoiding the situation where the recognized license plate characters are out of order, letters and numbers cannot be distinguished, or wrong characters are found, and the license plate character recognition accuracy is high.

[0054] The above-mentioned license plate detection and recognition method based on YOLOv8 is implemented by a license plate detection and recognition system based on YOLOv8, which includes an image detection module and a license plate recognition module;

[0055] The image detection module collects image data sets by connecting to a high-definition camera through a network. The image data sets include all image data with a size of 1280×720 pixels, a resolution of more than 300 dpi per inch, and image content containing vehicle license plates. The image detection module marks the center point ZD of each image in the image data set through the YOLOv8 algorithm, and establishes a two-dimensional coordinate system after calibrating the center points ZD of all images in the image data set as the origin. The center points ZD of multiple images are calibrated as the only origin, which can flexibly deal with original images of any size. The algorithm can accurately focus on the center point ZD of each image, and batch processing recognition efficiency is higher. The image detection module divides the image in the two-dimensional coordinate system into S×S grids through the YOLOv8 algorithm, and each grid predicts c bounding boxes, and calculates the total number of bounding boxes B of all grids, which can efficiently locate and classify multiple objects in the image. The image detection module transmits the image data set, the two-dimensional coordinate system and the total number of bounding boxes B to the license plate recognition module through the network;

[0056] The license plate recognition module is set with a confidence threshold ZXY. The license plate recognition module calculates the confidence score ZX of each bounding box through the YOLOv8 algorithm, and uses NMS to filter out high-score bounding boxes and low-score bounding boxes, retains the high-score bounding boxes for continued recognition, reduces duplicate bounding boxes to improve the detection accuracy, and the algorithm optimization model is more robust and stable. The license plate recognition module is set with an RNN algorithm model. The RNN algorithm model calculates and generates the target character group Mzf based on the high-score bounding box, outputs the license plate characters in combination with the context sequence, and converts them into computer-editable text data to avoid the situation where the recognized license plate characters are out of order, letters and numbers cannot be distinguished, or wrong text has high license plate character recognition accuracy.

[0057] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A license plate detection and recognition method based on YOLOv8, characterized by: The following steps are involved: Step 1: The image detection module uses a high-definition camera to capture the front and back images of the vehicle, and then uses the YOLOv8 algorithm to retain the images that meet the screening criteria to form an image dataset; Step 2: The image detection module marks the center point ZD of each image in the image data set through the YOLOv8 algorithm, and establishes a two-dimensional coordinate system after calibrating the center points ZD of all images in the image data set to coincide with the origin; Step 3: The image detection module divides the image in the two-dimensional coordinate system into S×S grids through the YOLOv8 algorithm, and each grid is predicted to have c bounding boxes, and the total number of bounding boxes B of all grids is calculated; Step 4: The license plate recognition module is set with a confidence threshold ZXY. The license plate recognition module calculates the confidence score ZX of each bounding box through the YOLOv8 algorithm, and uses NMS to filter out high-scoring bounding boxes and low-scoring bounding boxes, retaining the high-scoring bounding boxes for continued recognition; Step 5: The license plate recognition module is equipped with an RNN algorithm model. The RNN algorithm model calculates and generates the target character group Mzf based on the high-score bounding box, and outputs the license plate characters in the order of the context.

2. A license plate detection and recognition method based on YOLOv8 according to claim 1, characterized in that: In the step 1, the screening criteria require that the size of each image meets 1280×720 pixels, the resolution reaches above 300 dpi, and the content of each image contains the vehicle license plate.

3. A license plate detection and recognition method based on YOLOv8 according to claim 2, characterized in that: In step 1, the expression of the image data set is {T1, T2, T3, ..., T n }, T1 to T n Corresponding to each image data that meets the screening criteria, the digital numbers 1 to n indicate that there are n images that meet the screening criteria.

4. A license plate detection and recognition method based on YOLOv8 according to claim 3, characterized in that: In the step 2, the image center point ZD is calculated as follows: In the formula, ZD represents the center point of the image, T i K represents the width of the i-th image in the image dataset, T i The height of the i-th image in the G image dataset, It means calculating the center points of n images in the image data set at the same time, and then aligning the center points ZD of multiple images to the same origin, which is the origin of the two-dimensional coordinate system.

5. A license plate detection and recognition method based on YOLOv8 according to claim 4, characterized in that: In step 3, the calculation formula for the total number of bounding boxes B is as follows: B=S×c In the formula, B represents the total number of bounding boxes of all grids, S represents the number of grids into which the image is divided, and c represents the hyperparameter of the number of bounding boxes predicted by the YOLOv8 algorithm for each grid.

6. A license plate detection and recognition method based on YOLOv8 according to claim 5, characterized in that: In step 4, the confidence score ZX is calculated as follows: In the formula, ZX represents the confidence score of a single bounding box, M represents the recognition target, which includes quadrilateral license plates and characters, and B k represents the kth bounding box among all bounding boxes, Indicates the probability that the kth bounding box contains the recognized target, IOU practical Represents the overlap area ratio between the predicted bounding box and the true target area.

7. A license plate detection and recognition method based on YOLOv8 according to claim 6, characterized in that: In the step 4, if the confidence score ZX of a single bounding box is lower than the confidence threshold ZXY, it is a low-score bounding box, and the license plate recognition module stops recognizing the low-score bounding box.

8. A license plate detection and recognition method based on YOLOv8 according to claim 7, characterized in that: In step 4, if the confidence score ZX of a single bounding box is higher than the confidence threshold ZXY or is included in the confidence threshold ZXY range, it is a high-score bounding box, indicating that the bounding box contains multiple recognition targets, and the license plate recognition module continues to recognize the high-score bounding box.

9. A license plate detection and recognition method based on YOLOv8 according to claim 8, characterized in that: In step 5, the target character group Mzf is calculated as follows: In the formula, Mzf represents the target character group, T i represents the i-th image in the image dataset, represents the bounding box contained in the grid of the i-th image, It means that the RNN algorithm model recognizes the characters contained in the grid bounding boxes of n images at the same time.

10. A license plate detection and recognition system based on YOLOv8, applied to the license plate detection and recognition method based on YOLOv8 described in claims 1-9, characterized in that: Including image detection module and license plate recognition module; The image detection module collects an image data set by connecting a high-definition camera through a network. The image data set includes all image data with a size of 1280×720 pixels, a resolution of 300 dpi or more, and image content including vehicle license plates. The image detection module marks the center point ZD of each image in the image data set through the YOLOv8 algorithm, and establishes a two-dimensional coordinate system after calibrating the center points ZD of all images in the image data set to coincide with the origin. The image detection module divides the image in the two-dimensional coordinate system into S×S grids through the YOLOv8 algorithm, and each grid is predicted to have c bounding boxes, and the total number of bounding boxes B of all grids is calculated. The image detection module transmits the image data set, the two-dimensional coordinate system and the total number of bounding boxes B to the license plate recognition module through the network; The license plate recognition module is provided with a confidence threshold ZXY. The license plate recognition module calculates the confidence score ZX of each bounding box through the YOLOv8 algorithm, and uses NMS to filter out high-score bounding boxes and low-score bounding boxes, retaining the high-score bounding boxes for continued recognition. The license plate recognition module is provided with an RNN algorithm model. The RNN algorithm model calculates and generates a target character group Mzf based on the high-score bounding box, and outputs the license plate characters in combination with the context sequence.

Images