Bridge license plate fast recognition method in complex environment based on domain conversion

Abstract

The application discloses a bridge license plate fast recognition method under complex environment based on domain conversion, which comprises the following steps: establishing an initial license plate database; balancing the number of various characters in the initial license plate database based on a cGAN domain conversion method; expanding the license plate pictures of scarce backgrounds in the initial license plate database based on a cycleGAN domain conversion method; adopting a single-stage target recognition network YOLOv5s as a license plate recognition network, and training the license plate recognition network by using the adjusted license plate database; and embedding the trained license plate recognition network into a bridge side camera for license plate recognition of vehicles passing by on the bridge. The application generates license plate pictures of specified license plate characters through the domain conversion method, solves the problem of recognition failure caused by too few numbers of some license plate characters, generates license plate pictures of specified backgrounds through the domain conversion method, solves the problem of difficult collection of bridge license plate pictures under complex environment, and accelerates the efficiency of bridge license plate recognition by adopting the single-stage target recognition network as the license plate recognition network.

Description

Technical Field

[0001] This invention relates to license plate recognition technology, and in particular to a method for rapid license plate recognition on bridges in complex environments based on domain transformation. Background Technology

[0002] License plate recognition has always been a hot research topic in computer vision, pattern recognition, and visual analytics. Traditional license plate recognition methods involve three steps: license plate detection, character segmentation, and character recognition. License plate detection extracts the license plate region from an image based on predetermined conditions. Common methods are based on edge information, full-image information, text features, color features, character features, and core patterns. Character segmentation extracts each character from the license plate region. Common methods are based on pixel connectivity, projected contours, prior knowledge, and character texture. Character recognition classifies multiple characters. Typical methods are based on artificial neural networks, support vector machines, k-nearest neighbor classifiers, AdaBoost classifiers, and Bayesian classifiers. In recent years, end-to-end license plate recognition methods based on text compilation have been proposed. These methods only include license plate detection and text recognition, eliminating the need for character segmentation in traditional methods. The text recognition method primarily relies on recurrent neural networks (such as LSTM and GRU).

[0003] To enhance the target recognition capabilities of target recognition networks, a set of license plate images with sufficient quantity and balanced labels across various backgrounds is needed. However, the outdoor environment is constantly changing, resulting in significant differences in the backgrounds of license plate images (e.g., daytime and nighttime), making it extremely difficult to collect a set of images with sufficient quantity and balanced labels across various backgrounds. Therefore, a method for generating target image sets is urgently needed. Summary of the Invention

[0004] Purpose of the invention: To address the above problems, the purpose of this invention is to provide a method for rapid license plate recognition on bridges in complex environments based on domain transformation.

[0005] Technical solution: The present invention provides a method for fast license plate recognition on bridges in complex environments based on domain transformation, comprising:

[0006] Establish an initial license plate database;

[0007] A cGAN-based domain transformation method balances the number of characters of each type in the initial license plate database;

[0008] The domain transformation method based on cycleGAN is used to expand license plate images with scarce backgrounds in the initial license plate database;

[0009] The single-stage target recognition network YOLOv5s was used as the license plate recognition network, and the license plate recognition network was trained using an adjusted license plate database.

[0010] The trained license plate recognition network is embedded into the bridge-side camera for license plate recognition of vehicles passing on the bridge.

[0011] Furthermore, the step of establishing the initial license plate database includes:

[0012] An initial license plate database was created by collecting license plate images with various backgrounds. All license plate images in the initial database were manually labeled, and the borders and categories of each license plate character were defined. The number of different background images and the number of different character types in the initial license plate database were also counted.

[0013] Furthermore, the cGAN-based domain transformation method includes the following steps for balancing the number of various characters in the initial license plate database:

[0014] Establish a custom license plate database corresponding to the initial license plate database, use the initial license plate database as the output set and the custom license plate database as the input set, and train the domain transformation network cGAN.

[0015] Based on the statistical count of various characters in license plates, another self-compiled license plate character database is established. The trained domain transformation network cGAN is used to convert it into a database of real license plate images, which is then added to the initial license plate database.

[0016] Furthermore, the steps of expanding the license plate images with scarce backgrounds in the initial license plate database using the cycleGAN-based domain transformation method include:

[0017] The initial license plate database uses license plate images with a rare background as the output set and license plate images with a rich background as the input set to train the domain transformation network cycleGAN.

[0018] The trained training domain transformation network cycleGAN is used to transform the input set into a target background dataset, which is then added to the initial license plate database.

[0019] Beneficial effects: Compared with the prior art, the significant advantages of this invention are:

[0020] 1. This invention generates license plate images with specified license plate characters through a domain transformation method, which solves the problem of recognition failure caused by insufficient number of some license plate characters, and also avoids the calibration work of newly generated datasets;

[0021] 2. This invention generates license plate images with specified backgrounds through a domain transformation method, which solves the problem of collecting license plate images of bridges in complex environments and avoids the calibration work of newly generated datasets;

[0022] 3. This invention uses a single-stage target recognition network as the license plate recognition network, which accelerates the efficiency of license plate recognition on bridges. Attached Figure Description

[0023] Figure 1 This is a flowchart of a method for rapid license plate recognition on bridges in complex environments based on domain transformation, as illustrated in this embodiment.

[0024] Figure 2 This is a schematic diagram illustrating the generation of license plate images using domain transformation networks cGAN and cycleGAN in the embodiment.

[0025] Figure 3 The example shows the results of using a license plate recognition network to identify various characters on a daytime license plate. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments.

[0027] like Figure 1 The diagram shown is a flowchart of a method for rapid license plate recognition on bridges in complex environments based on domain transformation, as described in this embodiment. The method includes the following steps:

[0028] Step 1: Establish an initial license plate database.

[0029] Specifically, the steps for establishing the initial license plate database include:

[0030] An initial license plate database was created by collecting license plate images with various backgrounds. All license plate images in the initial database were manually labeled, and the borders and categories of each license plate character were defined. The number of different background images and the number of different character types in the initial license plate database were also counted.

[0031] Step 2: Use a cGAN-based domain transformation method to balance the number of characters of each type in the initial license plate database.

[0032] Specifically, the steps of the above-mentioned cGAN-based domain transformation method to balance the number of various characters in the initial license plate database include:

[0033] Establish a custom license plate database corresponding to the initial license plate database, use the initial license plate database as the output set and the custom license plate database as the input set, and train the domain transformation network cGAN.

[0034] Based on the statistical count of various license plate characters, another self-compiled license plate character database is established. According to the demand for each type of character, the trained domain transformation network cGAN is used to convert it into a database of real license plate images, which is then added to the initial license plate database. The labels of the calibration files are adjusted accordingly.

[0035] Step 3: Expand the license plate images with scarce backgrounds in the initial license plate database using the domain transformation method based on cycleGAN.

[0036] Specifically, the steps of the above-mentioned domain transformation method based on cycleGAN to expand license plate images with scarce backgrounds in the initial license plate database include:

[0037] The initial license plate database uses license plate images with a rare background as the output set and license plate images with a rich background as the input set to train the domain transformation network cycleGAN.

[0038] Based on the demand for various background images, the trained training domain transformation network cycleGAN is used to transform the input set into a target background dataset, which is then added to the initial license plate database.

[0039] Step 4: Use the single-stage target recognition network YOLOv5s as the license plate recognition network, and train the license plate recognition network using the adjusted license plate database.

[0040] Step 5: Embed the trained license plate recognition network into the bridge-side camera for license plate recognition of vehicles passing on the bridge.

[0041] After adjusting the initial license plate database, a license plate recognition network was trained using a license plate dataset that satisfies label balance and background richness. The trained network was then embedded into a camera on the side of the bridge for license plate recognition of vehicles passing on the bridge.

[0042] To further illustrate the effectiveness of the present invention, a typical bridge case is used to illustrate the specific implementation steps of the bridge license plate rapid recognition method in complex environments.

[0043] Step 1: Constructing the initial dataset.

[0044] 1200 images containing complete license plates were selected from the bridge-side cameras, including 1000 images with daytime backgrounds and 200 images with nighttime backgrounds. All images had a resolution of 320 pixels × 320 pixels. All license plate images were calibrated, and the number of each type of character was counted.

[0045] Step 2: Label balancing of the initial dataset.

[0046] Due to the imbalance in the number of labels in the dataset, the cGAN network was trained using 1000 daytime license plate images and 1000 corresponding self-compiled license plate images. Figure 2 As shown, the trained cGAN network is used to convert the required self-compiled license plate images into real license plate images, which are then added to the initial license plate dataset. Corresponding modifications are made to the license plate labeling files. Figure 2The middle section shows the cGAN network. The left side uses a self-made license plate image as input, and the right side shows the real license plate image output by the cGAN network.

[0047] Step 3: The dataset has a rich background.

[0048] Because there are too few license plate images with nighttime backgrounds, 1000 daytime license plate images and 200 nighttime background images were used to train the CycleGAN network. The trained CycleGAN network was then used to convert the daytime background license plate images into nighttime background images and supplement them into the initial dataset. Figure 2 As shown.

[0049] Step 4: Single-stage license plate recognition network.

[0050] The single-stage target recognition network YOLOv5s was adopted as the license plate recognition network.

[0051] Step 5: Training and Application.

[0052] The newly generated dataset was used to train a single-stage target recognition network, YOLOv5s, for 500 rounds to obtain the final fast license plate recognition network, which was then embedded into the camera on the side of the bridge.

[0053] For this typical bridge, the results of identifying each character of the daytime license plate using the method of this invention are as follows: Figure 3 As shown, a more balanced dataset yields better precision and recall. It's worth noting that the characters 'C' and 'Q' are the least numerous compared to other characters, and their recall is also the lowest. The results for recognizing license plates at night are shown in Table 1. The night images generated by CycleGAN can completely replace real night images. This method solves the problem of scarce night images, avoids the need for relabeling night images, and can be applied to images with other sparse backgrounds. This embodiment verifies the effectiveness and efficiency of the method for quickly recognizing license plates on bridges in complex environments.

[0054] Table 1 Results of nighttime license plate recognition

[0055]

Claims

1. A method for fast license plate recognition on bridges in complex environments based on domain transformation, characterized in that, include: Establish an initial license plate database; A cGAN-based domain transformation method balances the number of characters of each type in the initial license plate database; The domain transformation method based on cycleGAN is used to expand license plate images with scarce backgrounds in the initial license plate database; The single-stage target recognition network YOLOv5s was used as the license plate recognition network, and the license plate recognition network was trained using an adjusted license plate database. The trained license plate recognition network is embedded into the bridge-side camera for license plate recognition of vehicles passing on the bridge. The steps of the cGAN-based domain transformation method to balance the number of various characters in the initial license plate database include: Establish a custom license plate database corresponding to the initial license plate database, use the initial license plate database as the output set and the custom license plate database as the input set, and train the domain transformation network cGAN. Based on the statistical count of various characters in license plates, another self-compiled license plate character database is established. The trained domain transformation network cGAN is used to convert it into a database of real license plate images, which is then added to the initial license plate database.

2. The method for rapid recognition of license plates on bridges according to claim 1, characterized in that, The steps for establishing the initial license plate database include: An initial license plate database was created by collecting license plate images with various backgrounds. All license plate images in the initial database were manually labeled, and the borders and categories of each license plate character were defined. The number of different background images and the number of different character types in the initial license plate database were also counted.

3. The method for rapid recognition of license plates on bridges according to claim 1, characterized in that, The steps of the cycleGAN-based domain transformation method to expand license plate images with scarce backgrounds in the initial license plate database include: The initial license plate database uses license plate images with a rare background as the output set and license plate images with a rich background as the input set to train the domain transformation network cycleGAN. The trained training domain transformation network cycleGAN is used to transform the input set into a target background dataset, which is then added to the initial license plate database.

Images