A license plate recognition method in haze environment

By improving the dark channel prior algorithm and convolutional network joint defog model, the license plates in haze environment are defog and feature extraction, character segmentation is performed in combination with supersampling and connectivity domain methods, and the artificial neural network is used for recognition, which solves the problem of difficulty in identifying license plates in haze environments and achieves high-accurate license plate recognition.

CN114581914BActive Publication Date: 2025-04-29CHINA INFOMRAITON CONSULTING & DESIGNING INST CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210100244.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-27
Publication Date
2025-04-29
Estimated Expiration
2042-01-27

AI Technical Summary

Technical Problem

In a haze environment, existing license plate recognition technology is difficult to effectively identify license plates, mainly due to the haze causing image blurring and difficulty in feature extraction.

Method used

The improved dark channel prior algorithm and the joint defog and area extraction model with convolutional network are used to defog the blur image, the license plate bounding box is extracted, and character segmentation is performed through supersampling and connecting domain methods, and character recognition is finally performed using artificial neural network.

Benefits of technology

In a smog environment, the accuracy of license plate recognition is significantly improved, the haze interference is reduced, and the license plates that are defogging images can be accurately extracted and the license plate characters can be successfully identified.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114581914B_ABST
    Figure CN114581914B_ABST
Patent Text Reader

Abstract

The present invention provides a license plate recognition method in a haze environment, the purpose of which is to reduce the influence of haze on the boundaries and features of license plates in an image in a haze environment, so as to correctly identify license plates. First, a dark channel prior algorithm based on local estimation of atmospheric light values ​​is used to preliminarily defog the blurred image. Then, the image is further defogged, and on the basis of a target detection convolutional neural network, the license plate area is detected by a joint defogging and region extraction model JFRM, which greatly reduces the distortion of image restoration and accurately extracts the defogged image of the license plate. Finally, image super-resolution is achieved through convolution-enhanced SRCNN, so that license plate characters can be successfully recognized. The present invention improves the accuracy of license plate recognition in a haze environment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a license plate recognition method, in particular to a license plate recognition method in a haze environment. Background Art

[0002] With the rapid development of economy and the continuous expansion of cities, the transportation system has become an indispensable part of modern society. The license plate is the number and information registration of each vehicle. Its main function is to know the region where the vehicle belongs through the license plate, and to find the owner of the vehicle and the registration information of the vehicle according to the license plate.

[0003] License plate recognition technology can automatically identify license plates and distinguish vehicles. License plate recognition technology is widely used in applications such as vehicle tracking, traffic congestion monitoring, urban street management and electronic toll collection on toll roads.

[0004] However, as haze weather becomes more and more serious, the haze seriously blurs the boundaries and features of license plates, making them difficult to be found and recognized. Most of the existing license plate recognition technologies are based on feature extraction technology, and feature extraction is very difficult due to the fuzziness of the image caused by haze interference. Summary of the invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide a license plate recognition method in a haze environment in view of the shortcomings of the prior art.

[0006] In order to solve the above technical problems, the present invention discloses a license plate recognition method in a haze environment, comprising:

[0007] Step 1, improve the dark channel prior algorithm, perform preliminary defogging on the blurred image, and obtain a preliminary defogging image;

[0008] Step 2: The preliminary dehazed image is further dehazed by a joint dehazing and region extraction model with a convolutional network to obtain a license plate dehazed image;

[0009] Step 3, performing region extraction through a joint defogging and region extraction model with a convolutional network, extracting the license plate bounding box from the license plate defogging image, and correcting the position of the bounding box to obtain a corrected image;

[0010] Step 4, supersampling the corrected image to obtain a high-resolution image;

[0011] Step 5, using the connected domain method and template matching method, perform character segmentation on the high-resolution image, and use artificial neural network for character recognition, and finally complete the license plate recognition in the haze environment.

[0012] Step 1 in the present invention comprises:

[0013] Step 1-1, region division, divide each image into several regions with different fog concentrations;

[0014] Step 1-2, regional processing, processes regions with different fog concentrations separately.

[0015] The area division in step 1-1 of the present invention includes:

[0016] Perform grayscale processing on the image, and the grayscale value represents the brightness of each pixel;

[0017] Calculate the dark channel value of each pixel, take a rectangular window of a set size with each pixel as the center, take the minimum gray value in the rectangular window instead of the gray value of the central pixel, and get the dark channel value of the pixel;

[0018] The k-means clustering algorithm is used to divide the pixels into different fog density areas.

[0019] The regional processing described in step 1-2 of the present invention includes: after regional division, selecting 0.1% of pixels with the largest grayscale value of the dark channel image in each region, taking the maximum brightness as the regional atmospheric light component A of the region; calculating the transmittance t, the method is as follows:

[0020]

[0021] Where ω is the correction factor, I is the image to be defogged, the superscript c represents one of the three RGB channels, A is the regional atmospheric light component, Ω(x) represents a window centered on pixel x, and y is a pixel in the window;

[0022] Then the reconstructed image is obtained according to the following method:

[0023] I(x)=J(x)·t(x)+A·[1-t(x)]

[0024] Among them, I(x) is the existing foggy image, J(x) is the desired fog-free image, A is the global atmospheric light component, and t(x) is the transmittance.

[0025] The joint defogging and region extraction model JFRM with convolutional network described in step 2 of the present invention is composed of three modules: a multi-scale densely connected defogging optimization network; a feature extraction backbone network; a feature interaction detection network;

[0026] Among them, the preliminary dehazed image is further dehazed through a multi-scale densely connected dehazing optimization network; in the multi-scale densely connected dehazing optimization network, filters are added to form multi-scale features, and the convolution layer and the connection layer are combined to compensate for the missing information in the convolution process; residual blocks are added to deepen the perception of the haze structure in the image and accurately capture the boundary details of objects at different depths; thus, a license plate dehazed image is obtained.

[0027] Step 3 of the present invention includes performing region extraction by a joint defogging and region extraction model JFRM with a convolutional network to obtain a license plate bounding box, and the method includes:

[0028] In the feature extraction backbone network, the Darknet-53 network in YOLOv3 is used as the network structure for feature extraction; the network combines convolutional layers and residual blocks; the residual blocks produce residual results without changing the size of the input image and the output image; the residual blocks control the propagation of gradients to avoid the gradient diffusion problem;

[0029] In the feature interaction detection network, the yolo layer in YOLOv3 is used to interact with the features extracted by multiple downsampling of Darknet-53; the last three feature outputs of five downsamplings are mapped to three yolo layers of different scales; the feature map size of the smallest scale is 13*13, the feature map size of the middle scale is 26*26, and the feature map size of the largest scale is 52*52; three yolo layers of different scales apply a 1*1 convolution kernel to produce an output.

[0030] Step 3 of the present invention includes: using a non-maximum suppression method to eliminate overlapping bounding boxes, and calculating the ratio of the intersection and union of the predicted border and the real border by the intersection-union ratio. The specific method is as follows:

[0031] Step 3-1, traverse the license plate defogging images and the bounding boxes extracted from each image. For each image, calculate the intersection-over-union ratio IoU between any two bounding boxes bi and bj, compare the confidence scores Score, and execute steps 3-2 and 3-3;

[0032] Step 3-2, initialize the flag to 0;

[0033] Step 3-3, when the intersection-over-union ratio IoU(bi,bj) is greater than or equal to the threshold value and the confidence score Score(bi) of the bounding box bi is greater than or equal to the confidence score Score(bj) of the bounding box bj, delete the bounding box bj and its confidence score; otherwise, delete the bounding box bi and its confidence score; set the flag flag to 1;

[0034] Step 3-5, output the optimal bounding box and its confidence score.

[0035] In step 3 of the present invention: for tilted license plates, WPOD-NET is used to correct the position of the bounding box; first, a fixed-size rectangular corner box is set around the center of a pixel. If the object probability of this pixel is greater than a preset detection threshold, an affine matrix is ​​constructed using regression parameters to convert the rectangular box into a tilted license plate area; if it is less than, no processing is performed.

[0036] In step 4 of the present invention, a convolution-enhanced super-resolution neural network SRCNN is used to obtain a high-resolution image. First, the image is adjusted to a target size through bicubic interpolation, and then a nonlinear mapping is completed through a convolution layer. The convolution-enhanced SRCNN model structure has 6 layers, and the kernel size of the first layer of convolution is increased. From the 3rd to the 5th layers, convolution kernels of different sizes are continuously executed to further extract features, and the 6th layer reconstructs the mapping features through a 5*5*1 convolution kernel and generates a high-resolution image.

[0037] In step 5 of the present invention: Gaussian blur, grayscale processing and Otsu binarization are performed to remove noise artifacts; a possible character list and an alphanumeric connected domain are obtained using a connected domain method; these possible characters are matched with license plate characters to remove overlapping characters within the outline; the characters are segmented into a series of single-character blocks, and an artificial neural network is used to recognize license plate characters, and finally license plate recognition in a haze environment is completed.

[0038] Beneficial effects: In order to reduce the interference of haze, a license plate recognition method in haze environment is proposed, and the JFRM model is proposed, which greatly reduces the image reconstruction distortion caused by the cumulative error and accurately extracts the license plate of the dehazed image. The image super-resolution is achieved through the convolution-enhanced SRCNN, so that the license plate characters can be successfully recognized. The simulation results show that our method can improve the accuracy of license plate recognition in haze environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0040] Figure 1 The overall flow chart of this method is shown in FIG.

[0041] Figure 2 The effect of preliminary dehazing by improving the dark channel prior algorithm.

[0042] Figure 3 This is the effect diagram of processing a tilted license plate.

[0043] Figure 4 This is the effect of supersampling the image.

[0044] Figure 5 The figure is an effect diagram of license plate recognition using the method of the present invention. DETAILED DESCRIPTION

[0045] The invention provides a license plate recognition method in a haze environment, which aims to reduce the influence of haze on the boundary and features of the license plate in the haze environment and correctly recognize the characters of the license plate.

[0046] A license plate recognition method in a haze environment, such as Figure 1 As shown, the following steps are included:

[0047] Step 1: Improve the dark channel prior algorithm to perform preliminary dehazing on the blurred image.

[0048] Step 2: Further dehaze the image through joint dehazing to obtain a license plate dehazed image.

[0049] Step 3: Extract the bounding box using the region extraction model and correct the position of the bounding box.

[0050] Step 4: Supersample the image to obtain a high-resolution image.

[0051] Step 5: Use the connected domain method and template matching method to achieve character segmentation, and then use artificial neural network to achieve character recognition.

[0052] In the above-mentioned license plate recognition technology in a haze environment, in the above-mentioned step 1, each image is divided into several areas with different fog densities, and then the areas with different fog densities are processed respectively.

[0053] In the regional division, the following steps are included:

[0054] First, we convert the image into grayscale values, which represent the brightness of each pixel.

[0055] Then, we calculate the dark channel value of each pixel, take a rectangular window of a certain size with each pixel as the center, and take the minimum gray value in the rectangular window to replace the gray value of the central pixel, so as to obtain the dark channel value of the pixel.

[0056] Finally, the k-means clustering algorithm is used to automatically divide the pixels into different fog concentration areas.

[0057] In regional processing, after dividing the fog concentration area, in each area, we select the 0.1% pixels with the largest grayscale value of the dark channel image and take the maximum brightness as the regional atmospheric light component A of the area. Calculate the transmittance t, and then according to Get the reconstructed image.

[0058] The license plate recognition technology in a haze environment, in the step 2, the present invention proposes a joint defogging and region extraction model (JFRM) with a convolutional network, which consists of three modules: a multi-scale densely connected defogging optimization network; a feature extraction backbone network; and a feature interaction detection network.

[0059] In the multi-scale densely connected dehazing optimization network, we added some filters to form multi-scale features, and combined convolutional layers and connection layers to compensate for the missing information in the convolution process. We added some residual blocks to deepen the perception of the haze structure in the image, so as to accurately capture the boundary details of objects at different depths.

[0060] In the feature extraction backbone network, the Darknet-53 network in YOLOv3 (reference: J. Redmon and A. Farhadi, YOLOv3: An Incremental Improvement) is used as the main network structure for feature extraction. The network combines convolutional layers and residual blocks. The residual block will produce residual results without changing the size of the input image and the output image. The residual block can control the propagation of the gradient and avoid gradient dispersion and some problems with the gradient.

[0061] In feature interaction detection, we use the yolo layer in YOLOv3 to interact with the features extracted by multiple downsampling of Darknet-53. The last three feature outputs of five downsamplings are mapped to three yolo layers of different scales. The feature map size of the smallest scale is 13*13, the feature map size of the middle scale is 26*26, and the feature map size of the largest scale is 52*52. In this way, the three yolo layers of different scales can apply a 1*1 convolution kernel to produce an output similar to that of a fully connected layer.

[0062] In the aforementioned license plate recognition technology in a haze environment, in the aforementioned step 3, a non-maximum suppression algorithm is used to eliminate overlapping bounding boxes. The full name of IoU is Intersection over Union, which calculates the ratio of the intersection and union of the "predicted bounding box" and the "real bounding box". The algorithm is described as follows:

[0063] Step 3.1: Traverse the dehazed images and the bounding boxes extracted from each image. For each image, calculate the IoU value between any two bounding boxes (bi, bj) and compare the confidence scores Score.

[0064] Step 3.2: Initialize the flag to 0.

[0065] Step 3.3: When IoU(bi,bj)≥threshold and Score(bi)≥Score(bj), delete bj and its confidence score; otherwise, bi and its confidence score will be deleted. The flag is set to 1. Then, in the other two bounding boxes, calculate the IoU value and compare the confidence.

[0066] Step 3.4: Output the optimal bounding box and its confidence score

[0067] For tilted license plates, the bounding box is not accurate. Therefore, WPOD-NET is introduced to correct the position of the bounding box (reference: SM Silva and CR Jung, License Plate Detection and Recognition in Unconstrained Scenarios), such as Figure 5 As shown in Figure 2, WPOD-NET introduces the characteristics of the spatial transformer network, which can be used to detect non-rectangular regions.

[0068] For tilted license plates, we first consider a fixed-size rectangular corner box around the center of a pixel. If the object probability of this pixel is greater than a preset detection threshold, we use some regression parameters to construct an affine matrix to transform the rectangular box into the tilted license plate area.

[0069] In the aforementioned license plate recognition technology in a haze environment, in the aforementioned step 4, a convolution-enhanced SRCNN is used to obtain a high-resolution image. For low-resolution images, SRCNN first adjusts the image to the target size through bicubic interpolation, and then completes the nonlinear mapping through the convolution layer. The structure of the convolution-enhanced SRCNN has 6 layers, in which the kernel size of the first layer of convolution is increased. In the 3rd to 5th layers, convolution kernels of different sizes are continuously executed to further extract features, and the 6th layer reconstructs the mapping features through a 5*5*1 convolution kernel and generates a high-resolution image. SRCNN can refer to Chao Dong, ChenChange Loy's "Image Super-Resolution Using Deep Convolutional Networks"

[0070] In the above-mentioned license plate recognition technology in a haze environment, in the above-mentioned step 5, in the character recognition step, we focus on the area containing the letters and numbers we want to recognize and discard other areas. First, Gaussian blur, grayscale processing and Otsu binarization are performed to remove possible noise artifacts.

[0071] Secondly, the connected domain method is used to obtain a list of possible characters and the connected domain of alphanumeric characters. These possible characters are then matched with the license plate characters, removing the overlapping characters within the contour. Finally, the characters are segmented into a series of single-character blocks, and the license plate characters are recognized by artificial neural networks.

[0072] The present invention has the following advantages and beneficial effects: In order to reduce the interference of haze, a fog concentration defogging method is first studied, and the JFRM model is proposed, which greatly reduces the image reconstruction distortion caused by the cumulative error and accurately extracts the license plate of the defogging image. The image super-resolution is achieved through the convolution-enhanced SRCNN, so that the license plate characters can be successfully recognized. The simulation results show that our method can improve the accuracy of license plate recognition in haze environment.

[0073] Example

[0074] In order to make the contents of the present invention more clearly understood, the embodiments of the present invention are described by means of specific examples.

[0075] Real vehicle images are used as a dataset. These images include various natural scenes (such as daytime and nighttime), totaling 3,000 images. By adding noise, rotating and other image processing, the number of images can be increased to 12,000. The xml file of license plate features is obtained by manual annotation. In addition, 300 images with motion blur are used as a dataset for convolution-enhanced SRCNN.

[0076] Using the improved dark channel prior algorithm to perform preliminary dehazing on the blurred image can produce a better restored image.

[0077] In the regional division, the following steps are included:

[0078] First, we convert the image into grayscale values, which represent the brightness of each pixel.

[0079] Then, we calculate the dark channel value of each pixel, take a rectangular window of a certain size with each pixel as the center, and take the minimum grayscale value in the rectangular window instead of the grayscale value of the central pixel to obtain the dark channel value of the pixel.

[0080] Finally, the k-means clustering algorithm is used to automatically divide the pixels into different fog concentration areas.

[0081] In regional processing, after dividing the fog concentration area, in each area, we select the 0.1% pixels with the largest grayscale value of the dark channel image and take the maximum brightness as the regional atmospheric light component A of the area. By calculating the transmittance t, we can reconstruct the image based on it, as shown in Figure 2 As shown, the left side is the image before dehazing, and the right side is the image after dehazing.

[0082] The image is further dehazed by joint dehazing to obtain a license plate dehazed image.

[0083] In the multi-scale densely connected dehazing optimization network, we added some filters to form multi-scale features, and combined convolutional layers and connection layers to compensate for the missing information in the convolution process. We added some residual blocks to deepen the perception of the haze structure in the image, so as to accurately capture the boundary details of objects at different depths.

[0084] In the feature extraction backbone network, the Darknet-53 network in YOLOv3 is used as the main network structure for feature extraction. The network combines convolutional layers and residual blocks. The residual block will produce residual results without changing the size of the input image and the output image. The residual block can control the propagation of the gradient and avoid gradient dispersion and some gradient problems.

[0085] In feature interaction detection, we use the yolo layer to interact with the features extracted by multiple downsampling of Darknet-53. The last three feature outputs of five downsamplings are mapped to three yolo layers of different scales. The feature map size of the smallest scale is 13*13, the feature map size of the middle scale is 26*26, and the feature map size of the largest scale is 52*52. In this way, the three yolo layers of different scales can apply a 1*1 convolution kernel to produce an output similar to that of a fully connected layer.

[0086] The bounding box is extracted by the region extraction model and the position of the bounding box is corrected.

[0087] For tilted license plates, such as Figure 3 As shown, we first consider a fixed-size rectangular corner box around the center of a pixel. If the object probability of this pixel is greater than a preset detection threshold, some regression parameters are used to construct an affine matrix to transform the rectangular box into a tilted license plate area.

[0088] Supersampling the image, such as Figure 4 As shown in the figure. For low-resolution images, SRCNN first adjusts the image to the target size through bicubic interpolation, and then completes the nonlinear mapping through the convolution layer. The structure of the convolution-enhanced SRCNN has 6 layers, in which the kernel size of the first layer of convolution is increased and set to 11. In the 3rd to 5th layers, convolution kernels of different sizes are continuously executed to further extract features. The 6th layer reconstructs the mapping features through a 5*5*1 convolution kernel and generates a high-resolution image.

[0089] The connected component method and template matching method are used to achieve character segmentation, and then artificial neural network is used to achieve character recognition.

[0090] First, Gaussian blurring, grayscale processing, and Otsu binarization are performed to remove possible noise artifacts.

[0091] Secondly, the connected domain method is used to obtain a list of possible characters and a connected domain of letters and numbers. These possible characters are then matched with the license plate characters, and the overlapping characters within the contour are removed. Finally, the characters are segmented into a series of single-character blocks, and the license plate characters are recognized by artificial neural networks. The recognition effect is shown in Figure 2. Figure 5 shown.

[0092] The present invention provides a method and idea for license plate recognition in a haze environment. There are many methods and approaches to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A license plate recognition method in a haze environment, characterized in that: The following steps are involved: Step 1, improve the dark channel prior algorithm, perform preliminary defogging on the blurred image, and obtain a preliminary defogging image; Step 2: The preliminary dehazed image is further dehazed by a joint dehazing and region extraction model with a convolutional network to obtain a license plate dehazed image; Step 3, performing region extraction through a joint defogging and region extraction model with a convolutional network, extracting the license plate bounding box from the license plate defogging image, and correcting the position of the bounding box to obtain a corrected image; Step 4, supersampling the corrected image to obtain a high-resolution image; Step 5: Use the connected domain method and template matching method to perform character segmentation on the high-resolution image, and use artificial neural network for character recognition, and finally complete the license plate recognition in the haze environment; The joint defogging and region extraction model JFRM with convolutional network described in step 2 consists of three modules: a multi-scale densely connected defogging optimization network; a feature extraction backbone network; and a feature interaction detection network. Among them, the preliminary dehazed image is further dehazed through a multi-scale densely connected dehazing optimization network; in the multi-scale densely connected dehazing optimization network, filters are added to form multi-scale features, and the convolution layer and the connection layer are combined to compensate for the missing information in the convolution process; residual blocks are added to deepen the perception of the haze structure in the image and accurately capture the boundary details of objects at different depths; thus, a license plate dehazed image is obtained; Step 3 includes performing region extraction using a joint defogging and region extraction model JFRM with a convolutional network to obtain a license plate bounding box, the method including: In the feature extraction backbone network, the Darknet-53 network in YOLOv3 is used as the network structure for feature extraction; the network combines convolutional layers and residual blocks; the residual blocks produce residual results without changing the size of the input image and the output image; the residual blocks control the propagation of gradients to avoid the gradient diffusion problem; In the feature interaction detection network, the yolo layer in YOLOv3 is used to interact with the features extracted by multiple downsampling of Darknet-53; the last three feature outputs of five downsamplings are mapped to three yolo layers of different scales; the feature map size of the smallest scale is 13*13, the feature map size of the middle scale is 26*26, and the feature map size of the largest scale is 52*52; three yolo layers of different scales apply a 1*1 convolution kernel to produce an output.

2. The method for license plate recognition in a haze environment according to claim 1, characterized in that: Step 1 includes: Step 1-1, region division, divide each image into several regions with different fog concentrations; Step 1-2, regional processing, processes regions with different fog concentrations separately.

3. The method for license plate recognition in a haze environment according to claim 2, characterized in that: The area division in step 1-1 includes: Perform grayscale processing on the image, and the grayscale value represents the brightness of each pixel; Calculate the dark channel value of each pixel, take a rectangular window of a set size with each pixel as the center, take the minimum gray value in the rectangular window instead of the gray value of the central pixel, and get the dark channel value of the pixel; The k-means clustering algorithm is used to divide the pixels into different fog density areas.

4. The method for license plate recognition in a haze environment according to claim 3, characterized in that: The regional processing described in step 1-2 includes: after regional division, selecting the pixels with the largest 0.1% grayscale value of the dark channel image in each region, taking the maximum brightness as the regional atmospheric light component A of the region; calculating the transmittance t, the method is as follows: ; Where ω is the correction factor, is the image to be dehazed, the superscript c represents one of the three RGB channels, A is the regional atmospheric light component, Ω(x) represents a window centered on pixel x, and y is a pixel in the window; Then the reconstructed image is obtained according to the following method: ; Among them, I(x) is the existing foggy image, J(x) is the desired fog-free image, A is the global atmospheric light component, and t(x) is the transmittance.

5. The method for license plate recognition in a haze environment according to claim 4, characterized in that: Step 3 includes: using a non-maximum suppression method to eliminate overlapping bounding boxes, and calculating the ratio of the intersection and union of the predicted bounding box and the true bounding box by the intersection-union ratio, the specific method is as follows: Step 3-1, traverse the license plate defogging images and the bounding boxes extracted from each image. For each image, calculate the intersection-over-union ratio IoU between any two bounding boxes bi and bj, compare the confidence scores Score, and execute steps 3-2 and 3-3; Step 3-2, initialize the flag to 0; Step 3-3, when the intersection-over-union ratio IoU(bi,bj) is greater than or equal to the threshold value and the confidence score Score(bi) of the bounding box bi is greater than or equal to the confidence score Score(bj) of the bounding box bj, delete the bounding box bj and its confidence score; otherwise, delete the bounding box bi and its confidence score; set the flag flag to 1; Step 3-5, output the optimal bounding box and its confidence score.

6. The method for license plate recognition in a haze environment according to claim 5, characterized in that: In step 3: For tilted license plates, WPOD-NET is used to correct the position of the bounding box; first, a fixed-size rectangular corner box is set around the center of a pixel. If the object probability of this pixel is greater than the preset detection threshold, the regression parameters are used to construct an affine matrix to convert the rectangular box into the tilted license plate area; if it is less than, no processing is performed.

7. The method for license plate recognition in a haze environment according to claim 6, characterized in that: In step 4, a convolution-enhanced super-resolution neural network SRCNN is used to obtain a high-resolution image. First, the image is adjusted to the target size through bicubic interpolation, and then the nonlinear mapping is completed through the convolution layer. The convolution-enhanced SRCNN model structure has 6 layers, and the kernel size of the first layer of convolution is increased; in the 3rd to 5th layers, convolution kernels of different sizes are continuously executed to further extract features, and the 6th layer reconstructs the mapping features through a 5*5*1 convolution kernel and generates a high-resolution image.

8. The method for license plate recognition in a haze environment according to claim 7, characterized in that: In step 5: Gaussian blur, grayscale processing and Otsu binarization are performed to remove noise artifacts; The connected domain method is used to obtain a list of possible characters and a connected domain of alphanumeric characters; these possible characters are matched with the license plate characters, and overlapping characters within the outline are removed; the characters are segmented into a series of single-character blocks, and an artificial neural network is used to recognize the license plate characters, ultimately completing license plate recognition in a haze environment.

Citation Information

Patent Citations

  • A Multi-Scale Feature Fusion Network based on GANs for Haze Removal

    AU2020100274A4

  • Atmospheric scattering model and deep learning-based license plate recognizing system in dense fog weather

    CN108734189A