License plate recognition enhancement method for smart city unattended charging system

Abstract

The invention relates to the technical field of image recognition, in particular to a license plate recognition enhancement method for a smart city unattended charging system, and the method comprises the steps: segmenting a license plate gray image, and obtaining a character image through a plurality of times of clustering; preliminarily comparing each character image with each template image to obtain a high-probability template image of each character image; scaling the high-probability template image by combining the angular point features of the character image and the template image; and matching all character images with the scaled high-probability template image according to angular point features to obtain a license plate recognition result. According to the method, the license plate image is segmented, and each character image without a dirty area is obtained through multiple times of clustering, so that the dirty area is prevented from influencing subsequent analysis; and scaling and adjusting the scale by combining the angular point features, so that the template characters are consistent with the character image scale as much as possible during matching, and the license plate recognition accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to a method for enhancing license plate recognition for an unattended toll collection system in a smart city. Background Art

[0002] The unattended toll collection system in a smart city can improve the efficiency of the toll collection system at toll stations to cope with the increasing vehicle flow. The unattended toll collection system can greatly improve the traffic efficiency and reduce the queuing waiting time by automatically recognizing license plates. For example, the unattended toll collection system on highways collects and photographs vehicle license plates, and matches each character in the license plate according to the template matching technology, so as to identify and extract the information of the license plate characters.

[0003] Dirt such as dust or muddy water often adheres to license plates. After the dirt covers the license plate characters, the characters on the collected license plate image will be missing or blurred. In the traditional template matching process, it is necessary to match the template characters with the collected license plate character images to obtain the result. When directly using template matching for license plate recognition of a license plate with missing or blurred characters, due to the influence of dirt, there may be an error in the scale adjustment during the proportional transformation of the template characters, resulting in inaccurate recognition, obtaining an incorrect license plate recognition result, and reducing the working efficiency of the unattended toll collection system in a smart city. Summary of the Invention

[0004] The present invention provides a method for enhancing license plate recognition for an unattended toll collection system in a smart city to solve the problem that the existing error occurs during the proportional transformation of template characters due to the influence of dirt, resulting in inaccurate recognition and reducing the working efficiency of the unattended toll collection system in a smart city.

[0005] The method for enhancing license plate recognition for an unattended toll collection system in a smart city of the present invention adopts the following technical solutions:

[0006] An embodiment of the present invention provides a method for enhancing license plate recognition for an unattended toll collection system in a smart city, and the method includes the following steps:

[0007] Obtain a grayscale image of the license plate; segment the grayscale image of the license plate to obtain a single-character image of each character; perform clustering on the grayscale values of the single-character images several times to obtain several clusters for each clustering of the single-character images; obtain the cluster with the highest probability for each clustering according to the gray stability of the pixel points in each cluster in each other clustering; obtain the character image according to the number of times the pixel points of the single-character image appear in the cluster with the highest probability.

[0008] Perform corner detection on the character image and all prior template images to obtain all the corner points on the character image and the template images; obtain the corner similarity between each character image and each template image according to the similarity of the corner point distributions between the character images and the template images; obtain all the high-probability template images of each character image.

[0009] Establish a three-dimensional coordinate system according to the relative position characteristics of each corner point and its adjacent corner points; obtain the similar corner point pairs between the character image and all the corner points of each high-probability template image of the character image according to the density of the corner points of the character image and each high-probability template image of the character image in the three-dimensional coordinate system; scale each high-probability template image according to the distances between the two pairs of similar corner point pairs with the closest coordinates in the three-dimensional coordinate system between the character image and each high-probability template image to obtain all the comparison template images of the character image; obtain the character matching degree between the character image and each comparison template image according to the difference in the number of corner points and the difference in the corner point distribution between all the character images and each comparison template image; record the template image with the largest character matching degree among the comparison template images as the character recognition result of the character image, and obtain the text information of the license plate recognition result.

[0010] Further, the specific method for obtaining the cluster with the maximum probability in each clustering is as follows:

[0011] For the target single-character image, denote the corresponding cluster of the \(i\)-th cluster in the \(n\)-th clustering result in the \(m\)-th clustering as the \(j\)-th cluster, and the corresponding cluster difference \(\alpha\) of the \(i\)-th cluster in the \(n\)-th clustering result n,i is calculated as follows:

[0012]

[0013] In the formula: \(M\) represents the number of clusterings, \(M - 1\) represents the number of clusterings except the \(n\)-th clustering, \(m\) represents the \(m\)-th clustering except the \(n\)-th clustering, \(E\) n,i represents the number of pixel points in the \(i\)-th cluster in the \(n\)-th clustering result of the target single-character image, \(x\) represents the \(x\)-th pixel point in the \(i\)-th cluster in the \(n\)-th clustering result of the target single-character image, \(P\) n,i,x represents the gray value of the \(x\)-th pixel point in the \(i\)-th cluster in the \(n\)-th clustering result of the target single-character image, represents the average gray value of all pixel points in the \(i\)-th cluster in the \(n\)-th clustering result of the target single-character image, \(E\) m,j represents the number of pixel points in the \(j\)-th cluster in the \(m\)-th clustering result of the target single-character image, \(y\) represents the \(y\)-th pixel point in the \(j\)-th cluster in the \(m\)-th clustering result of the target single-character image, \(P\) m,j,y represents the gray value of the \(y\)-th pixel point in the \(j\)-th cluster in the \(m\)-th clustering result of the target single-character image, denotes the mean of the grayscale values of all pixel points in the j-th cluster of the m-th clustering result of the target single-character image; || denotes the absolute value function;

[0014] The character probability β of the i-th cluster in the n-th clustering result of the target single-character image n,i is calculated as follows:

[0015]

[0016] In the formula: denotes the mean of the grayscale values of all pixel points in the i-th cluster in the n-th clustering result of the target single-character image, exp() denotes the exponential function with the natural constant as the base, and Sigmoid[] denotes the Sigmoid function.

[0017] Furthermore, the specific method for obtaining the corresponding cluster is:

[0018] Map the cluster centers of all clusters obtained from all clusterings of the target single-character image onto the same image. Denote the distance between the cluster centers of every two clusters as the distance between the two clusters. In the m-th clustering result, the j-th cluster that has the minimum distance from the i-th cluster in the n-th clustering result is denoted as the corresponding cluster of the i-th cluster in the n-th clustering result in the m-th clustering result.

[0019] Furthermore, obtaining the character image according to the number of times the pixel points of the single-character image appear in the cluster with the maximum probability includes:

[0020] Denote the cluster with the maximum character probability among all clusters in each clustering result of each single-character image as the cluster with the maximum probability in each clustering result. For any pixel point of the single-character image, if the pixel point is included in the cluster with the maximum probability in one clustering result, record that the pixel point appears once, obtain the number of appearances of each pixel point, and denote the pixel points whose number of appearances exceeds the preset repetition threshold as the pixel points of the actual character area.

[0021] Set the grayscale values of all pixel points of the actual character area of the single-character image to 0, and set the grayscale values of all non-actual character area pixel points to 1. The processed result image is denoted as the character image.

[0022] Furthermore, obtaining the corner similarity between each character image and each template image according to the similarity of the corner distributions between each character image and each template image includes:

[0023] The corner similarity δ between the r-th character image and the t-th template image r,t is calculated as follows:

[0024]

[0025] In the formula: Denote the mean of the character probabilities of the cluster with the highest probability among all clustering results of the r-th character image. Denote the mean of the feature included angles of all corner points of the r-th character image. Denote the mean of the feature included angles of all corner points of the t-th template image; norm[] represents the linear normalization function; exp() represents the exponential function with the natural constant as the base.

[0026] Preset a similarity threshold, and denote all template images whose corner similarity with the r-th character image is greater than the preset similarity threshold as all high-probability template images of the r-th character image.

[0027] Furthermore, the specific method for obtaining the feature included angle of the corner point is as follows:

[0028] For each corner point in the character image and the template image, denote the included angle between the connection line of the other corner point with the smallest distance to each corner point and the horizontal direction as the feature included angle of each corner point.

[0029] Furthermore, the establishment of the three-dimensional coordinate system according to the relative position features of each corner point and its adjacent corner points includes:

[0030] Respectively obtain the included angles between the connection lines of the three corner points with the smallest, second smallest, and third smallest Euclidean distances to the position of each corner point in the character image and the horizontal direction, and denote them as the first attribute, second attribute, and third attribute of each corner point in the character image; obtain the first attribute, second attribute, and third attribute of all corner points in the character image and the high-probability template image, and construct a three-dimensional coordinate system with the first attribute of all corner points as the x-axis coordinate, the second attribute as the y-axis coordinate, and the third attribute as the z-axis coordinate.

[0031] Furthermore, the obtaining of the similar corner point pairs between the character image and all high-probability template images of the character image according to the densities of all corner points of the character image and each high-probability template image of the character image in the three-dimensional coordinate system includes:

[0032] In the three-dimensional coordinate system, perform the first DBSCAN density clustering with the first clustering radius and the first number of clustering samples to obtain a dense clustering result; for all corner points except the dense clustering result, perform the second DBSCAN density clustering with the second clustering radius and the second number of clustering samples to obtain a sparse clustering result; among all clusters of the sparse clustering result, denote the clusters that meet the conditions that only two corner points are included in one cluster and the two corner points belong to the character image and the high-probability template image respectively as a similar corner point pair between the character image and the high-probability template image of the character image.

[0033] Further, scaling each high - probability template image according to the distances between the two pairs of similar corner point pairs with the closest coordinates in the three - dimensional coordinate system between the character image and each high - probability template image to obtain all comparison template images of the character image, including:

[0034] For the r - th character image and the h - th high - probability template image of the r - th character image, denote the two pairs of similar corner point pairs with the smallest Euclidean distance in the three - dimensional coordinate system between the r - th character image and the h - th high - probability template image of the r - th character image as the a - th pair of similar corner point pairs and the b - th pair of similar corner point pairs. For the A - th corner point of the r - th character image and the A′ - th corner point of the h - th high - probability template image of the r - th character image in the a - th pair of similar corner point pairs, and the B - th corner point of the r - th character image and the B′ - th corner point of the h - th high - probability template image of the r - th character image in the b - th pair of similar corner point pairs; denote the ratio of the length of the line segment connecting the A - th corner point to the B - th corner point to the length of the line segment connecting the A′ - th corner point to the B′ - th corner point as the scaling scale of the h - th high - probability template image of the r - th character image; scale the h - th high - probability template image of the r - th character image proportionally by the scaling scale to obtain the h - th comparison template image.

[0035] Further, obtaining the character matching degree between the character image and each comparison template image according to the difference in the number of corner points and the difference in the corner point distribution between all character images and each comparison template image, including:

[0036] The position deviation value μ r,h,(A,B) between the r - th character image and the h - th comparison template image is calculated as follows:

[0037]

[0038] In the formula: d r,(A,B) represents the length of the line segment connecting the A - th corner point to the B - th corner point in the r - th character image, θ r,(A,B) represents the angle between the line segment connecting the A - th corner point to the B - th corner point in the r - th character image and the horizontal direction, d r,h,(A′,B′) represents the length of the line segment connecting the A′ - th corner point to the B′ - th corner point in the h - th comparison template image, θ r,h,(A′,B′) represents the angle between the line segment connecting the A′ - th corner point to the B′ - th corner point in the h - th comparison template image and the horizontal direction;

[0039] The character matching degree σ r,h between the r - th character image and the h - th comparison template image is calculated as follows:

[0040]

[0041] In the formula: δ r,tdenotes the corner similarity between the r-th character image and the h-th comparison template image, W r denotes the number of all corners in the r-th character image, W r,h denotes the number of all corners in the h-th comparison template image of the r-th character image, U r,h denotes the number of similar corner pairs between the r-th character image and the h-th comparison template image, exp[] represents the exponential function with the natural constant as the base; || represents the absolute value function.

[0042] The beneficial effects of the present invention are as follows:

[0043] The present invention segments the license plate grayscale image and obtains character images through several clustering operations; preliminarily compares each character image with each template image to obtain the high-probability template images of each character image; combines the corner features of the character image and the template image to scale the high-probability template images; matches all character images with the scaled high-probability template images according to the corner features to obtain the license plate recognition result. The present invention segments the license plate image and obtains each character image without dirty areas through several clustering operations, avoiding the influence of dirty areas on subsequent analysis; combines corner features to adjust the scale, making the scale of the template characters as consistent as possible with that of the character images during matching, improving the license plate recognition accuracy, and thus enhancing the working efficiency of the unattended toll collection system in the smart city. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0045] Figure 1 is the step flow chart of the license plate recognition enhancement method for the unattended toll collection system in the smart city of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following, in combination with the drawings and preferred embodiments, details the specific implementation manners, structures, features and effects of the license plate recognition enhancement method for the unattended toll collection system in the smart city proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this invention belongs.

[0048] The following specifically describes the specific solution of the license plate recognition enhancement method for the unattended toll collection system in a smart city provided by the present invention in conjunction with the accompanying drawings.

[0049] Please refer to Figure 1 , which shows a flowchart of the steps of the license plate recognition enhancement method for the unattended toll collection system in a smart city provided by an embodiment of the present invention. The method includes the following steps:

[0050] Step S001, obtain a license plate grayscale image.

[0051] The purpose of this embodiment is to recognize license plate characters. Therefore, it is first necessary to collect license plate images. When a vehicle approaches the toll station, the toll collection system senses the arrival of the vehicle through sensor devices. Sensor devices for sensing the arrival of vehicles include, for example, infrared rays, radar, magnetic sensors, etc., which are not specifically limited in this embodiment. After sensing the arrival of the vehicle, the toll collection system triggers a high-resolution camera to capture the original license plate image of the vehicle. Since in the toll stations on the highways of smart cities, the high-resolution camera of the toll collection system is located on the left side of the road, the license plate image will be deformed due to perspective when shooting the license plate image of the vehicle, and it is necessary to perform correction preprocessing on the captured license plate image.

[0052] Specifically, collect the original license plate image of the vehicle through the high-resolution camera of the toll collection system, perform perspective correction on the original license plate image to obtain the corrected license plate image, and perform average grayscale processing on the corrected license plate image to obtain the license plate grayscale image.

[0053] Step S002, segment the grayscale image to obtain single-character images; cluster the grayscale values of the single-character images to obtain the cluster with the highest probability for each clustering; obtain the character images according to the number of times the pixel points of the single-character images appear in the cluster with the highest probability.

[0054] Since the license plate specifications are unified, the license plate grayscale image is segmented through the proportional positions of each character in the entire license plate to obtain single-character images of each character. Each single-character image contains only one character of the license plate, and the contained character is completely located within the single-character image. It should be noted that the proportional positions of each character in the entire license plate are well-known scenarios and will not be elaborated in detail in this embodiment.

[0055] Since the gray values of the actual character regions and the background regions are more uniform compared to those of the soiled regions, the clusters of the actual character regions and the background regions have better stability. This stability is manifested in the results of multiple clusterings as follows: for the gray values of each single-character image, performing multiple K-means clusterings. The K-means clustering results of the actual character regions and the background regions change less under different K values, while the K-means clustering results of the soiled regions change significantly and may be divided into multiple different clusters under different K values. Therefore, stable clusters can be selected based on the stability of the clusters. The stable clusters are the actual character regions or the background regions that are not affected by the soiled coverage. Furthermore, since the gray values of the actual character regions are larger than those of the background regions, the clusters of the actual character regions are distinguished from the clusters corresponding to the background regions according to the magnitude of the gray values, and then the actual character regions in each single-character image are determined.

[0056] Specifically, perform M times of K-means clustering on the gray values of each single-character image. In this embodiment, the K values are respectively set to 3, 4, 5, 6, 7, and 8, and M = 6 times of clustering are performed. Other embodiments can set other K values and the number of clusterings, and this embodiment does not make specific limitations. Obtain the clustering results after six times of K-means clustering for each single-character image; map the clustering centers of all the clusters obtained from all the clusterings to the same image, record the distance between the clustering centers of every two clusters as the distance between the two clusters, and record the cluster that satisfies the minimum distance between two clusters in the result of another clustering for a cluster in the current clustering as the corresponding cluster of the cluster in the current clustering in the result of another clustering. As an example, if the distance between the first cluster in the first clustering and the third cluster in the second clustering is the smallest, then it is said that the third cluster in the second clustering is the corresponding cluster of the first cluster in the first clustering in the result of the second clustering. Each cluster in each clustering has a corresponding cluster in the results of other clusterings.

[0057] Furthermore, for the target single-character image, denote the corresponding cluster of the i-th cluster in the n-th clustering result in the m-th clustering as the j-th cluster, and the difference α n,i between the corresponding clusters of the i-th cluster in the n-th clustering result is calculated as follows:

[0058]

[0059] In the formula: M represents the number of clusterings, M - 1 represents the number of clusterings except for the n-th clustering, m represents the m-th clustering except for the n-th clustering, E n,i represents the number of pixel points in the i-th cluster in the n-th clustering result of the target single-character image, x represents the x-th pixel point in the i-th cluster in the n-th clustering result of the target single-character image, and P n,i,x represents the gray value of the x-th pixel point in the i-th cluster in the n-th clustering result of the target single-character image. represents the mean of the grayscale values of all pixel points in the \(i\)-th cluster in the \(n\)-th clustering result of the target single-character image, \(E\) m,j represents the number of pixel points in the \(j\)-th cluster in the \(m\)-th clustering result of the target single-character image, \(y\) represents the \(y\)-th pixel point in the \(j\)-th cluster in the \(m\)-th clustering result of the target single-character image, \(P\) m,j,y represents the grayscale value of the \(y\)-th pixel point in the \(j\)-th cluster in the \(m\)-th clustering result of the target single-character image represents the mean of the grayscale values of all pixel points in the \(j\)-th cluster in the \(m\)-th clustering result of the target single-character image; \(||\) represents the absolute value function

[0060] It should be noted that represents the degree of uniformity of the grayscale values of pixel points in the \(i\)-th cluster in the \(n\)-th clustering result represents the degree of uniformity of the grayscale values of pixel points in the corresponding cluster of the \(i\)-th cluster in the \(m\)-th clustering result outside the \(n\)-th clustering represents the difference in the average uniformity of grayscale values between the \(i\)-th cluster in the \(n\)-th clustering result and its corresponding clusters in multiple clustering results. The smaller this difference is, the smaller the difference between the corresponding clusters of the \(i\)-th cluster, and the greater the stability of the \(i\)-th cluster in multiple clustering

[0061] The character probability \(\beta\) of the \(i\)-th cluster in the \(n\)-th clustering result of the target single-character image n,i is calculated as follows

[0062]

[0063] In the formula represents the mean of the grayscale values of all pixel points in the \(i\)-th cluster in the \(n\)-th clustering result of the target single-character image, \(\exp()\) represents the exponential function with the natural constant as the base, and \(\text{Sigmoid}[]\) represents the Sigmoid function

[0064] It should be noted that \(\alpha\) n,i represents the corresponding cluster difference of the \(i\)-th cluster in the \(n\)-th clustering result of the target single-character image. The smaller this value is, the higher the stability of the \(i\)-th cluster in the \(n\)-th clustering result in multiple clustering. Furthermore, the \(i\)-th cluster in the \(n\)-th clustering result is less likely to be a dirty area, but rather an actual character area or a background area The larger it is, the larger the average grayscale value of the \(i\)-th cluster in the \(n\)-th clustering result. Then, among the two cases of the actual character area and the background area, the \(i\)-th cluster in the \(n\)-th clustering result is more likely to be the actual character area The larger the result of \(\cdots\) is, it indicates that while the \(i\)-th cluster in the \(n\)-th clustering result has high stability and the average grayscale value of the pixel points within the cluster is large, the \(i\)-th cluster in the \(n\)-th clustering result is more likely to be the actual character area

[0065] Similarly, obtain the character probabilities of all clusters of all single-character images; obtain the cluster with the highest character probability among all clusters of each single-character image in each clustering result, denoted as the cluster with the highest probability in each clustering result; for any pixel point of a single-character image, if the pixel point is included in the cluster with the highest probability in a clustering result, record that the pixel point appears once, obtain the number of appearances of each pixel point, and record the pixel points with the number of appearances exceeding the preset repetition threshold as the actual character region pixel points. In this embodiment, the preset repetition threshold is described as 3, and other embodiments can be set to other values, which are not specifically limited in this embodiment; perform binarization processing on each single-character image according to all the actual character region pixel points of each single-character image, set the gray value of all the actual character region pixel points of the single-character image to 0, and set the gray value of all non-actual character region pixel points to 1. The processed result image is denoted as the character image.

[0066] Step S003: Obtain all the corner points on the character image and the template image; obtain the corner point similarity between each character image and each template image according to the similarity of the corner point distribution between each character image and each template image; obtain all the high-probability template images of each character image.

[0067] It should be noted that since all the characters that will appear on the license plate have specifications and fonts stipulated by the standard font, the template images of all the characters that will appear can be obtained according to the standard font regulations. By the difference in the number of corner points and the similarity of the corner point distribution between each character image and each template image, the template image with the highest similarity to each character image can be obtained.

[0068] Specifically, obtain the template images of all the characters on the license plate, perform corner point detection on all the character images and all the template images respectively to obtain the corner points of the character images and the template images. In this embodiment, the scale-invariant feature transform (SIFT) corner point detection algorithm is used for corner point detection. Other embodiments can select other corner point detection algorithms, which are not specifically limited in this embodiment; for each corner point in the character image and the template image, record the included angle between the connection line of the other corner point with the smallest distance to each corner point and the horizontal direction as the feature included angle of each corner point. It should be noted that the included angles in this embodiment all start from the horizontal line, rotate counterclockwise to the line where the connection segment is located until they coincide, and the angle passed during the rotation process. The value range of the angle is [0, 180°); similarly, obtain the feature included angle of each corner point in the template image.

[0069] The corner point similarity δ between the r-th character image and the t-th template image r,t is calculated as follows:

[0070]

[0071] In the formula: represents the mean of the character probabilities of the cluster with the highest probability among all clustering results of the r-th character image, represents the mean of the characteristic angles between all corner points of the r-th character image, represents the mean of the characteristic angles between all corner points of the t-th template image; norm[] represents the linear normalization function; exp() represents the exponential function with the natural constant as the base.

[0072] It should be noted that, represents the difference in the mean of the characteristic angles between all corner points of the r-th character image and the t-th template image, which reflects the similarity of the corner point distributions of the r-th character image and the t-th template image. The smaller this value is, the more similar the corner point distribution rules of the r-th character image and the t-th template image are, and the greater the corner point similarity between the r-th character image and the t-th template image. exp() is used here to map this value to the range of (0, +∞) to avoid the situation of a denominator of 0; The value of is used as the confidence parameter. The larger this value is, the greater the trustworthiness of the corner point similarity between the r-th character image and the t-th template image.

[0073] Similarly, obtain the corner point similarities between all character images and all template images; preset a similarity threshold. For the r-th character image, all template images with a corner point similarity greater than the preset similarity threshold to the r-th character image are recorded as all high-probability template images of the r-th character image. In this embodiment, the preset similarity threshold is described as 0.68, and other embodiments can be set to other values, which are not specifically limited in this embodiment; obtain the high-probability template images of all character images.

[0074] Step S004: Scale each high-probability template image to obtain all comparison template images of the character image; obtain the character matching degree between the character image and each comparison template image according to the difference in the number of corner points and the difference in the corner point distribution between all character images and each comparison template image; obtain the character recognition result, and further obtain the recognition result text information of the license plate.

[0075] It should be noted that since the distance between each vehicle and the camera is not fixed when collecting images, the scaling scale of the license plate images collected by the camera is also different, which may cause errors when comparing the template image with the character image. Therefore, first obtain multiple pairs of similar corner points between the high-probability template characters and the actual characters, regarded as a pair of corner points at the corresponding positions in the character content of the template image and the character image. Then, according to the position characteristics of the similar corner point pairs, scale the character image to the same scale as the template image, and calculate the character matching degree between the scaled character image and each high-probability template image.

[0076] Specifically, the angles between the lines connecting the three corner points with the smallest, second smallest, and third smallest Euclidean distances from the positions of each corner point in the character image to the horizontal direction are obtained respectively, and are denoted as the first attribute, second attribute, and third attribute of each corner point in the character image; the first attribute, second attribute, and third attribute of all corner points in the character image and the high-probability template image are obtained, and the first attribute of all corner points is used as the x-axis coordinate, the second attribute as the y-axis coordinate, and the third attribute as the z-axis coordinate to construct a three-dimensional coordinate system. All corner points are projected into the three-dimensional coordinate system to obtain the coordinates of each corner point in a character image. Similarly, the coordinates of each corner point in a high-probability template image of the character image are obtained.

[0077] Within the three-dimensional coordinate system, the first DBSCAN density clustering is performed with the first clustering radius and the first number of clustering samples to obtain a dense clustering result; it is preset that the first clustering radius is 10 and the first number of clustering samples is 8 for DBSCAN density clustering. Other embodiments can set other clustering radii and numbers of clustering samples, and this embodiment does not make specific limitations.

[0078] Since the dirty area will generate dense corner points, all dense clustering results are excluded. For all corner points except the dense clustering results, the second DBSCAN density clustering is performed with the second clustering radius and the second number of clustering samples to obtain a sparse clustering result; it is preset that the second clustering radius is 10 and the second number of clustering samples is 2 for DBSCAN density clustering. Other embodiments can set other clustering radii and numbers of clustering samples, and this embodiment does not make specific limitations; among all clusters in the sparse clustering result, a cluster that satisfies that only two corner points are included in a cluster and the two corner points belong to the character image and the high-probability template image respectively is denoted as a similar corner point pair between the character image and the high-probability template image of the character image, and all similar corner point pairs between the character image and the high-probability template image are obtained; similarly, all similar corner point pairs between all character images and their all high-probability template images are obtained.

[0079] Further, for the r-th character image and the h-th high-probability template image of the r-th character image, the two pairs of similar corner point pairs with the smallest Euclidean distance between the r-th character image and the h-th high-probability template image of the r-th character image in the three-dimensional coordinate system are denoted as the a-th pair of similar corner point pairs and the b-th pair of similar corner point pairs. For the A-th corner point belonging to the r-th character image in the a-th pair of similar corner point pairs, and the A'-th corner point belonging to the h-th high-probability template image of the r-th character image, as well as the B-th corner point belonging to the r-th character image in the b-th pair of similar corner point pairs, and the B'-th corner point belonging to the h-th high-probability template image of the r-th character image; the ratio of the length of the line connecting the A-th corner point to the B-th corner point to the length of the line connecting the A'-th corner point to the B'-th corner point is denoted as the scaling scale of the h-th high-probability template image of the r-th character image; the h-th high-probability template image of the r-th character image is scaled proportionally by the scaling scale to obtain the h-th comparison template image; the r-th character is compared with the h-th comparison template image again. Since it is proportional scaling, the included angle relationship remains unchanged while the line segment length changes; the position deviation value μ of the a-th pair of similar corner point pairs and the b-th pair of similar corner point pairs between the r-th character image and the h-th comparison template image is obtained. r,h,(A,B) , and the calculation formula is as follows:

[0080]

[0081] In the formula: d r,(A,B) represents the length of the line connecting the A-th corner point to the B-th corner point in the r-th character image, θ r,(A,B) represents the included angle between the line connecting the A-th corner point to the B-th corner point in the r-th character image and the horizontal direction, d r,h,(A′,B′) represents the length of the line connecting the A'-th corner point to the B'-th corner point in the h-th comparison template image, θ r,h,(A′,B′) represents the included angle between the line connecting the A'-th corner point to the B'-th corner point in the h-th comparison template image and the horizontal direction.

[0082] It should be noted that represents the position deviation between the A-th corner point and the B-th corner point in the r-th character image and the A'-th corner point and the B'-th corner point in its h-th comparison template image. The more similar the positions and distributions of the a-th pair of similar corner point pairs and the b-th pair of similar corner point pairs between the r-th character image and the h-th comparison template image are, the smaller this value is.

[0083] According to the position deviation values of each two pairs of similar corner point pairs between the r-th character image and the h-th comparison template image, and the number of corner points and corner point similarity between the r-th character image and the h-th comparison template image, the character matching degree σ between the r-th character image and the h-th comparison template image is obtained. r,h , and the calculation formula is as follows:

[0084]

[0085] In the formula: δ r,t represents the corner similarity between the r-th character image and the h-th comparison template image, W r represents the number of all corners in the r-th character image, W r,h represents the number of all corners in the h-th comparison template image of the r-th character image, U r,h represents the number of similar corner pairs between the r-th character image and the h-th comparison template image, exp[] represents the exponential function with the natural constant as the base; || represents the absolute value function.

[0086] Among them, it should be noted that δ r,t serves as a weight here. The larger this value is, the more similar the corner distribution of the r-th character image is to its h-th comparison template image, and thus the greater the character matching degree; |W r -W r,h | represents the difference in the number of corners between the r-th character image and the h-th comparison template image. The smaller this value is, the closer the number of corners of the r-th character image is to that of its h-th comparison template image, and the greater the character matching degree; μ r,h,(A,B) represents the position deviation value between the a-th pair of similar corner pairs and the b-th pair of similar corner pairs of the r-th character image and the h-th comparison template image, represents the result of summing up the position deviation values of each pair of similar corner pairs in the r-th character image and the h-th comparison template image after inverse proportional normalization, indicating the corner distribution difference between the r-th character image and the h-th comparison template image. The larger this value is, the smaller the position deviation value of each pair of similar corner pairs between the r-th character image and the h-th comparison template image, the higher the coincidence degree of the similar corner pairs, and the greater the character matching degree.

[0087] Obtain the character matching degree of each character image with all its comparison template images, select the text information of the character corresponding to the comparison template image with the highest character matching degree for each character image, record it as the character recognition result of each character image, obtain the character recognition results of all character images, and arrange them in the order of each character image in the original license plate image to obtain the recognized result text information of the license plate, thus completing the recognition enhancement of the license plate.

[0088] It should be noted that the exp(-x) model used in this embodiment only represents a negative correlation relationship and constrains the output result of the model to be within the interval (0, 1]. Here, x is the input of this model. In specific implementation, it can be replaced with other models with the same purpose. This embodiment only takes the exp(-x) model as an example for description and is not specifically limited.

[0089] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A license plate recognition enhancement method for an unattended toll collection system in a smart city, characterized in that: The method comprises the following steps: Obtain a grayscale image of the license plate; segment the grayscale image of the license plate to obtain a single word image of each character; cluster the grayscale values ​​of the single word image several times to obtain several clusters of each clustering of the single word image; obtain the cluster with the maximum probability of each clustering according to the grayscale stability of the pixel points of each cluster in each cluster of other sub-clusters; obtain the character image according to the number of times the pixel points of the single word image appear in the cluster with the maximum probability; Perform corner point detection on the character image and all prior template images to obtain all corner points on the character image and the template image; obtain the corner point similarity between each character image and each template image based on the similarity of the corner point distribution between each character image and each template image; obtain all high-probability template images for each character image; A three-dimensional coordinate system is established based on the relative position features of each corner point and adjacent corner points; based on the density of all corner points of the character image and each high-probability template image of the character image in the three-dimensional coordinate system, similar corner point pairs of the character image and all high-probability template images of the character image are obtained; based on the distance between the two closest pairs of similar corner point pairs of the character image and each high-probability template image in the three-dimensional coordinate system, each high-probability template image is scaled to obtain all comparison template images of the character image; based on the difference in the number of corner points and the difference in the distribution of corner points between all character images and each comparison template image, the character matching degree between the character image and each comparison template image is obtained; the template image with the largest character matching degree of the comparison template image is recorded as the character recognition result of the character image, and the recognition result text information of the license plate is obtained.

2. According to claim 1, the license plate recognition enhancement method for the smart city unattended toll collection system is characterized in that: The specific method for obtaining the cluster with the maximum probability in each clustering is: For the target single-word image, the corresponding cluster of the i-th cluster in the m-th clustering result is recorded as the j-th cluster, and the corresponding inter-cluster difference α of the i-th cluster in the n-th clustering result is n,i The calculation method is: Where: M represents the number of clustering, M-1 represents the number of clustering except the nth clustering, m represents the mth clustering except the nth clustering, E n,i represents the number of pixels in the ith cluster of the nth clustering result of the target single word image, x represents the xth pixel in the ith cluster of the nth clustering result of the target single word image, P n,i,x Represents the gray value of the xth pixel in the ith cluster in the nth clustering result of the target word image, represents the mean grayscale value of all pixels in the ith cluster in the nth clustering result of the target word image, E m,j represents the number of pixels in the jth cluster of the mth clustering result of the target single word image, y represents the yth pixel in the jth cluster of the mth clustering result of the target single word image, P m,j,y Represents the gray value of the yth pixel in the jth cluster of the mth clustering result of the target word image, represents the mean of the grayscale values ​​of all pixels in the jth cluster of the mth clustering result of the target single word image; || represents the absolute value function; The character probability β of the target single word image in the i-th cluster in the n-th clustering result n,i The calculation method is: Where: represents the mean grayscale value of all pixels in the i-th cluster of the target word image in the n-th clustering result, exp() represents an exponential function with a natural constant as the base, and Sigmoid[] represents the Sigmoid function.

3. According to claim 2, the license plate recognition enhancement method for the smart city unattended toll collection system is characterized in that: The specific method for obtaining the corresponding cluster is: The cluster centers of all clusters obtained from all sub-clustering of the target single-word image are mapped to the same image, the distance between the cluster centers of every two clusters is recorded as the distance between the two clusters, and the jth cluster of the i-th cluster of the n-th clustering result that satisfies the minimum distance with the i-th cluster in the m-th clustering result is recorded as the corresponding cluster of the i-th cluster of the n-th clustering result in the m-th clustering result.

4. According to claim 1, the license plate recognition enhancement method for the smart city unattended toll collection system is characterized in that: The step of obtaining a character image according to the number of times a pixel point of a single character image appears in a cluster with the maximum probability comprises: The cluster with the highest character probability among all clusters of each clustering result for each single-word image is recorded as the cluster with the highest probability for each clustering result; for any pixel point of the single-word image, if the pixel point is included in the cluster with the highest probability of a clustering result, the pixel point is recorded as appearing once, the number of occurrences of each pixel point is obtained, and the pixel points whose number of occurrences exceeds the preset repetition threshold are recorded as the actual character area pixel points; The grayscale values ​​of all pixels in the actual character area of ​​the single-word image are set to 0, and the grayscale values ​​of all pixels in the non-actual character area are set to 1. The processed result image is recorded as the character image.

5. According to claim 1, the license plate recognition enhancement method for the smart city unattended toll collection system is characterized in that: The step of obtaining the corner point similarity between each character image and each template image according to the similarity of the corner point distribution between each character image and each template image comprises: Corner similarity δ between the rth character image and the tth template image r,t The calculation method is: Where: Represents the mean of the character probabilities of the cluster with the largest probability among all clustering results of the r-th character image, represents the mean of the feature angles of all corner points of the r-th character image, represents the mean of the feature angles of all corner points of the t-th template image; norm[] represents the linear normalization function; exp() represents the exponential function with a natural constant as the base; A similarity threshold is preset, and all template images whose similarity with the corner points of the r-th character image is greater than the preset similarity threshold are recorded as all high-probability template images of the r-th character image.

6. The method for enhancing license plate recognition for an unattended toll collection system in a smart city according to claim 5 is characterized in that: The specific method for obtaining the characteristic angle of the corner point is: For each corner point in the character image and the template image, the angle between the line connecting another corner point with the smallest distance from each corner point and the horizontal direction is recorded as the characteristic angle of each corner point.

7. The method for enhancing license plate recognition for an unattended toll collection system in a smart city according to claim 1, characterized in that: The step of establishing a three-dimensional coordinate system according to the relative position characteristics of each corner point and adjacent corner points includes: Obtain the angles between the lines connecting the three corner points with the smallest, second smallest, and third smallest Euclidean distances to the position of each corner point in the character image and the horizontal direction, and record them as the first attribute, second attribute, and third attribute of each corner point in the character image; obtain the first attribute, second attribute, and third attribute of all corner points in the character image and the high-probability template image, and construct a three-dimensional coordinate system using the first attributes of all corner points as the x-axis coordinates, the second attributes as the y-axis coordinates, and the third attributes as the z-axis coordinates.

8. The method for enhancing license plate recognition for an unattended toll collection system in a smart city according to claim 1, characterized in that: The method of obtaining similar corner point pairs between the character image and all high-probability template images of the character image according to the density of all corner points of the character image and each high-probability template image of the character image in the three-dimensional coordinate system comprises: In the three-dimensional coordinate system, the first DBSCAN density clustering is performed with the first clustering radius and the first clustering sample number to obtain a dense clustering result; for all corner points except the dense clustering result, the second DBSCAN density clustering is performed with the second clustering radius and the second clustering sample number to obtain a sparse clustering result; among all the clusters of the sparse clustering result, the cluster that satisfies the condition that a cluster contains only two corner points and the two corner points belong to the character image and the high-probability template image respectively is recorded as a similar corner point pair of the character image and the high-probability template image of the character image.

9. The method for enhancing license plate recognition for an unattended toll collection system in a smart city according to claim 1, characterized in that: The method of scaling each high-probability template image according to the distance between the two pairs of similar corner points of the character image and each high-probability template image in the three-dimensional coordinate system to obtain all comparison template images of the character image includes: For the r-th character image and the h-th high probability template image of the r-th character image, the two pairs of similar corner point pairs with the smallest Euclidean distance between the r-th character image and the h-th high probability template image of the r-th character image in the three-dimensional coordinate system are recorded as the a-th pair of similar corner point pairs and the b-th pair of similar corner point pairs. For the A-th corner point belonging to the r-th character image in the a-th pair of similar corner point pairs, and the A-th corner point belonging to the h-th high probability template image of the r-th character image, ′ corner points, and the Bth corner point of the bth pair of similar corner points belonging to the rth character image, and the Bth corner point of the hth high probability template image belonging to the rth character image ′ corner points; the length of the line from the Ath corner point to the Bth corner point and the length of the line from the Ath corner point to the Bth corner point ′ Corner point to B ′ The ratio of the lengths of the lines connecting the corner points is recorded as the scaling scale of the h-th high probability template image of the r-th character image; the h-th high probability template image of the r-th character image is scaled proportionally according to the scaling scale to obtain the h-th comparison template image.

10. The method for enhancing license plate recognition for an unattended toll collection system in a smart city according to claim 9, characterized in that: The step of obtaining the character matching degree between the character image and each comparison template image according to the difference in the number of corner points and the difference in the distribution of corner points between all the character images and each comparison template image includes: The position deviation value μ between the rth character image and the hth comparison template image r,h,(A,B) The calculation method is: Where: d r,(A,B) represents the length of the line from the Ath corner point to the Bth corner point in the rth character image, θ r,(A,B) represents the angle between the line from the Ath corner point to the Bth corner point in the rth character image and the horizontal direction, d r,h,(A′,B′) Indicates the Ath in the hth comparison template image ′ Corner point to B ′ The length of the line connecting the corner points, θ r,h,(A′,B′) Indicates the Ath in the hth comparison template image ′ Corner point to B ′ The angle between the line connecting the corner points and the horizontal direction; The character matching degree σ between the rth character image and the hth comparison template image r,h The calculation method is: Where: r,t represents the corner similarity between the rth character image and the hth comparison template image, W r represents the number of all corner points in the rth character image, W r,h represents the number of all corner points in the hth contrast template image of the rth character image, U r,h It represents the number of similar corner point pairs between the rth character image and the hth comparison template image. exp[] represents an exponential function with a natural constant as the base; || represents the absolute value function.

Images