License plate recognition enhancement method for unattended toll collection system in smart cities

Abstract

The present invention relates to the field of image recognition technology, and more specifically to a license plate recognition enhancement method for an unmanned toll collection system in a smart city, comprising: segmenting a license plate grayscale image and obtaining character images through multiple clustering operations; preliminarily comparing each character image with each template image to obtain a high-probability template image for each character image; scaling the high-probability template image based on corner point features of the character image and the template image; and matching all character images with the scaled high-probability template image based on corner point features to obtain a license plate recognition result. The present invention segments the license plate image and obtains each character image that does not contain dirty areas through multiple clustering operations, thereby preventing the dirty areas from affecting subsequent analysis. Scaling and adjusting the scale based on corner point features ensures that the template characters are as consistent as possible with the character image scale during matching, thereby improving the accuracy of license plate recognition.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a license plate recognition enhancement method for an unmanned toll collection system in a smart city. Background Art

[0002] Smart city unmanned toll collection systems can improve the efficiency of toll booths to cope with the growing vehicle traffic. By automatically recognizing license plates, unmanned toll collection systems can significantly improve traffic efficiency and reduce waiting times. For example, unmanned highway toll collection systems capture and photograph vehicle license plates and use template matching technology to match each character in the license plate, thereby identifying and extracting the license plate character information.

[0003] License plates are often contaminated with dust, mud, and other dirt. When the dirt covers the license plate characters, the characters on the collected license plate image will be missing or blurred. In the traditional template matching process, the template characters need to be matched with the collected license plate character image to obtain the result. Directly using template matching for license plate recognition of missing or blurred license plates may cause errors in scale adjustment when proportionally transforming the template characters due to the influence of dirt, which in turn leads to inaccurate recognition and incorrect license plate recognition results, reducing the working efficiency of the smart city unmanned toll collection system. Summary of the Invention

[0004] The present invention provides a license plate recognition enhancement method for a smart city unmanned toll collection system to solve the existing problem that errors occur when proportionally transforming template characters due to dirt, resulting in inaccurate recognition and reduced working efficiency of the smart city unmanned toll collection system.

[0005] The method for enhancing license plate recognition for the smart city unmanned toll collection system of the present invention adopts the following technical solutions:

[0006] One embodiment of the present invention provides a method for enhancing license plate recognition for a smart city unmanned toll collection system, the method comprising the following steps:

[0007] Obtain a grayscale image of the license plate; segment the grayscale image of the license plate to obtain a single word image of each character; cluster the grayscale values of the single word image several times to obtain several clusters of each clustering of the single word image; obtain the cluster with the highest probability for each clustering based on the grayscale stability of the pixel points of each cluster in each cluster of other sub-clusters; obtain the character image based on the number of times the pixel points of the single word image appear in the cluster with the highest probability;

[0008] Perform corner point detection on the character image and all prior template images to obtain all corner points on the character image and template image; obtain the corner point similarity between each character image and each template image based on the similarity of the corner point distribution between each character image and each template image; obtain all high-probability template images for each character image;

[0009] A three-dimensional coordinate system is established based on the relative position features of each corner point and the adjacent corner points; based on the density of all corner points of the character image and each high-probability template image of the character image in the three-dimensional coordinate system, similar corner point pairs of the character image and all high-probability template images of the character image are obtained; based on the distance between the two pairs of similar corner point pairs with the closest coordinates between the character image and each high-probability template image in the three-dimensional coordinate system, each high-probability template image is scaled to obtain all comparison template images of the character image; based on the difference in the number of corner points and the difference in the distribution of corner points between all character images and each comparison template image, the character matching degree between the character image and each comparison template image is obtained; the template image with the largest character matching degree of the comparison template image is recorded as the character recognition result of the character image, and the recognition result text information of the license plate is obtained.

[0010] Furthermore, the specific method for obtaining the cluster with the maximum probability in each clustering is:

[0011] For the target single word image, the corresponding cluster of the i-th cluster in the m-th clustering result is recorded as the j-th cluster, and the corresponding inter-cluster difference α of the i-th cluster in the n-th clustering result is n,i The calculation method is:

[0012]

[0013] Where: M represents the number of clustering, M-1 represents the number of clustering except the nth clustering, m represents the mth clustering except the nth clustering, E n,i represents the number of pixels in the i-th cluster of the n-th clustering result of the target single word image, x represents the x-th pixel in the i-th cluster of the n-th clustering result of the target single word image, P n,i,x Represents the grayscale value of the x-th pixel in the ith cluster of the n-th clustering result of the target word image, represents the mean grayscale value of all pixels in the i-th cluster in the n-th clustering result of the target word image, E m,j represents the number of pixels in the jth cluster of the mth clustering result of the target single word image, y represents the yth pixel in the jth cluster of the mth clustering result of the target single word image, P m,j,y Represents the grayscale value of the yth pixel in the jth cluster of the mth clustering result of the target single word image, represents the mean of the grayscale values of all pixels in the jth cluster of the mth clustering result of the target single word image; || represents the absolute value function;

[0014] The character probability β of the target single word image in the i-th cluster in the n-th clustering result n,i The calculation method is:

[0015]

[0016] Where: represents the mean grayscale value of all pixels in the i-th cluster of the target word image in the n-th clustering result, exp() represents the exponential function with a natural constant as the base, and Sigmoid[] represents the Sigmoid function.

[0017] Furthermore, the specific method for obtaining the corresponding cluster is:

[0018] The cluster centers of all clusters obtained from all sub-clustering of the target single-word image are mapped to the same image, and the distance between the cluster centers of every two clusters is recorded as the distance between the two clusters. The jth cluster of the i-th cluster in the n-th clustering result that satisfies the minimum distance to the i-th cluster in the m-th clustering result is recorded as the corresponding cluster of the i-th cluster in the n-th clustering result in the m-th clustering result.

[0019] Furthermore, the step of obtaining a character image according to the number of times a pixel point of a single character image appears in a cluster with the maximum probability includes:

[0020] For each single-word image, the cluster with the highest character probability among all clusters in each clustering result is recorded as the cluster with the highest probability in each clustering result. For any pixel point in the single-word image, if the pixel point is included in the cluster with the highest probability in a clustering result, the pixel point is recorded as appearing once. The number of occurrences of each pixel point is obtained, and the pixel points whose number of occurrences exceeds the preset repetition threshold are recorded as the actual character area pixel points.

[0021] The grayscale values of all pixels in the actual character area of the single-word image are set to 0, and the grayscale values of all pixels in the non-actual character area are set to 1. The processed result image is recorded as the character image.

[0022] Furthermore, obtaining the corner point similarity between each character image and each template image based on the similarity of corner point distribution between each character image and each template image includes:

[0023] Corner similarity δ between the rth character image and the tth template image r,t The calculation method is:

[0024]

[0025] Where: Represents the mean of the character probabilities of the cluster with the largest probability among all clustering results of the r-th character image, represents the mean of the feature angles of all corner points of the r-th character image, Represents the mean of the feature angles of all corner points of the t-th template image; norm[] represents the linear normalization function; exp() represents the exponential function with a natural constant as the base;

[0026] A similarity threshold is preset, and all template images whose similarity with the corner points of the r-th character image is greater than the preset similarity threshold are recorded as all high-probability template images of the r-th character image.

[0027] Furthermore, the specific method for obtaining the characteristic angle of the corner point is:

[0028] For each corner point in the character image and the template image, the angle between the line connecting the other corner point with the smallest distance from each corner point and the horizontal direction is recorded as the characteristic angle of each corner point.

[0029] Furthermore, establishing a three-dimensional coordinate system based on the relative position characteristics of each corner point and adjacent corner points includes:

[0030] Obtain the angles between the lines connecting the three corner points with the smallest, second smallest, and third smallest Euclidean distances to the position of each corner point in the character image and the horizontal direction, and record them as the first attribute, second attribute, and third attribute of each corner point in the character image; obtain the first attribute, second attribute, and third attribute of all corner points in the character image and the high-probability template image, and use the first attribute of all corner points as the x-axis coordinate, the second attribute as the y-axis coordinate, and the third attribute as the z-axis coordinate to construct a three-dimensional coordinate system.

[0031] Furthermore, the obtaining of similar corner point pairs between the character image and all high-probability template images of the character image according to the density of all corner points of the character image and each high-probability template image of the character image in the three-dimensional coordinate system includes:

[0032] In the three-dimensional coordinate system, the first DBSCAN density clustering is performed with the first clustering radius and the first clustering sample number to obtain a dense clustering result; for all corner points except the dense clustering result, the second DBSCAN density clustering is performed with the second clustering radius and the second clustering sample number to obtain a sparse clustering result; among all clusters of the sparse clustering result, the cluster that satisfies the requirement of containing only two corner points in a cluster, and the two corner points belong to the character image and the high-probability template image respectively, is recorded as a similar corner point pair of the character image and the high-probability template image of the character image.

[0033] Furthermore, scaling each high-probability template image according to the distance between the two closest pairs of similar corner points of the character image and each high-probability template image in the three-dimensional coordinate system to obtain all comparison template images of the character image includes:

[0034] For the r-th character image and the h-th high-probability template image of the r-th character image, the two pairs of similar corner point pairs with the smallest Euclidean distance between the r-th character image and the h-th high-probability template image of the r-th character image in the three-dimensional coordinate system are recorded as the a-th pair of similar corner point pairs and the b-th pair of similar corner point pairs. For the A-th corner point belonging to the r-th character image in the a-th pair of similar corner point pairs and the A′-th corner point belonging to the h-th high-probability template image of the r-th character image, as well as the B-th corner point belonging to the r-th character image in the b-th pair of similar corner point pairs and the B′-th corner point belonging to the h-th high-probability template image of the r-th character image; the ratio of the length of the line connecting the A-th corner point to the B-th corner point to the length of the line connecting the A′-th corner point to the B′-th corner point is recorded as the scaling scale of the h-th high-probability template image of the r-th character image; the h-th high-probability template image of the r-th character image is scaled proportionally according to the scaling scale to obtain the h-th comparison template image.

[0035] Furthermore, obtaining the character matching degree between the character image and each comparison template image based on the difference in the number of corner points and the difference in the distribution of corner points between all the character images and each comparison template image includes:

[0036] The position deviation value μ between the rth character image and the hth comparison template image r,h,(A,B) The calculation method is:

[0037]

[0038] Where: d r,(A,B) represents the length of the line from the Ath corner point to the Bth corner point in the rth character image, θ r,(A,B) represents the angle between the line connecting the Ath corner point to the Bth corner point in the rth character image and the horizontal direction, d r,h,(A′,B′) represents the length of the line from the A′th corner point to the B′th corner point in the hth comparison template image, θ r,h,(A′,B′) represents the angle between the line connecting the A′th corner point to the B′th corner point in the hth comparison template image and the horizontal direction;

[0039] The character matching degree σ between the rth character image and the hth comparison template image r,h The calculation method is:

[0040]

[0041] Where: δ r,tW represents the similarity between the corners of the rth character image and the hth comparison template image, r represents the number of all corner points in the r-th character image, W r,h represents the number of all corner points in the hth contrast template image of the rth character image, U r,h It represents the number of similar corner point pairs between the rth character image and the hth comparison template image, exp[] represents an exponential function with a natural constant as the base; || represents the absolute value function.

[0042] The beneficial effects of the present invention are as follows:

[0043] The present invention segments a license plate grayscale image and performs several clustering operations to obtain character images. Each character image is initially compared with each template image to obtain a high-probability template image for each character image. The high-probability template image is scaled based on the corner features of the character and template images. All character images are then matched with the scaled high-probability template image based on the corner features to obtain license plate recognition results. The present invention segments the license plate image and performs several clustering operations to obtain character images that do not contain dirty areas, preventing dirty areas from affecting subsequent analysis. Scaling and adjusting the scale based on the corner features ensures that the template characters are as consistent as possible with the character image scale during matching, thereby improving license plate recognition accuracy and, in turn, enhancing the efficiency of the smart city unmanned toll collection system. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 This is a flowchart of the steps of the license plate recognition enhancement method for the smart city unmanned toll collection system of the present invention. DETAILED DESCRIPTION

[0046] To further illustrate the technical means and effectiveness of the present invention in achieving its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of the license plate recognition enhancement method for a smart city unmanned toll collection system proposed by the present invention. In the following description, different references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0047] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0048] The specific scheme of the license plate recognition enhancement method for the smart city unmanned toll collection system provided by the present invention is described in detail below with reference to the accompanying drawings.

[0049] See also Figure 1 , which shows a flowchart of a method for enhancing license plate recognition for a smart city unattended toll collection system according to an embodiment of the present invention, the method comprising the following steps:

[0050] Step S001: Obtain a grayscale image of a license plate.

[0051] The purpose of this embodiment is to recognize license plate characters, so it is necessary to first collect a license plate image; when a vehicle approaches a toll station, the toll collection system senses the arrival of the vehicle through a sensor device. The sensor devices used to sense the arrival of the vehicle include, for example, infrared, radar, magnetic sensors, etc., which are not specifically limited in this embodiment; after sensing the arrival of the vehicle, the toll collection system triggers a high-resolution camera to capture the original image of the vehicle's license plate; since in the highway toll stations of smart cities, the high-resolution camera of the toll collection system is located on the left side of the road, when capturing the vehicle's license plate image, the license plate image will be deformed due to perspective, and the captured license plate image needs to be corrected and preprocessed.

[0052] Specifically, the original image of the vehicle's license plate is collected by a high-resolution camera of the toll collection system, and the original image of the license plate is perspective corrected to obtain a corrected license plate image; the corrected license plate image is averaged grayscale processed to obtain a license plate grayscale image.

[0053] Step S002, segmenting the grayscale image to obtain a single-word image; clustering the grayscale values of the single-word image to obtain the cluster with the maximum probability for each clustering; and obtaining a character image based on the number of times the pixel points of the single-word image appear in the cluster with the maximum probability.

[0054] Since the specifications of license plates are uniform, the grayscale image of the license plate is segmented according to the proportional position of each character in the entire license plate to obtain a single-word image of each character. Each single-word image contains only one character of the license plate, and the contained character is completely located in the single-word image. It should be noted that the proportional position of each character in the license plate in the entire license plate is a well-known scenario and will not be described in detail in this embodiment.

[0055] Since the grayscale values of the actual character area and the background area are more uniform than the grayscale values of the dirty area, the clusters of the actual character area and the background area have better stability. The stability is manifested in the results of multiple clustering. The grayscale values of each single word image are subjected to multiple K-means clustering. The actual character area and the background area vary less in the K-means clustering results under different K values, while the dirty area varies more in the K-means clustering results under different K values, and may be divided into multiple different clusters under different K values; therefore, stable clusters can be screened out according to the stability of the clusters, and the stable clusters are the actual character areas or background areas that are not affected by dirt coverage; and since the actual character area has a larger grayscale value than the background area, the clusters of the actual character area and the clusters corresponding to the background area are distinguished according to the size of the grayscale value, and the actual character area in each single word image is determined.

[0056] Specifically, K-means clustering is performed M times on the grayscale value of each single word image. In this embodiment, K values are set to 3, 4, 5, 6, 7, and 8 respectively, and M=6 clusterings are performed. Other embodiments may set other K values and clustering times. This embodiment does not make specific limitations, and the clustering results after six K-means clustering of each single word image are obtained; the cluster centers of all clusters obtained by all clustering are mapped to the same image, and the distance between the cluster centers of every two clusters is recorded as the distance between the two clusters. A cluster of the current clustering that satisfies the minimum distance between the two clusters in another clustering result is recorded as the corresponding cluster of the cluster of the current clustering in the other clustering result; as an example, if the distance between the first cluster in the first clustering and the third cluster in the second clustering is the smallest, then the third cluster of the second clustering is called the corresponding cluster of the first cluster of the first clustering in the second clustering result, and each cluster of each clustering has a corresponding cluster in the other clustering results.

[0057] Furthermore, for the target single-word image, the corresponding cluster of the i-th cluster in the m-th clustering result is recorded as the j-th cluster, and the corresponding inter-cluster difference α of the i-th cluster in the n-th clustering result is n,i The calculation method is:

[0058]

[0059] Where: M represents the number of clustering, M-1 represents the number of clustering except the nth clustering, m represents the mth clustering except the nth clustering, E n,i represents the number of pixels in the i-th cluster of the n-th clustering result of the target single word image, x represents the x-th pixel in the i-th cluster of the n-th clustering result of the target single word image, P n,i,x Represents the grayscale value of the x-th pixel in the ith cluster of the n-th clustering result of the target word image, represents the mean grayscale value of all pixels in the i-th cluster in the n-th clustering result of the target word image, E m,j represents the number of pixels in the jth cluster of the mth clustering result of the target single word image, y represents the yth pixel in the jth cluster of the mth clustering result of the target single word image, P m,j,y Represents the grayscale value of the yth pixel in the jth cluster of the mth clustering result of the target single word image, represents the mean grayscale value of all pixels in the jth cluster of the mth clustering result of the target single word image; || represents the absolute value function.

[0060] It should be noted that, Indicates the uniformity of the grayscale values of pixels in the i-th cluster of the n-th clustering result. Indicates the uniformity of the grayscale values of the pixels in the corresponding cluster of the i-th cluster of the m-th clustering result outside the n-th clustering. It represents the difference in average uniformity of grayscale values between the i-th cluster in the n-th clustering result and the corresponding cluster in its multiple clustering results. The smaller the difference, the smaller the inter-cluster difference of the i-th cluster, and the greater the stability of the i-th cluster in multiple clustering.

[0061] The character probability β of the target single word image in the i-th cluster in the n-th clustering result n,i The calculation method is:

[0062]

[0063] Where: It represents the mean grayscale value of all pixels in the i-th cluster of the n-th clustering result of the target word image, exp() represents the exponential function with a natural constant as the base, and Sigmoid[] represents the Sigmoid function.

[0064] It should be noted that α n,i Indicates the corresponding inter-cluster difference of the i-th cluster of the n-th clustering result of the target single-word image. The smaller the value, the higher the stability of the i-th cluster of the n-th clustering result in multiple clustering. Therefore, the i-th cluster of the n-th clustering result is less likely to be a dirty area, but rather an actual character area or background area. The larger , the larger the average gray value of the i-th cluster of the n-th clustering result is, and the i-th cluster of the n-th clustering result is more likely to be the actual character area in both the actual character area and the background area; The larger the result of , the greater the stability of the i-th cluster of the n-th clustering result. At the same time, the larger the average gray value of the pixels in the cluster, the more likely the i-th cluster of the n-th clustering result is to be the actual character area.

[0065] Similarly, obtain the character probabilities of all clusters of all clustering results of all single-word images; obtain the cluster with the largest character probability among all clusters of each clustering result for each single-word image, and record it as the cluster with the largest probability for each clustering result; for any pixel point of the single-word image, if the pixel point is included in the cluster with the largest probability of a clustering result, record the pixel point as appearing once, obtain the number of occurrences of each pixel point, and record the pixel points whose number of occurrences exceeds the preset repetition threshold as actual character area pixel points. This embodiment is described with the preset repetition threshold as 3, and other embodiments can be set to other values, which are not specifically limited in this embodiment; binarize each single-word image according to all the actual character area pixel points of each single-word image, set the grayscale values of all actual character area pixel points of the single-word image to 0, and set the grayscale values of all non-actual character area pixel points to 1, and the processed result image is recorded as the character image.

[0066] Step S003: Obtain all corner points on the character image and the template image; obtain the corner point similarity between each character image and each template image based on the similarity of the corner point distribution between each character image and each template image; obtain all high-probability template images for each character image.

[0067] It should be noted that since all characters that appear in the license plate have specifications and fonts specified by the standard font when the license plate is produced, the template images of all characters that appear can be obtained according to the standard font regulations. By comparing the difference in the number of corner points and the similarity in the distribution of corner points between each character image and each template image, the template image with the highest similarity to each character image can be obtained.

[0068] Specifically, a template image of all characters of the license plate is obtained, and corner detection is performed on all character images and all template images to obtain the corner points of the character image and the template image respectively. In this embodiment, the corner detection algorithm uses the SIFT (Scale-Invariant Feature Transform) corner detection algorithm. Other embodiments may choose other corner detection algorithms, which are not specifically limited in this embodiment. For each corner point in the character image and the template image, the angle between the line connecting another corner point with the smallest distance from each corner point and the horizontal direction is recorded as the characteristic angle of each corner point. It should be noted that the angles in this embodiment all start from the horizontal straight line, and rotate counterclockwise toward the straight line where the connecting line segment is located until they coincide. The angle traversed during the rotation is in the range of [0,180°). Similarly, the characteristic angle of each corner point in the template image is obtained.

[0069] Corner similarity δ between the rth character image and the tth template image r,t The calculation method is:

[0070]

[0071] Where: Represents the mean of the character probabilities of the cluster with the largest probability among all clustering results of the r-th character image, represents the mean of the feature angles of all corner points of the r-th character image, Represents the mean of the feature angles of all corner points of the t-th template image; norm[] represents the linear normalization function; exp() represents the exponential function with a natural constant as the base.

[0072] Among them, it should be noted that Represents the difference between the mean of the feature angles of all corner points of the r-th character image and the t-th template image, which shows the similarity of the corner point distribution of the r-th character image and the t-th template image. The smaller the value, the more similar the corner point distribution of the r-th character image and the t-th template image is, and the greater the similarity of the corner points of the r-th character image and the t-th template image is. exp() is used here to map the value to the range of (0, +∞) to avoid the denominator being 0. The value of is used as the confidence parameter. The larger the value is, the more trustworthy the corner similarity between the r-th character image and the t-th template image is.

[0073] Similarly, the similarity of the corner points of all character images and all template images is obtained; a similarity threshold is preset, and for the r-th character image, all template images whose corner point similarity with the r-th character image is greater than the preset similarity threshold are recorded as all high-probability template images of the r-th character image. This embodiment is described with the preset similarity threshold as 0.68, and other embodiments can be set to other values, which are not specifically limited in this embodiment; high-probability template images of all character images are obtained.

[0074] Step S004: Scale each high-probability template image to obtain all comparison template images of the character image; obtain the character matching degree between the character image and each comparison template image based on the difference in the number of corner points and the difference in the distribution of corner points between all character images and each comparison template image; obtain the character recognition result, and then obtain the recognition result text information of the license plate.

[0075] It should be noted that since the distance between each car and the camera is not fixed when collecting images, the scaling scale of the license plate images collected by the camera is also different, which may lead to errors when comparing the template image with the character image. Therefore, we first obtain multiple high-probability template characters and similar corner point pairs in the actual characters, which are regarded as a pair of corner points in the corresponding positions of the template image and the character image in the character content. Then, based on the positional features of the similar corner point pairs, we scale the character image to the same scale as the template image, and calculate the character matching degree between the scaled character image and each high-probability template image.

[0076] Specifically, the angles between the lines connecting the three corner points with the smallest, second smallest, and third smallest Euclidean distances to the position of each corner point in the character image and the horizontal direction are obtained respectively, and recorded as the first attribute, second attribute, and third attribute of each corner point in the character image; the first attribute, second attribute, and third attribute of all corner points in the character image and the high-probability template image are obtained, and a three-dimensional coordinate system is constructed by taking the first attributes of all corner points as the x-axis coordinates, the second attributes as the y-axis coordinates, and the third attributes as the z-axis coordinates, and projecting all corner points into the three-dimensional coordinate system to obtain the coordinates of each corner point in a character image. Similarly, the coordinates of each corner point in a high-probability template image of the character image are obtained.

[0077] In the three-dimensional coordinate system, the first DBSCAN density clustering is performed with the first cluster radius and the first cluster sample number to obtain a dense clustering result; the first cluster radius is preset to 10 and the first cluster sample number is preset to 8 for DBSCAN density clustering. Other embodiments may set other cluster radii and cluster sample numbers, which are not specifically limited in this embodiment.

[0078] Since dirty areas will produce dense corner points, all dense clustering results are excluded, and for all corner points except the dense clustering results, a second DBSCAN density clustering is performed with the second clustering radius and the second clustering sample number to obtain a sparse clustering result; the second clustering radius is preset to 10 and the second clustering sample number is preset to 2 for DBSCAN density clustering. Other embodiments may set other clustering radii and clustering sample numbers, which are not specifically limited in this embodiment; among all clusters of the sparse clustering results, the cluster that contains only two corner points in a cluster, and the two corner points belong to the character image and the high-probability template image respectively, is recorded as a similar corner point pair of the character image and the high-probability template image of the character image, and all similar corner point pairs of the character image and the high-probability template image are obtained; similarly, similar corner point pairs of all character images and all their high-probability template images are obtained.

[0079] Furthermore, for the r-th character image and the h-th high probability template image of the r-th character image, the two pairs of similar corner point pairs with the smallest Euclidean distance between the r-th character image and the h-th high probability template image of the r-th character image in the three-dimensional coordinate system are recorded as the a-th pair of similar corner point pairs and the b-th pair of similar corner point pairs, for the A-th corner point belonging to the r-th character image in the a-th pair of similar corner point pairs, and the A′-th corner point belonging to the h-th high probability template image of the r-th character image, as well as the B-th corner point belonging to the r-th character image in the b-th pair of similar corner point pairs, and the h-th high probability template image belonging to the r-th character image B′th corner point; the ratio of the length of the line connecting the Ath corner point to the Bth corner point to the length of the line connecting the A′th corner point to the B′th corner point is recorded as the scaling scale of the hth high-probability template image of the rth character image; the hth high-probability template image of the rth character image is scaled proportionally by the scaling scale to obtain the hth comparison template image; the rth character is compared with the hth comparison template image again. Since it is proportional scaling, the angle relationship remains unchanged and the length of the connecting line segment changes; the position deviation value μ between the ath pair of similar corner points and the bth pair of similar corner points of the rth character image and the hth comparison template image is obtained. r,h,(A,B) , the calculation formula is as follows:

[0080]

[0081] Where: d r,(A,B) represents the length of the line from the Ath corner point to the Bth corner point in the rth character image, θ r,(A,B) represents the angle between the line connecting the Ath corner point to the Bth corner point in the rth character image and the horizontal direction, d r,h,(A′,B′) represents the length of the line from the A′th corner point to the B′th corner point in the hth comparison template image, θ r,h,(A′,B′) It represents the angle between the line connecting the A′th corner point to the B′th corner point in the hth comparison template image and the horizontal direction.

[0082] It should be noted that, It represents the position deviation between the Ath and Bth corner points in the rth character image and the A′th and B′th corner points in its hth comparison template image. The more similar the position and distribution of the ath and bth pairs of similar corner points between the rth character image and the hth comparison template image are, the smaller the value.

[0083] According to the position deviation value of each pair of similar corner points between the rth character image and the hth comparison template image, and the number of corner points and the similarity of corner points between the rth character image and the hth comparison template image, the character matching degree σ between the rth character image and the hth comparison template image is obtained. r,h , the calculation formula is as follows:

[0084]

[0085] Where: δ r,t W represents the similarity between the corners of the rth character image and the hth comparison template image, r represents the number of all corner points in the r-th character image, W r,h represents the number of all corner points in the hth contrast template image of the rth character image, U r,h It represents the number of similar corner point pairs between the rth character image and the hth comparison template image, exp[] represents an exponential function with a natural constant as the base; || represents the absolute value function.

[0086] It should be noted that δ r,t Here, it is used as a weight. The larger the value, the more similar the corner point distribution of the rth character image is to its hth comparison template image, and the greater the character matching degree. r -W r,h | represents the difference in the number of corner points between the rth character image and the hth comparison template image. The smaller the value, the closer the number of corner points between the rth character image and its hth comparison template image is, and the greater the character matching degree; μ r,h,(A,B) It represents the position deviation value of the ath similar corner point pair and the bth similar corner point pair between the rth character image and the hth comparison template image, It represents the result of accumulating the position deviation values of every two pairs of similar corner points in the rth character image and the hth comparison template image after inverse proportional normalization, which represents the difference in corner point distribution between the rth character image and the hth comparison template image. The larger the value, the smaller the position deviation value of every two pairs of similar corner points in the rth character image and the hth comparison template image, the higher the overlap of similar corner point pairs, and the greater the character matching degree.

[0087] Obtain the character matching degree of each character image with all its comparison template images, select the text information of the character corresponding to the comparison template image with the highest character matching degree for each character image, record it as the character recognition result of each character image, obtain the character recognition results of all character images, and arrange them according to the order of each character image in the original license plate image to obtain the recognition result text information of the license plate, and complete the recognition enhancement of the license plate.

[0088] It should be noted that the exp(-x) model used in this embodiment only indicates that the negative correlation and the result of the constraint model output are in the interval (0,1], where x is the input of this model. In specific implementation, it can be replaced with other models with the same purpose. This embodiment only uses the exp(-x) model as an example for description without specific limitation.

[0089] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A license plate recognition enhancement method for a smart city unattended toll collection system, characterized in that: The method comprises the following steps: Obtain a grayscale image of the license plate; segment the grayscale image of the license plate to obtain a single word image of each character; cluster the grayscale values of the single word image several times to obtain several clusters of each clustering of the single word image; obtain the cluster with the highest probability for each clustering based on the grayscale stability of the pixel points of each cluster in each cluster of other sub-clusters; obtain the character image based on the number of times the pixel points of the single word image appear in the cluster with the highest probability; Perform corner point detection on the character image and all prior template images to obtain all corner points on the character image and template image; obtain the corner point similarity between each character image and each template image based on the similarity of the corner point distribution between each character image and each template image; obtain all high-probability template images for each character image; A three-dimensional coordinate system is established based on the relative position features of each corner point and the adjacent corner points; based on the density of all corner points of the character image and each high-probability template image of the character image in the three-dimensional coordinate system, similar corner point pairs of the character image and all high-probability template images of the character image are obtained; based on the distance between the two pairs of similar corner point pairs with the closest coordinates between the character image and each high-probability template image in the three-dimensional coordinate system, each high-probability template image is scaled to obtain all comparison template images of the character image; based on the difference in the number of corner points and the difference in the distribution of corner points between all character images and each comparison template image, the character matching degree between the character image and each comparison template image is obtained; the template image with the largest character matching degree of the comparison template image is recorded as the character recognition result of the character image, and the recognition result text information of the license plate is obtained; The specific method for obtaining the cluster with the maximum probability in each clustering is: For the target single word image, the corresponding cluster of the i-th cluster in the m-th clustering result is recorded as the j-th cluster, and the corresponding inter-cluster difference α of the i-th cluster in the n-th clustering result is n,i The calculation method is: Where: M represents the number of clustering, M-1 represents the number of clustering except the nth clustering, m represents the mth clustering except the nth clustering, E n,i represents the number of pixels in the i-th cluster of the n-th clustering result of the target single word image, x represents the x-th pixel in the i-th cluster of the n-th clustering result of the target single word image, P n,i,x Represents the grayscale value of the x-th pixel in the ith cluster of the n-th clustering result of the target word image, represents the mean grayscale value of all pixels in the i-th cluster in the n-th clustering result of the target word image, E m,j represents the number of pixels in the jth cluster of the mth clustering result of the target single word image, y represents the yth pixel in the jth cluster of the mth clustering result of the target single word image, P m,j,y Represents the grayscale value of the yth pixel in the jth cluster of the mth clustering result of the target single word image, represents the mean of the grayscale values of all pixels in the jth cluster of the mth clustering result of the target single word image; || represents the absolute value function; The character probability β of the target single word image in the i-th cluster in the n-th clustering result n,i The calculation method is: Where: represents the mean grayscale value of all pixels in the i-th cluster of the target word image in the n-th clustering result, exp() represents the exponential function with a natural constant as the base, and Sigmoid[] represents the Sigmoid function.

2. The license plate recognition enhancement method for the smart city unmanned toll collection system according to claim 1 is characterized in that: The specific method for obtaining the corresponding cluster is: The cluster centers of all clusters obtained from all sub-clustering of the target single-word image are mapped to the same image, and the distance between the cluster centers of every two clusters is recorded as the distance between the two clusters. The jth cluster of the i-th cluster in the n-th clustering result that satisfies the minimum distance to the i-th cluster in the m-th clustering result is recorded as the corresponding cluster of the i-th cluster in the n-th clustering result in the m-th clustering result.

3. The license plate recognition enhancement method for the smart city unmanned toll collection system according to claim 1 is characterized in that: The step of obtaining a character image according to the number of times a pixel point of a single character image appears in a cluster with the maximum probability comprises: For each single-word image, the cluster with the highest character probability among all clusters in each clustering result is recorded as the cluster with the highest probability in each clustering result. For any pixel point in the single-word image, if the pixel point is included in the cluster with the highest probability in a clustering result, the pixel point is recorded as appearing once. The number of occurrences of each pixel point is obtained, and the pixel points whose number of occurrences exceeds the preset repetition threshold are recorded as the actual character area pixel points. The grayscale values of all pixels in the actual character area of the single-word image are set to 0, and the grayscale values of all pixels in the non-actual character area are set to 1. The processed result image is recorded as the character image.

4. The license plate recognition enhancement method for the smart city unmanned toll collection system according to claim 1 is characterized in that: The obtaining of the corner point similarity between each character image and each template image according to the similarity of the corner point distribution between each character image and each template image includes: Corner similarity δ between the rth character image and the tth template image r,t The calculation method is: Where: Represents the mean of the character probabilities of the cluster with the largest probability among all clustering results of the r-th character image, represents the mean of the feature angles of all corner points of the r-th character image, Represents the mean of the feature angles of all corner points of the t-th template image; norm[] represents the linear normalization function; exp() represents the exponential function with a natural constant as the base; A similarity threshold is preset, and all template images whose similarity with the corner points of the r-th character image is greater than the preset similarity threshold are recorded as all high-probability template images of the r-th character image.

5. The license plate recognition enhancement method for the smart city unattended toll collection system according to claim 4 is characterized in that: The specific method for obtaining the characteristic angle of the corner point is: For each corner point in the character image and the template image, the angle between the line connecting the other corner point with the smallest distance from each corner point and the horizontal direction is recorded as the characteristic angle of each corner point.

6. The license plate recognition enhancement method for the smart city unmanned toll collection system according to claim 1 is characterized in that: The step of establishing a three-dimensional coordinate system based on the relative position characteristics of each corner point and adjacent corner points includes: Obtain the angles between the lines connecting the three corner points with the smallest, second smallest, and third smallest Euclidean distances to the position of each corner point in the character image and the horizontal direction, and record them as the first attribute, second attribute, and third attribute of each corner point in the character image; obtain the first attribute, second attribute, and third attribute of all corner points in the character image and the high-probability template image, and use the first attribute of all corner points as the x-axis coordinate, the second attribute as the y-axis coordinate, and the third attribute as the z-axis coordinate to construct a three-dimensional coordinate system.

7. The license plate recognition enhancement method for the smart city unmanned toll collection system according to claim 1 is characterized in that: The obtaining of similar corner point pairs between the character image and all high-probability template images of the character image according to the density of all corner points of the character image and each high-probability template image of the character image in the three-dimensional coordinate system includes: In the three-dimensional coordinate system, the first DBSCAN density clustering is performed with the first clustering radius and the first clustering sample number to obtain a dense clustering result; for all corner points except the dense clustering result, the second DBSCAN density clustering is performed with the second clustering radius and the second clustering sample number to obtain a sparse clustering result; among all clusters of the sparse clustering result, the cluster that satisfies the requirement of containing only two corner points in a cluster, and the two corner points belong to the character image and the high-probability template image respectively, is recorded as a similar corner point pair of the character image and the high-probability template image of the character image.

8. The method for enhancing license plate recognition for a smart city unmanned toll collection system according to claim 1 is characterized in that: The method of scaling each high-probability template image according to the distance between the two closest pairs of similar corner points of the character image and each high-probability template image in the three-dimensional coordinate system to obtain all comparison template images of the character image includes: For the r-th character image and the h-th high-probability template image of the r-th character image, the two pairs of similar corner point pairs with the smallest Euclidean distance between the r-th character image and the h-th high-probability template image of the r-th character image in the three-dimensional coordinate system are recorded as the a-th pair of similar corner point pairs and the b-th pair of similar corner point pairs. For the A-th corner point belonging to the r-th character image in the a-th pair of similar corner point pairs and the A′-th corner point belonging to the h-th high-probability template image of the r-th character image, as well as the B-th corner point belonging to the r-th character image in the b-th pair of similar corner point pairs and the B′-th corner point belonging to the h-th high-probability template image of the r-th character image; the ratio of the length of the line connecting the A-th corner point to the B-th corner point to the length of the line connecting the A′-th corner point to the B′-th corner point is recorded as the scaling scale of the h-th high-probability template image of the r-th character image; the h-th high-probability template image of the r-th character image is scaled proportionally according to the scaling scale to obtain the h-th comparison template image.

9. The license plate recognition enhancement method for the smart city unmanned toll collection system according to claim 8 is characterized in that: The obtaining of the character matching degree between the character image and each comparison template image according to the difference in the number of corner points and the difference in the distribution of corner points between all the character images and each comparison template image includes: The position deviation value μ between the rth character image and the hth comparison template image r , h,(A,B) The calculation method is: Where: d r,(A,B) represents the length of the line from the Ath corner point to the Bth corner point in the rth character image, θ r,(A,B) represents the angle between the line from the Ath corner point to the Bth corner point in the rth character image and the horizontal direction, d r,h,(A′,B′) represents the length of the line from the A′th corner point to the B′th corner point in the hth comparison template image, θ r,h,(A′,B′) represents the angle between the line connecting the A′th corner point to the B′th corner point in the hth comparison template image and the horizontal direction; The character matching degree σ between the rth character image and the hth comparison template image r,h The calculation method is: Where: δ r,t W represents the similarity between the corners of the rth character image and the hth comparison template image, r represents the number of all corner points in the r-th character image, W r,h represents the number of all corner points in the hth contrast template image of the rth character image, U r,h It represents the number of similar corner point pairs between the rth character image and the hth comparison template image, exp[] represents an exponential function with a natural constant as the base; || represents the absolute value function.

Images