Vehicle Visual Recognition Method for Unattended Parking Lot Toll System

Abstract

The present invention relates to the technical field of image recognition, and specifically relates to a vehicle vision recognition method for an unmanned parking lot charging system, including: obtaining the quality of the character region in the video frame according to the gray-scale distribution of the pixel points in the character region in the video frame, thereby assigning a calculation weight for subsequent recognition of the characters corresponding to each character region, recognizing each character region under each video frame, obtaining the primary recognition results of the character regions corresponding to the same character under different video frames, checking the accuracy of the primary recognition results, and obtaining the accurately and inaccurately recognized character regions; further training the recognition model of the unrecognized group to recognize the character regions with inaccurate primary recognition results. By quantifying the degree of influence of illumination on each video frame before recognizing the characters in the license plate and evaluating the quality of each character region in each video frame as the calculation weight when recognizing characters, the present invention reduces the interference caused by illumination influence.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to a vehicle vision recognition method for an unmanned parking lot charging system. Background Art

[0002] The unmanned parking lot charging system uses computer vision and image processing technologies to automatically detect a vehicle, locate the license plate, and recognize characters when the vehicle enters the unmanned parking lot, so as to obtain the parking duration of the vehicle in the unmanned parking lot and charge the corresponding fees. However, when recognizing the characters on the license plate in traditional unmanned parking lots, due to the continuous change of the position between the vehicle and the camera, each character on the license plate is affected by external light during the recognition process and the recognition is inaccurate. Therefore, the vehicle needs to stop at the entrance and exit of the unmanned parking lot for a period of time to collect and analyze the license plate information of the vehicle, resulting in low efficiency when the vehicle enters the unmanned parking lot. Summary of the Invention

[0003] The present invention provides a vehicle vision recognition method for an unmanned parking lot charging system to solve the existing problem that due to the continuous change of the position between the vehicle and the camera, each character on the license plate is affected by external light during the recognition process and the recognition is inaccurate.

[0004] The vehicle vision recognition method for an unmanned parking lot charging system of the present invention adopts the following technical solutions:

[0005] It includes the following steps:

[0006] Collect the vehicle driving video and obtain the license plate area in each video frame of the video.

[0007] Divide the license plate area in each video frame into several character areas and assign character area index labels; according to the gray-scale distribution of the pixel points in the character area of the video frame, obtain the gray-scale gradation of the character area in the video frame; according to the gray-scale gradation of the character areas with the same index label in the video frame and the video frames in its previous local range, obtain the quality of the character area in the video frame.

[0008] Group the character areas with the same index label in all video frames into the same character area group; obtain the character recognition result and the confidence of the recognition result of each character area in the character area group, and combine the quality of the character area to obtain the credibility of each type of character area in the character area group and the primary recognition result of the character area group; according to the recognition result and the credibility of each type of character area in the character area group, obtain the error degree of the primary recognition result of the character area group to distinguish the unrecognized group and the recognized group, and obtain the characters corresponding to the recognized group.

[0009] Based on the distribution of characteristic points of motor vehicle license plate characters and skeleton pixel points, determine the structure labels of the characteristic points, and construct an unrecognized group recognition model therewith; based on the distribution of characteristic points in each character region of the unrecognized group, cluster to obtain characteristic point clusters, and according to the structure labels included in the characteristic point clusters and the distribution of the cluster centers, obtain the characters corresponding to the unrecognized group through the model.

[0010] Preferably, the method of dividing the license plate area in each video frame into several character areas and assigning character area index labels specifically includes:

[0011] For the license plate area in any video frame, use the Otsu method to divide the license plate area into a foreground part and a background part, and obtain the average channel values of all pixel points in the background part of the license plate area in the R, G, and B channels respectively, denoted as , , , according to , , Obtain the greenness degree of the background part of the license plate area, and its specific calculation formula is:

[0012]

[0013] In the formula, represents the greenness degree of the background part of the license plate area; represents the average channel value of all pixel points in the background part of the license plate area in the G channel; represents the average channel value of all pixel points in the background part of the license plate area in the R channel; represents the average channel value of all pixel points in the background part of the license plate area in the B channel; represents the sigmoid function;

[0014] Preset a greenness degree threshold , if is greater than or equal to , then the license plate area is the license plate area of a new energy vehicle, and divide the license plate area into 8 character areas according to the distribution of characters in the license plate area of the new energy vehicle, and at the same time assign index labels to each character area from left to right; when is less than , then the license plate area is the license plate area of an ordinary car, and divide the license plate area into 7 character areas according to the distribution of characters in the license plate area of the ordinary car, and at the same time assign index labels to each character area from left to right.

[0015] Preferably, the method of obtaining the gray-scale gradation of the character area in the video frame according to the gray-scale distribution of the pixel points in the character area of the video frame specifically includes:

[0016] For any character region within any video frame, the pixel point with the maximum gray value in the character region within the video frame is denoted as the reference pixel point. Based on the gray value difference and distance between each pixel point in the character region within the video frame and the reference pixel point, the gray gradation of the character region within the video frame is obtained. The specific calculation formula is as follows:

[0017]

[0018] In the formula, represents the gray gradation of the character region within the video frame; represents the number of pixel points in the character region within the video frame; represents the gray value of the th pixel point in the character region within the video frame; represents the gray value of the th pixel point in the character region within the video frame; represents the distance between the th pixel point and the reference pixel point in the character region within the video frame; represents the distance between the th pixel point and the reference pixel point in the character region within the video frame; represents the sign function.

[0019] Preferably, the method for obtaining the quality of the character region within the video frame based on the gray gradation of the character regions with the same index label in the video frame and its previous local range includes the following specific method:

[0020] Preset a local time range ; for any character region within any video frame, the video frames within the previous seconds of the video frame are used as the local video frames of the video frame. The character regions with the same index label as the character region within the video frame in the local video frames of the video frame are used as the corresponding character regions within the local video frames of the video frame; based on the gray gradation of the corresponding character regions within the local video frames of the video frame and combined with the gray gradation of the character region within the video frame, the quality of the character region within the video frame is obtained. The specific calculation formula is as follows:

[0021]

[0022] In the formula, represents the change in the gray gradation of the corresponding character region within the th local video frame of the video frame; indicating the gray-scale gradual change of the corresponding character area within the th local video frame of the video frame; indicating the gray-scale gradual change of the corresponding character area within the th local video frame of the video frame; indicating the gray-scale gradual change of the character area within the video frame; indicating the quality of the character area within the video frame; indicating the number of local video frames of the video frame; indicating the change in the gray-scale gradual change of the corresponding character area within the th local video frame of the video frame; indicating the sign function; indicating the absolute value function; indicating the sigmoid function.

[0023] Preferably, obtaining the character recognition result and the confidence level of the recognition result for each character area in the character area group, and combining the quality of the character area to obtain the credibility of each type of character area in the character area group and the primary recognition result of the character area group, the specific method included is:

[0024] For any character area group, using OCR technology to recognize all character areas in the character area group, obtaining the recognition result and the confidence level of the recognition result for each character area in the character area group; classifying the character areas with the same recognition result in the character area group as the same type of character area, and according to the recognition result and the confidence level of all character areas in each type of character area in the character area group, and combining the quality of all character areas in each type of character area in the character area group, obtaining the credibility of the character recognition result of each type of character area in the character area group, and its specific calculation formula is:

[0025]

[0026] In the formula, indicates the credibility of the character recognition result of the th type of character area in the character area group; indicates the number of character areas in the th type of character area in the character area group; indicates the confidence level of the recognition result of the th character area in the th type of character area in the character area group; indicates the quality of the th character area in the th type of character area in the character area group;

[0027] Obtain the confidence levels of the character recognition results for each type of character region in the character region group, and use the recognition result corresponding to the maximum confidence level as the primary recognition result of the character region group.

[0028] Preferably, the method for obtaining the error degree of the primary recognition result of the character region group based on the recognition results and confidence levels of each type of character region in the character region group includes the following specific method:

[0029] For any character region group, obtain the error degree of the primary recognition result of the character region group according to the recognition results and the confidence levels of the recognition results corresponding to all types of character regions in the character region group. The specific calculation formula is as follows:

[0030]

[0031] In the formula, represents the error degree of the primary recognition result of the character region group; represents the number of all types of character regions in the character region group; represents the proportion of the character region in the th type of character region in the character region group in the character region group; represents the th type of character region in the character region group; represents the logarithmic function with base 2; represents the sigmoid function.

[0032] Preferably, the method for distinguishing the unrecognized group and the recognized group and obtaining the characters corresponding to the recognized group includes the following specific method:

[0033] Preset an error degree threshold ; for any character region group, if the error degree of the primary recognition result of the character region group is greater than or equal to , record the character region group as the unrecognized group. If the error degree of the primary recognition result of the character region group is less than , record the character region group as the recognized group, and use the primary recognition result of the recognized group as the character corresponding to the recognized group.

[0034] Preferably, the method for constructing the unrecognized group recognition model includes the following specific method:

[0035] Obtain all the vehicle license plate characters. For any vehicle license plate character, use the SIFT algorithm to extract all the feature points of the character, use the Guo-Hall algorithm to obtain the skeleton of the character, and record the pixel points located on the skeleton as the skeleton pixel points;

[0036] For any feature point of the character, the pixel point in the skeleton of the character that is closest to the feature point is denoted as the reference point, and a -sized local window is constructed with the reference point as the center. All the skeleton pixel points within the local window are used as the local pixel points of the reference point, where the is the preset side length of the local window. If the number of skeleton pixel points in the eight-neighborhood of any local pixel point of the reference point is greater than or equal to 3, the structure label of the feature point is an intersection point; if the number of skeleton pixel points in the eight-neighborhood of all local pixel points of the reference point is less than 3, the structure label of the feature point is an end point; the structure label of the feature point is obtained;

[0037] Obtain the structure labels of all feature points in each motor vehicle license plate character. The structure labels of all feature points in each motor vehicle license plate character are constructed as the training samples of each motor vehicle license plate character, and the character corresponding to each training sample is used as the training label. All the training samples are input into the CNN model for training, where the loss function used is the cross-entropy loss function, and an unrecognized group recognition model is obtained.

[0038] Preferably, the method for obtaining the character corresponding to the unrecognized group specifically includes:

[0039] For any character region in any unrecognized group, obtain all the feature points and the structure labels of the feature points of the character region;

[0040] Take the last character region in the unrecognized group as the feature space, and use the Gaussian pyramid algorithm to map the feature points of all character regions in the unrecognized group into the feature space; the feature space of the unrecognized group is obtained;

[0041] Through the unrecognized group recognition model and the feature space of the unrecognized group, obtain the character corresponding to the unrecognized group.

[0042] Preferably, the method for obtaining the character corresponding to the unrecognized group through the unrecognized group recognition model and the feature space of the unrecognized group specifically includes:

[0043] For any character region in any unrecognized group, using the Euclidean distance between feature points in the feature space as the metric distance, cluster all the feature points in the feature space through the DBSCAN clustering algorithm to obtain several feature point clusters of the unrecognized group;

[0044] Denote the cluster center of any feature point cluster class in the unrecognized group as the target point, and use the structure label with the largest number of feature points in the feature point cluster class as the structure label of the target point, so as to obtain the structure labels of all target points in the unrecognized group. Input the structure labels of all target points in the unrecognized group into the unrecognized group recognition model to obtain the characters corresponding to the unrecognized group.

[0045] The beneficial effects of the technical solution of the present invention are as follows: By collecting and analyzing the degree of illumination influence on each character area in each video frame when the vehicle enters the unmanned parking lot, the quality of each character area in each video frame is evaluated, and a calculation weight is assigned to the subsequent recognition of the characters corresponding to each character area, so as to reduce the interference caused by illumination influence. Furthermore, according to the quality of each character area in each video frame, each character area in each video frame is recognized to obtain the primary recognition results of the character areas corresponding to the same character in different video frames. Further, according to the primary recognition results obtained from the character areas corresponding to the same character in different video frames, it is verified whether the primary recognition results are accurate, and several unrecognized groups, several recognized groups, and the characters corresponding to the recognized groups are obtained, where the primary recognition result of the recognized group is the character corresponding to the recognized group; while the primary recognition result of the unrecognized group is not necessarily the character corresponding to the unrecognized group.

[0046] Therefore, the unrecognized group recognition model is trained through the vehicle license plate characters; further, the feature points in all character areas in the unrecognized group are extracted and structure labels are assigned to the feature points. According to the structure labels and distribution positions of all feature points, several target points that can represent all feature points are obtained, and the structure labels of the target points are assigned according to the structure labels of the feature points corresponding to the target points. The structure labels and distribution positions of the target points are input into the unrecognized group recognition model, so as to accurately obtain the characters corresponding to the unrecognized group; finally, the vehicle is recognized during the vehicle driving process. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0048] Figure 1 It is a flowchart of the steps of the vehicle vision recognition method for the unmanned parking lot charging system of the present invention;

[0049] Figure 2 It is a schematic diagram of the entrance and exit of the unmanned parking lot. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following specifically describes, in conjunction with the accompanying drawings and preferred embodiments, the vehicle vision recognition method for an unmanned parking lot charging system according to the present invention, its specific implementation manners, structures, features, and effects as follows. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0052] The following specifically describes the specific solution of the vehicle vision recognition method for an unmanned parking lot charging system provided by the present invention in conjunction with the accompanying drawings.

[0053] Please refer to Figure 1 , which shows a flowchart of the steps of a vehicle vision recognition method for an unmanned parking lot charging system provided by an embodiment of the present invention. The method includes the following steps:

[0054] Step S001: Collect the vehicle driving video and obtain the license plate area in each video frame of the video.

[0055] It should be noted that due to the continuous change in the position between the vehicle and the camera, the recognition of each character in the license plate is inaccurate due to the influence of external light during the recognition process. Therefore, it is necessary to make the vehicle stop at the entrance and exit of the unmanned parking lot for a period of time to collect, analyze, and recognize the license plate information of the vehicle, resulting in low efficiency when the vehicle enters the unmanned parking lot. Therefore, this embodiment proposes a vehicle vision recognition method for an unmanned parking lot charging system. Specifically, by collecting the video of the vehicle during the driving process of entering the unmanned parking lot and analyzing the video during the driving process, the license plate of the vehicle can be recognized during the driving process without having to stop at the entrance and exit of the parking lot, thereby improving the efficiency of entering the unmanned parking lot.

[0056] Specifically, install a high-definition camera at the entrance and exit of the unmanned parking lot, and bury a vehicle detector coil underground at the entrance and exit of the unmanned parking lot. As Figure 2 shown, Figure 2 is a schematic diagram of the entrance and exit of the unmanned parking lot. The vehicle detector coil senses whether a vehicle enters the unmanned parking lot; when a vehicle is about to enter the unmanned parking lot, collect the driving video of the vehicle through the high-definition camera, and obtain the license plate area in each video frame of the driving video of the vehicle through semantic segmentation. Since semantic segmentation is a well-known prior art, it will not be elaborated in this embodiment.

[0057] Step S002: Divide the license plate area within each video frame into several character areas and assign index labels to the character areas; according to the gray-scale distribution of pixel points in the character areas within the video frame, obtain the gray-scale graduality of the character areas within the video frame; according to the video frame and the gray-scale graduality of the character areas with the same index labels in the video frames within its previous local range, obtain the quality of the character areas within the video frame.

[0058] It should be noted that, as a vehicle vision recognition method for an unmanned parking lot charging system, this embodiment specifically identifies the license plate in the vehicle by analyzing the video during the process of the vehicle driving into the unmanned parking lot; during the driving process of the vehicle, it may be affected by external light, and during the process of the vehicle driving into the unmanned parking lot, due to the continuous change of the position between the vehicle and the camera, the recognition of each character in the license plate is inaccurate due to the influence of external light during the recognition process; in order to accurately recognize each character in the license plate, it is necessary to obtain the quality of each character in the license plate in different video frames to reduce the influence of external light on the recognition of license plate characters.

[0059] It should be further noted that, in order to better evaluate the quality of each character in different video frames, it is necessary to divide the license plate area into several character areas, and each character area contains one character; since there are 7 characters in the license plate of ordinary cars and 8 characters in the license plate of new energy vehicles, it is necessary to judge the number of characters in the license plate before dividing the license plate area into several character areas; and since the background of the license plate of new energy vehicles is green while the background of the license plate of ordinary cars is not green, the type of license plate can be distinguished based on this and the license plate area can be divided into several character areas.

[0060] Preferably, in a specific embodiment of the present invention, for the license plate area within any video frame, use the Otsu method to divide the license plate area into a foreground part and a background part (the part with a larger area is the background part). Since the Otsu method is a well-known existing technology, it will not be elaborated in this embodiment; obtain the mean values of the channel values of all pixel points in the background part of the license plate area in the R, G, and B channels respectively, denoted as 、 、 , according to 、 、 obtain the green degree of the background part of the license plate area, and its specific calculation formula is:

[0061]

[0062] In the formula, represents the green degree of the background part of the license plate area; represents the mean value of the channel values of all pixel points in the background part of the license plate area in the G channel; represents the average channel value of all pixel points in the background part of the license plate area in the R channel; represents the average channel value of all pixel points in the background part of the license plate area in the B channel; represents the sigmoid function, which is used for normalization processing operations in this embodiment.

[0063] Furthermore, a green degree threshold is preset , the specific value of which can be set according to the actual situation, and there is no strict requirement in this embodiment. In this embodiment, is taken as an example for description. If is greater than or equal to , then the license plate area is the license plate area of a new energy vehicle, and the license plate area is divided into 8 character areas according to the distribution of characters in the license plate area of the new energy vehicle. At the same time, index labels are assigned to each character area from left to right; when is less than , then the license plate area is the license plate area of an ordinary car, and the license plate area is divided into 7 character areas according to the distribution of characters in the license plate area of the ordinary car. At the same time, index labels are assigned to each character area from left to right.

[0064] It should be noted that each character area of the video frame contains a complete character of the license plate; and when the character area is affected by light, there will be a region with a large gray value in the character area, and the gray values of the pixel points in this region will show a gradient feature of decreasing from the highest gray value to the surrounding. Therefore, the gray gradient of the character area in the video frame can be obtained to provide a theoretical basis for evaluating the quality of the character area in the video frame later.

[0065] Preferably, in a specific embodiment of the present invention, for any character area in any video frame, the pixel point with the largest gray value in the character area in the video frame is recorded as the reference pixel point. According to the gray difference and distance between each pixel point in the character area in the video frame and the reference pixel point, the gray gradient of the character area in the video frame is obtained. The specific calculation formula is:

[0066]

[0067] In the formula, represents the gray gradient of the character area in the video frame; represents the number of pixel points in the character area in the video frame; represents the gray value of the th pixel point in the character area in the video frame; represents the grayscale value of the th pixel in the character region within the video frame; represents the distance between the th pixel and the reference pixel in the character region within the video frame; represents the distance between the th pixel and the reference pixel in the character region within the video frame; represents the sign function.

[0068] It should be noted that the grayscale gradient of the character region means that the grayscale values of the pixels in the character region show a gradient feature of decreasing from the highest grayscale to the surrounding areas. The larger its value, the greater the influence of light on the character region, and the reference pixel is the pixel with the largest grayscale value in the character region; therefore, when the difference between the grayscale value of the pixel in the character region and the grayscale value of the reference pixel is larger, and at the same time the distance between the pixel in the character region and the reference pixel is farther, the character region has more gradient features. When has a positive value, it indicates that the grayscale value of the th pixel in the character region is greater than the grayscale value of the th pixel. At this time, if the character region has a gradient feature, the distance between the th pixel and the reference pixel should be less than the distance between the th pixel and the reference pixel; and has a positive value, which means that the distance between the th pixel and the reference pixel should be less than the distance between the th pixel and the reference pixel. Therefore the larger its value, the stronger the grayscale gradient of the character region.

[0069] It should be further noted that since when the vehicle enters the unmanned parking lot, the influence of light on the character region in the license plate has a certain variation law, and the variation law of the influence of light on the character region is as follows: as the vehicle travels, the influence of light on the character region in the license plate gradually rises to the maximum value and then gradually decreases, or the influence of light on the character region gradually decreases, and there will be no situation where the influence of light on the character region is large and small; and since the larger the value of the grayscale gradient of the character region, the more likely the character region is affected by light, the quality of the character region can be obtained by analyzing the change of the grayscale gradient of the character region within a local time.

[0070] Preferably, in a specific embodiment of the present invention, a local time range is preset, and the specific value of the can be set according to the actual situation by itself, and this embodiment does not make a rigid requirement. In this embodiment, it is taken as will be described by taking... as an example; for any character region within any video frame, the video frames within the previous seconds of the video frame are used as the local video frames of the video frame. The character region within the local video frames of the video frame that has the same index label as the character region within the video frame is used as the corresponding character region within the local video frames of the video frame; according to the gray-scale gradation of the corresponding character region within the local video frames of the video frame, combined with the gray-scale gradation of the character region within the video frame, the quality of the character region within the video frame is obtained. The specific calculation formula is as follows:

[0071]

[0072] In the formula, represents the change in the gray-scale gradation of the corresponding character region within the th local video frame of the video frame; represents the gray-scale gradation of the corresponding character region within the th local video frame of the video frame; represents the gray-scale gradation of the corresponding character region within the th local video frame of the video frame; represents the gray-scale gradation of the character region within the video frame; represents the quality of the character region within the video frame; represents the number of local video frames of the video frame; represents the change in the gray-scale gradation of the corresponding character region within the th local video frame of the video frame; represents the sign function; represents the absolute value function; represents the sigmoid function, which is used for normalization operations in this embodiment.

[0073] It should be noted that character regions with the same index label are the corresponding character regions of the same character in the license plate in different video frames; A positive value of indicates an increase in the influence of light on the character region, and a negative value of indicates a decrease in the influence of light on the character region; therefore, when is closer to 0, it means that the gray-scale gradation of the corresponding character region of the same character in different video frames within the local time range is more regular and more likely to be affected by light, that is, the quality of the character region is lower; and the greater the gray-scale gradation of the character region, the greater the degree of influence of light on the character region, and the lower the quality of the character region. Therefore,

[0074] Thus, the quality of the character regions within the video frames is obtained.

[0075] Step S003: Group the character regions with the same index label in all video frames into the same character region group; obtain the character recognition results and the confidence levels of the recognition results for each character region in the character region group, and combine with the quality of the character regions to obtain the credibility of each type of character region in the character region group and the primary recognition result of the character region group; according to the recognition results and credibility of each type of character region in the character region group, obtain the error degree of the primary recognition result of the character region group to distinguish the unrecognized group and the recognized group, and obtain the characters corresponding to the recognized group.

[0076] It should be noted that after obtaining the quality of each character region within all video frames through step S002, a calculation weight can be assigned to recognize each character region in each video frame according to the quality of each character region in each video frame, so as to avoid the interference caused by the character regions affected by light and improve the accuracy of character recognition.

[0077] Preferably, in a specific embodiment of the present invention, the character regions with the same index label in all video frames are grouped into the same character region group. For any character region group, use OCR (Optical Character Recognition) technology to recognize all character regions in the character region group to obtain the character recognition results and the confidence levels of the recognition results for each character region in the character region group. Since OCR technology is a well-known existing technology, it will not be elaborated in this embodiment; group the character regions with the same character recognition results in the character region group into the same type of character region. According to the character recognition results and the confidence levels of the recognition results for all character regions within each type of character region in the character region group, and combine with the quality of all character regions within each type of character region in the character region group, obtain the credibility of the character recognition results for each type of character region in the character region group. The specific calculation formula is:

[0078]

[0079] In the formula, represents the credibility of the character recognition result for the th type of character region in the character region group; represents the number of character regions within the th type of character region in the character region group; represents the confidence level of the recognition result of the th character region within the th type of character region in the character region group; represents the quality of the th character region within the th type of character region in the character region group.

[0080] It should be noted that the character regions in the character region group are the character regions of the same character in different video frames of the license plate; and the greater the confidence level of the character region recognition result and the greater the character quality, the less the recognition result is affected by light interference, and the more reliable the corresponding recognition result is. Therefore, the most reliable recognition result can be used as the primary recognition result of the character region group.

[0081] Specifically, for any character region group, obtain the confidence levels of the character recognition results of each type of character region in the character region group, and use the recognition result corresponding to the maximum confidence level of all types of character region character recognition results in the character region group as the primary recognition result of the character region group.

[0082] It should be noted that to ensure that the primary recognition result of the character region group is the character corresponding to the character region group, it is also necessary to further evaluate the accuracy of the primary recognition result. Based on the accuracy of the primary recognition result, the recognized character region group and the unrecognized character region group can be obtained.

[0083] Preferably, in a specific embodiment of the present invention, for any character region group, according to the recognition results corresponding to all types of character regions in the character region group and the confidence levels of the recognition results, obtain the error degree of the primary recognition result of the character region group. The specific calculation formula is as follows:

[0084]

[0085] In the formula, represents the error degree of the primary recognition result of the character region group; represents the number of all types of character regions in the character region group; represents the proportion of the character region in the th type of character region in the character region group; represents the th type of character region character recognition result confidence level in the character region group; represents the logarithmic function with base 2; represents the sigmoid function, which is used for normalization operation in this embodiment.

[0086] It should be noted that is the existing information entropy calculation formula, It represents the degree of confusion of all character region recognition results in the character region group. The greater the degree of confusion, the more types of recognition results obtained by OCR technology for recognizing the character region group, and the more likely the recognition results obtained by OCR technology are incorrect. However, when calculating the information entropy, the credibility of each recognition result is not considered. Therefore, when evaluating the accuracy of the recognition results through information entropy, a calculation weight is assigned to the recognition results based on the credibility of the recognition results to constrain the impact of untrustworthy recognition results, so as to accurately evaluate the error degree of the recognition results. That is, according to the error degree of the primary recognition results of the character region group, it can be judged whether the primary recognition results are the characters corresponding to the character region group.

[0087] Specifically, a threshold of the error degree is preset , the specific value of which can be set according to the actual situation by itself, and there is no rigid requirement in this embodiment. In this embodiment, it is described by taking as an example; for any character region group, if the error degree of the primary recognition results of the character region group is greater than or equal to , the character region group is recorded as an unrecognized group. If the error degree of the primary recognition results of the character region group is less than , the character region group is recorded as a recognized group, and the primary recognition results of the recognized group are used as the characters corresponding to the recognized group.

[0088] It should be noted that the primary recognition results of the recognized group are the characters corresponding to the recognized group; while the primary recognition results of the unrecognized group are not necessarily the characters corresponding to the unrecognized group.

[0089] So far, several unrecognized groups, several recognized groups and the characters corresponding to the recognized groups are obtained.

[0090] Step S004: According to the distribution of the feature points and the skeleton pixel points of the motor vehicle license plate characters, determine the structure labels of the feature points, and construct an unrecognized group recognition model based on this; based on the distribution of the feature points of each character region in the unrecognized group, cluster to obtain feature point clusters, and according to the structure labels and the distribution of the cluster centers included in the feature point clusters, obtain the characters corresponding to the unrecognized group through the model.

[0091] It should be noted that the character regions corresponding to the unrecognized groups obtained in step S003 are always affected by light during the process of the vehicle driving into the unmanned parking lot, resulting in that the primary recognition results of the unrecognized groups are not necessarily the characters corresponding to the unrecognized groups. Therefore, it is necessary to further construct a character recognition model for recognizing the unrecognized groups to recognize the characters corresponding to the unrecognized groups.

[0092] Specifically, all motor vehicle license plate characters (the motor vehicle license plate characters are the characters that may appear on the license plate) are obtained. For any motor vehicle license plate character, the Guo-Hall algorithm is used to obtain the skeleton of the character, and the pixel points located on the skeleton are recorded as skeleton pixel points. Since the Guo-Hall algorithm and the SIFT algorithm are well-known existing technologies, they will not be elaborated in this embodiment;

[0093] For any feature point of the character, the pixel point on the skeleton of the character that is closest to the feature point is recorded as the reference point, and a -sized local window is constructed with the reference point as the center. All the skeleton pixel points within the local window are used as the local pixel points of the reference point. The is the preset side length of the local window, The specific value of can be set according to the actual situation and is not strictly required in this embodiment. In this embodiment, is used as an example for description; If there are 3 or more skeleton pixel points in the eight-neighborhood of any local pixel point of the reference point, the structure label of the feature point is an intersection point; If there are no 3 or more skeleton pixel points in the eight-neighborhood of all the local pixel points of the reference point, the structure label of the feature point is an end point; The structure label of the feature point is obtained; For example;

[0094] Furthermore, all the feature points and the structure labels of the feature points in each character that may appear on the license plate are used. All the feature points in each character that may appear on the license plate are used as each training sample, and the character corresponding to each training sample is used as the training label. All the training samples are input into a CNN (Convolutional Neural Network) model for training. The loss function used is the cross-entropy loss function. Since the specific training process of the CNN model is a well-known existing technology, it will not be elaborated in this embodiment, and an unrecognized group recognition model is obtained.

[0095] It should be noted that the structure label is a custom feature of the character feature points in this embodiment, while the training label is the correct answer marked for the training samples during the training of the model.

[0096] It should be further noted that although the character regions corresponding to the unrecognized groups are always affected by light during the process of the vehicle driving into the unmanned parking lot, as the vehicle moves, the positions affected by light in different character regions corresponding to the unrecognized groups are different. For example, in the first character region corresponding to the unrecognized group, the left side is severely affected by light, but the right side is not severely affected. In the last character region corresponding to the unrecognized group, the right side is severely affected by light, but the left side is not severely affected. By extracting the feature points of all character regions in the unrecognized group, mapping the feature points of all character regions in the unrecognized group to the same space, and based on the structure labels and distribution positions of all feature points in the space, a number of target points that can represent all feature points are obtained. The structure labels of the target points are assigned according to the structure labels of the corresponding feature points, and the structure labels and distribution positions of the target points are input into the unrecognized group recognition model, so as to accurately obtain the characters corresponding to the unrecognized group; ultimately, the vehicle recognition is completed during the vehicle driving process.

[0097] Preferably, in a specific embodiment of the present invention, for any character region in any unrecognized group, all feature points of the character region and the structure labels of the feature points are obtained; the process of obtaining all feature points of the character region and the structure labels of the feature points is the same as that of obtaining the feature points and the structure labels of the characters that may appear on the license plate, so this embodiment will not be elaborated here;

[0098] Furthermore, taking the last character region in the unrecognized group as the feature space, using the Gaussian pyramid algorithm to map the feature points of all character regions in the unrecognized group to the feature space. Taking the Euclidean distance between feature points in the feature space as the metric distance, all feature points in the feature space are clustered by the DBSCAN clustering algorithm to obtain several feature point cluster classes of the unrecognized group. Since both the Gaussian pyramid algorithm and the DBSCAN clustering algorithm are well-known prior arts, they will not be elaborated in this embodiment;

[0099] For any feature point cluster class of the unrecognized group, the cluster center of the feature point cluster class is denoted as the target point, and the result label corresponding to the feature point with the most structure labels in the feature point cluster class is used as the structure label of the target point, obtaining the structure labels of all target points of the unrecognized group. The structure labels of all target points of the unrecognized group are input into the unrecognized group recognition model to obtain the characters corresponding to the unrecognized group.

[0100] It should be noted that, before recognizing the characters in the license plate, the present application quantifies the degree of influence of light on each video frame, evaluates the quality of each character region in each video frame as the calculation weight when recognizing characters, so as to reduce the interference caused by light influence; at the same time, it checks whether the recognition result is accurate. For the characters with inaccurate recognition, the recognition model is trained to further recognize them, so as to avoid the influence of light received during the vehicle driving process and achieve the recognition of the vehicle during the vehicle driving process.

[0101] So far, this embodiment is completed.

[0102] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A vehicle vision recognition method for an unattended parking lot charging system, characterized in that The method includes the following steps: Collect vehicle driving videos and obtain the license plate regions within each video frame in the videos; Divide the license plate regions within each video frame into several character regions and assign index labels to the character regions; according to the gray-scale distribution of the pixel points in the character regions within the video frame, obtain the gray-scale gradual change of the character regions within the video frame; according to the gray-scale gradual change of the character regions with the same index labels within the video frame and the video frames in its previous local range, obtain the quality of the character regions within the video frame; Group the character regions with the same index labels in all video frames into the same character region group; obtain the character recognition results and the confidence levels of the recognition results for each character region in the character region group, and combine with the quality of the character regions to obtain the credibility of each type of character region in the character region group and the primary recognition result of the character region group; according to the recognition results and the credibility of each type of character region in the character region group, obtain the error degree of the primary recognition result of the character region group to distinguish the unrecognized group and the recognized group, and obtain the characters corresponding to the recognized group; wherein, group the character regions with the same recognition results in the character region group into the same type of character regions; Determine the structure labels of the feature points according to the distribution of the feature points and the skeleton pixel points of the motor vehicle license plate characters, and construct an unrecognized group recognition model based on this; based on the distribution of the feature points in each character region in the unrecognized group, cluster to obtain feature point cluster classes, and according to the structure labels and the distribution of the cluster class centers included in the feature point cluster classes, obtain the characters corresponding to the unrecognized group through the model; The specific method for obtaining the quality of the character regions within the video frame according to the gray-scale gradual change of the character regions with the same index labels within the video frame and the video frames in its previous local range includes: Preset a local time range ; For any character region within any video frame, use the video frames within seconds before the said video frame as the local video frames of the said video frame, and use the character region within the local video frames of the said video frame that has the same character region index label as the said character region within the said video frame as the corresponding character region within the local video frames of the said video frame; According to the gray-scale gradual change property of the corresponding character region within the local video frames of the said video frame, and in combination with the gray-scale gradual change property of the said character region within the said video frame, obtain the quality of the said character region within the said video frame, and its specific calculation formula is: In the formula, represents the change in the gray-scale gradient of the corresponding character region within the th local video frame of the video frame; represents the gray-scale gradient of the corresponding character region within the th local video frame of the video frame; represents the gray-scale gradient of the corresponding character region within the th local video frame of the video frame; represents the gray-scale gradient of the character region within the video frame; represents the quality of the character region within the video frame; represents the number of local video frames of the video frame; represents the change in the gray-scale gradient of the corresponding character region within the th local video frame of the video frame; represents the sign function; represents the absolute value function; represents the sigmoid function.

2. The vehicle vision recognition method for the unmanned parking lot charging system according to claim 1, wherein, The specific method for dividing the license plate regions within each video frame into several character regions and assigning index labels to the character regions includes: For the license plate area within any video frame, the Otsu method is used to divide the license plate area into a foreground part and a background part, and the mean values of the channel values of all pixel points in the background part of the license plate area in the R, G, and B channels are respectively denoted as , , . According to , , the greenness degree of the background part of the license plate area is obtained, and its specific calculation formula is: In the formula, represents the green degree of the background part of the license plate area; represents the average channel value of all pixel points in the G channel of the background part of the license plate area; represents the average channel value of all pixel points in the R channel of the background part of the license plate area; represents the average channel value of all pixel points in the B channel of the background part of the license plate area; represents the sigmoid function; Preset a greenness threshold , if is greater than or equal to , then the license plate area is the license plate area of a new energy vehicle, and the license plate area is divided into 8 character areas according to the distribution of characters in the license plate area of the new energy vehicle, and index labels are assigned to each character area from left to right; When less than , the license plate area is the license plate area of a regular vehicle, and the license plate area is divided into seven character areas according to the distribution of characters in the license plate area of the regular vehicle. At the same time, index labels are assigned to each character area from left to right.

3. The vehicle vision recognition method for the unmanned parking lot charging system according to claim 1, wherein, The specific method for obtaining the gray-scale gradual change of the character regions within the video frame according to the gray-scale distribution of the pixel points in the character regions within the video frame includes: For any character region in any video frame, record the pixel point with the largest gray value in the character region within the video frame as the reference pixel point, and obtain the gray-scale gradual change of the character region within the video frame according to the gray-scale difference and the distance between each pixel point and the reference pixel point in the character region within the video frame. The specific calculation formula is: In the formula, represents the gray-scale gradation of the character area within the video frame; represents the number of pixel points in the character area within the video frame; represents the gray-scale value of the th pixel point in the character area within the video frame; gray-scale value of the th pixel point in the character area within the video frame; represents the distance between the th pixel point and the reference pixel point in the character area within the video frame; represents the distance between the th pixel point and the reference pixel point in the character area within the video frame; represents the sign function.

4. The vehicle vision recognition method for the unmanned parking lot charging system according to claim 1, characterized in that, The specific method for obtaining the character recognition results and the confidence levels of the recognition results for each character region in the character region group, and combining with the quality of the character regions to obtain the credibility of each type of character region in the character region group and the primary recognition result of the character region group includes: For any character region group, use OCR technology to recognize all character regions in the character region group, obtain the recognition result and the confidence level of the recognition result for each character region in the character region group; according to the recognition results and the confidence levels of all character regions in each type of character region in the character region group, and combining the quality of all character regions in each type of character region in the character region group, obtain the credibility of the character recognition result for each type of character region in the character region group. The specific calculation formula is as follows: In the formula, represents the confidence level of the character recognition result of the th character region in the character region group; represents the number of character regions in the th character region in the character region group; represents the confidence level of the recognition result of the th character region in the th character region in the character region group; represents the quality of the th character region in the th character region in the character region group; Obtain the credibility of the character recognition result for each type of character region in the character region group, and take the recognition result corresponding to the maximum credibility of the character recognition results of all types of character regions in the character region group as the primary recognition result of the character region group.

5. The vehicle vision recognition method for an unmanned parking lot charging system according to claim 4, characterized in that, The method for obtaining the error degree of the primary recognition result of the character region group according to the recognition results and credibility of each type of character region in the character region group includes the following specific methods: For any character region group, obtain the error degree of the primary recognition result of the character region group according to the recognition results and the credibility of the recognition results corresponding to all types of character regions in the character region group. The specific calculation formula is as follows: In the formula, represents the error degree of the primary recognition result of the character region group; represents the number of all types of character regions in the character region group; represents the proportion of the character region in the th type of character region in the character region group; represents the confidence level of the character recognition result of the th type of character region in the character region group; represents the logarithmic function with base 2; represents the sigmoid function.

6. The vehicle vision recognition method for the unmanned parking lot charging system according to claim 1, characterized in that The method for distinguishing the unrecognized group and the recognized group and obtaining the characters corresponding to the recognized group includes the following specific methods: Preset an error degree threshold ; For any character region group, if the error degree of the primary recognition result of the character region group is greater than or equal to , mark the character region group as an unrecognized group. If the error degree of the primary recognition result of the character region group is less than , mark the character region group as a recognized group, and use the primary recognition result of the recognized group as the character corresponding to the recognized group.

7. The vehicle vision recognition method for the unmanned parking lot charging system according to claim 1, characterized in that The method for constructing the unrecognized group recognition model includes the following specific methods: Obtain all motor vehicle license plate characters. For any motor vehicle license plate character, use the SIFT algorithm to extract all feature points of the character, use the Guo-Hall algorithm to obtain the skeleton of the character, and record the pixel points located on the skeleton as skeleton pixel points; For any feature point of the character, the pixel point in the skeleton of the character that is closest to the feature point is denoted as the reference point, and a -sized local window is constructed with the reference point as the center. All the skeleton pixel points within the local window are used as the local pixel points of the reference point. The is the preset side length of the local window. If the number of skeleton pixel points of any local pixel point of the reference point in the eight-neighborhood is greater than or equal to 3, the structure label of the feature point is an intersection point. If the number of skeleton pixel points of all local pixel points of the reference point in the eight-neighborhood is less than 3, the structure label of the feature point is an endpoint. The structure label of the feature point is obtained; Obtain the structure labels of all feature points in each motor vehicle license plate character, construct the structure labels of all feature points in each motor vehicle license plate character as the training samples of each motor vehicle license plate character, use the character corresponding to each training sample as the training label, and input all training samples into the CNN model for training. The loss function used is the cross-entropy loss function to obtain the unrecognized group recognition model.

8. The vehicle vision recognition method for the unmanned parking lot charging system according to claim 7, characterized in that, The method for obtaining the characters corresponding to the unrecognized group includes the following specific methods: For any character region in any unrecognized group, obtain all feature points and the structure labels of the feature points of the character region; Take the last character region in the unrecognized group as the feature space, and use the Gaussian pyramid algorithm to map the feature points of all character regions in the unrecognized group into the feature space; Obtain the feature space of the unrecognized group; Obtain the characters corresponding to the unrecognized group through the unrecognized group recognition model and the feature space of the unrecognized group.

9. The vehicle vision recognition method for the unmanned parking lot charging system according to claim 8, characterized in that, The method for obtaining the characters corresponding to the unrecognized group through the unrecognized group recognition model and the feature space of the unrecognized group includes the following specific methods: For any character region in any unrecognized group, use the Euclidean distance between feature points in the feature space as the metric distance, and perform clustering on all feature points in the feature space through the DBSCAN clustering algorithm to obtain several feature point clusters of the unrecognized group; Denote the cluster center of any feature point cluster class in the unrecognized group as the target point, and use the structure label with the largest number of feature points in the feature point cluster class as the structure label of the target point, so as to obtain the structure labels of all target points in the unrecognized group. Input the structure labels of all target points in the unrecognized group into the unrecognized group recognition model to obtain the corresponding character of the unrecognized group.

Images