Vehicle visual identification method for unmanned parking lot charging system

Abstract

The invention relates to the technical field of image recognition, in particular to a vehicle visual recognition method for an unmanned parking lot charging system, which comprises the following steps: acquiring the quality of a character region in a video frame according to the gray level distribution of pixel points in the character region in the video frame so as to endow a calculation weight for the character corresponding to each character region in subsequent recognition; identifying each character region under each video frame, acquiring a primary identification result of the character region corresponding to the same character under different video frames, and checking the accuracy of the primary identification result to obtain character regions which are identified accurately and inaccurately; and further training an unidentified group recognition model to recognize a character region with an inaccurate primary recognition result. According to the method, before the characters in the license plate are recognized, the influence degree of illumination on each video frame is quantified, and the quality of each character region in each video frame is evaluated as the calculation weight when the characters are recognized, so that interference caused by illumination influence is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to a vehicle vision recognition method for an unmanned parking lot charging system. Background Art

[0002] An unmanned parking lot charging system relies on computer vision and image processing technologies. When a vehicle enters an unmanned parking lot, it automatically detects the vehicle, locates the license plate, and recognizes the characters to obtain the parking duration of the vehicle in the unmanned parking lot, and accordingly charges the corresponding fees. However, in the traditional unmanned parking lot, when recognizing the characters on the license plate, due to the continuous change of the position between the vehicle and the camera, each character on the license plate is affected by external light during the recognition process and the recognition is inaccurate. As a result, the vehicle needs to stop at the entrance and exit of the unmanned parking lot for a period of time to collect and analyze the license plate information of the vehicle, leading to low efficiency when the vehicle enters the unmanned parking lot. Summary of the Invention

[0003] The present invention provides a vehicle vision recognition method for an unmanned parking lot charging system to solve the existing problem that due to the continuous change of the position between the vehicle and the camera, each character on the license plate is affected by external light during the recognition process and the recognition is inaccurate.

[0004] The vehicle vision recognition method for an unmanned parking lot charging system of the present invention adopts the following technical solutions: Including the following steps: Collect the vehicle driving video and obtain the license plate area in each video frame of the video; Divide the license plate area in each video frame into several character areas and assign character area index labels; according to the gray-scale distribution of the pixel points in the character area in the video frame, obtain the gray-scale gradual change of the character area in the video frame; according to the gray-scale gradual change of the character areas with the same index label in the video frame and the video frames in its previous local range, obtain the quality of the character area in the video frame; Group the character areas with the same index label in all video frames into the same character area group; obtain the character recognition result and the confidence level of the recognition result of each character area in the character area group, and combine with the quality of the character area to obtain the credibility of each type of character area in the character area group and the primary recognition result of the character area group; according to the recognition result and the credibility of each type of character area in the character area group, obtain the error degree of the primary recognition result of the character area group to distinguish the unrecognized group and the recognized group, and obtain the characters corresponding to the recognized group; According to the distribution of feature points and skeleton pixel points of motor vehicle license plate characters, determine the structure labels of the feature points, and construct an unrecognized group recognition model based on this; based on the distribution of feature points in each character area of the unrecognized group, cluster to obtain feature point clusters, and according to the structure labels and cluster center distributions included in the feature point clusters, obtain the characters corresponding to the unrecognized group through the model.

[0005] Preferably, the method of dividing the license plate area in each video frame into several character areas and assigning character area index labels specifically includes: For the license plate area in any video frame, use the Otsu method to divide the license plate area into a foreground part and a background part, and obtain the mean values of the channel values of all pixel points in the background part of the license plate area in the R, G, and B channels respectively, which are denoted as 、 、 , according to 、 、 obtain the green degree of the background part of the license plate area, and its specific calculation formula is: In the formula, represents the green degree of the background part of the license plate area; represents the mean value of the channel values of all pixel points in the G channel of the background part of the license plate area; represents the mean value of the channel values of all pixel points in the R channel of the background part of the license plate area; represents the mean value of the channel values of all pixel points in the B channel of the background part of the license plate area; represents the sigmoid function; Preset a green degree threshold , if is greater than or equal to , then the license plate area is the license plate area of a new energy vehicle, and the license plate area is divided into 8 character areas according to the distribution of characters in the license plate area of the new energy vehicle, and at the same time, index labels are assigned to each character area from left to right; when is less than , then the license plate area is the license plate area of an ordinary car, and the license plate area is divided into 7 character areas according to the distribution of characters in the license plate area of the ordinary car, and at the same time, index labels are assigned to each character area from left to right.

[0006] Preferably, the method of obtaining the gray-scale gradation of the character area in the video frame according to the gray-scale distribution of pixel points in the character area specifically includes: For any character region within any video frame, the pixel point with the maximum gray value in the character region within the video frame is denoted as the reference pixel point. According to the gray value difference and distance between each pixel point in the character region within the video frame and the reference pixel point, the gray gradation of the character region within the video frame is obtained. The specific calculation formula is as follows: In the formula, represents the gray gradation of the character region within the video frame; represents the number of pixel points in the character region within the video frame; represents the gray value of the th pixel point in the character region within the video frame; represents the gray value of the th pixel point in the character region within the video frame; represents the distance between the th pixel point and the reference pixel point in the character region within the video frame; represents the distance between the th pixel point and the reference pixel point in the character region within the video frame; represents the sign function.

[0007] Preferably, obtaining the quality of the character region within the video frame according to the gray gradation of the character regions with the same index label within the video frame and its previous local range includes the following specific method: Preset a local time range ; for any character region within any video frame, the video frames within the previous seconds of the video frame are used as the local video frames of the video frame. The character regions with the same index label as the character region within the video frame in the local video frames of the video frame are used as the corresponding character regions within the local video frames of the video frame; according to the gray gradation of the corresponding character regions within the local video frames of the video frame, combined with the gray gradation of the character region within the video frame, the quality of the character region within the video frame is obtained. The specific calculation formula is as follows: In the formula, represents the change in the gray gradation of the corresponding character region within the th local video frame of the video frame; represents the gray gradation of the corresponding character region within the th local video frame of the video frame; represents the The gray-scale gradual change of the corresponding character region within a local video frame; Indicates the gray-scale gradual change of the character region within the video frame; Indicates the quality of the character region within the video frame; Indicates the number of local video frames of the video frame; Indicates the Change in the gray-scale gradual change of the corresponding character region within the local video frame; Indicates the sign function; Indicates the absolute value function; Indicates the sigmoid function.

[0008] Preferably, obtaining the character recognition result and the confidence of the recognition result for each character region in the character region group, and combining the quality of the character region to obtain the credibility of each type of character region in the character region group and the primary recognition result of the character region group, including the specific method as follows: For any character region group, use OCR technology to recognize all character regions in the character region group to obtain the recognition result and the confidence of the recognition result for each character region in the character region group; classify the character regions with the same recognition result in the character region group as the same type of character region, and according to the recognition result and the confidence of the recognition result for all character regions within each type of character region in the character region group, and combining the quality of all character regions within each type of character region in the character region group, obtain the credibility of the character recognition result for each type of character region in the character region group, and its specific calculation formula is: In the formula, Indicates the credibility of the character recognition result for the th type of character region in the character region group; Indicates the number of character regions within the th type of character region in the character region group; Indicates the confidence of the recognition result of the th character region within the th type of character region in the character region group; Indicates the quality of the th character region within the th type of character region in the character region group; Obtain the credibility of the character recognition result for each type of character region in the character region group, and take the recognition result corresponding to the maximum credibility as the primary recognition result of the character region group.

[0009] Preferably, obtaining the error degree of the primary recognition result of the character region group according to the recognition result and confidence of each character region in the character region group includes the following specific method: For any character region group, obtain the error degree of the primary recognition result of the character region group according to the recognition results and the confidence levels of the recognition results corresponding to all types of character regions in the character region group. The specific calculation formula is: In the formula, represents the error degree of the primary recognition result of the character region group; represents the number of all types of character regions in the character region group; represents the proportion of the character regions in the th type of character region in the character region group; represents the confidence level of the character recognition result of the th type of character region in the character region group; represents the logarithmic function with base 2; represents the sigmoid function.

[0010] Preferably, the method for distinguishing the unrecognized group and the recognized group and obtaining the characters corresponding to the recognized group includes the following specific method: Preset an error degree threshold ; for any character region group, if the error degree of the primary recognition result of the character region group is greater than or equal to , record the character region group as the unrecognized group. If the error degree of the primary recognition result of the character region group is less than , record the character region group as the recognized group, and use the primary recognition result of the recognized group as the character corresponding to the recognized group.

[0011] Preferably, the method for constructing the unrecognized group recognition model includes the following specific method: Obtain all motor vehicle license plate characters. For any motor vehicle license plate character, use the SIFT algorithm to extract all feature points of the character, use the Guo-Hall algorithm to obtain the skeleton of the character, and record the pixel points on the skeleton as skeleton pixel points; For any feature point of the character, record the pixel point on the skeleton of the character that is closest to the feature point as the reference point, construct a -sized local window centered on the reference point, and use all the skeleton pixel points within the local window as the local pixel points of the reference point. The is the preset side length of the local window; if the number of skeleton pixel points of any local pixel point of the reference point in the eight-neighborhood is greater than or equal to 3, the structure label of the feature point is an intersection point; if the number of skeleton pixel points of all local pixel points of the reference point in the eight-neighborhood is less than 3, the structure label of the feature point is an end point; obtain the structure label of the feature point; Obtain the structure labels of all feature points in each motor vehicle license plate character, construct the structure labels of all feature points in each motor vehicle license plate character as the training samples of each motor vehicle license plate character, use the corresponding character of each training sample as the training label, and input all the training samples into the CNN model for training, where the loss function used is the cross-entropy loss function, and obtain the unrecognized group recognition model.

[0012] Preferably, the method for obtaining the character corresponding to the unrecognized group specifically includes: For any character region in any unrecognized group, obtain all the feature points and the structure labels of the feature points of the character region; Take the last character region in the unrecognized group as the feature space, and use the Gaussian pyramid algorithm to map the feature points of all character regions in the unrecognized group into the feature space; obtain the feature space of the unrecognized group; Through the unrecognized group recognition model and the feature space of the unrecognized group, obtain the character corresponding to the unrecognized group.

[0013] Preferably, the method for obtaining the character corresponding to the unrecognized group through the unrecognized group recognition model and the feature space of the unrecognized group specifically includes: For any character region in any unrecognized group, use the Euclidean distance between feature points in the feature space as the metric distance, and cluster all the feature points in the feature space through the DBSCAN clustering algorithm to obtain several feature point cluster classes of the unrecognized group; Denote the cluster center of any feature point cluster class of the unrecognized group as the target point, use the structure label with the largest number of feature points in the feature point cluster class as the structure label of the target point, obtain the structure labels of all target points of the unrecognized group, and input the structure labels of all target points of the unrecognized group into the unrecognized group recognition model to obtain the character corresponding to the unrecognized group.

[0014] The beneficial effects of the technical solution of the present invention are as follows: By collecting and analyzing the degree of illumination influence on each character area in each video frame when the vehicle enters the unmanned parking lot, the quality of each character area in each video frame is evaluated, and a calculation weight is assigned to the subsequent recognition of the characters corresponding to each character area, thereby reducing the interference caused by illumination influence; furthermore, according to the quality of each character area in each video frame, each character area in each video frame is recognized to obtain the primary recognition results of the character areas corresponding to the same character in different video frames. Further, according to the primary recognition results obtained from the character areas corresponding to the same character in different video frames, it is verified whether the primary recognition results are accurate, and several unrecognized groups, several recognized ones, and the characters corresponding to the recognized groups are obtained, where the primary recognition result of the recognized group is the character corresponding to the recognized group; while the primary recognition result of the unrecognized group is not necessarily the character corresponding to the unrecognized group.

[0015] Therefore, the unrecognized group recognition model is trained through the license plate characters of the motor vehicle; further, the feature points in all the character areas of the unrecognized group are extracted and structure labels are assigned to the feature points. According to the structure labels and distribution positions of all the feature points, several target points that can represent all the feature points are obtained, and the structure labels of the target points are assigned according to the structure labels of the corresponding feature points. The structure labels and distribution positions of the target points are input into the unrecognized group recognition model, so as to accurately obtain the characters corresponding to the unrecognized group; finally, the vehicle recognition is completed during the vehicle driving process. Description of the Drawings

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0017] Figure 1 It is the step flow chart of the vehicle vision recognition method for the unmanned parking lot charging system of the present invention; Figure 2 It is the schematic diagram of the entrance and exit of the unmanned parking lot. Detailed Embodiments

[0018] To further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following specifically describes, in conjunction with the accompanying drawings and preferred embodiments, the vehicle vision recognition method for an unmanned parking lot charging system according to the present invention, its specific implementation manner, structure, features, and effects, as follows. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0020] The following specifically describes, in conjunction with the accompanying drawings, the specific solution of the vehicle vision recognition method provided by the present invention for an unmanned parking lot charging system.

[0021] Please refer to Figure 1 , which shows the step flowchart of the vehicle vision recognition method for an unmanned parking lot charging system provided by an embodiment of the present invention. The method includes the following steps: Step S001: Collect the vehicle driving video and obtain the license plate area in each video frame of the video.

[0022] It should be noted that due to the continuous change in the position between the vehicle and the camera, the recognition of each character in the license plate is inaccurate due to the influence of external light during the recognition process. Therefore, it is necessary to make the vehicle stop at the entrance and exit of the unmanned parking lot for a period of time to collect and analyze the license plate information of the vehicle, resulting in low efficiency when the vehicle enters the unmanned parking lot. Therefore, this embodiment proposes a vehicle vision recognition method for an unmanned parking lot charging system. Specifically, by collecting the video of the vehicle during the driving process of entering the unmanned parking lot and analyzing the video during the driving process, the license plate of the vehicle can be recognized during the driving process, without having to stop at the entrance and exit of the parking lot, thereby improving the efficiency of entering the unmanned parking lot.

[0023] Specifically, install a high-definition camera at the entrance and exit of the unmanned parking lot, and bury a ground loop coil underground at the entrance and exit of the unmanned parking lot. As Figure 2 shown, Figure 2 is a schematic diagram of the entrance and exit of the unmanned parking lot. The ground loop coil senses whether a vehicle enters the unmanned parking lot; when a vehicle is about to enter the unmanned parking lot, collect the driving video of the vehicle through the high-definition camera, and obtain the license plate area in each video frame of the driving video of the vehicle through semantic segmentation. Since semantic segmentation is a well-known prior art, it will not be elaborated in this embodiment.

[0024] Step S002: Divide the license plate area within each video frame into several character areas and assign index labels to the character areas; according to the gray-scale distribution of the pixel points in the character areas within the video frame, obtain the gray-scale gradual change of the character areas within the video frame; according to the video frame and the gray-scale gradual change of the character areas with the same index labels in the video frames within its previous local range, obtain the quality of the character areas within the video frame.

[0025] It should be noted that, as a vehicle vision recognition method for an unmanned parking lot charging system, this embodiment specifically identifies the license plate in the vehicle by analyzing the video during the process of the vehicle driving into the unmanned parking lot; during the driving process of the vehicle, it may be affected by external light, and during the process of the vehicle driving into the unmanned parking lot, due to the continuous change of the position between the vehicle and the camera, the recognition of each character in the license plate is inaccurate due to the influence of external light; in order to accurately recognize each character in the license plate, it is necessary to obtain the quality of each character in the license plate in different video frames, so as to reduce the influence of external light on the recognition of license plate characters.

[0026] It should be further noted that, in order to better evaluate the quality of each character in different video frames, it is necessary to divide the license plate area into several character areas, and each character area contains one character; since there are 7 characters in the license plate of ordinary cars and 8 characters in the license plate of new energy vehicles, it is necessary to judge the number of characters in the license plate before dividing the license plate area into several character areas; and since the background of the license plate of new energy vehicles is green while the background of the license plate of ordinary cars is not green, the type of license plate can be distinguished based on this and the license plate area can be divided into several character areas.

[0027] Preferably, in a specific embodiment of the present invention, for the license plate area within any video frame, use the Otsu method to divide the license plate area into a foreground part and a background part (the part with a larger area is the background part). Since the Otsu method is a well-known existing technology, it will not be elaborated in this embodiment; obtain the average channel values of all pixel points in the background part of the license plate area in the R, G, and B channels respectively, denoted as 、 、 , according to 、 、 obtain the green degree of the background part of the license plate area, and its specific calculation formula is: In the formula, represents the green degree of the background part of the license plate area; represents the average channel value of all pixel points in the background part of the license plate area in the G channel; represents the average channel value of all pixel points in the background part of the license plate area in the R channel; represents the average channel value of all pixel points in the background part of the license plate area in the B channel; represents the sigmoid function, which is used for normalization processing operations in this embodiment.

[0028] Furthermore, a green degree threshold is preset , and the specific value of can be set according to the actual situation, and there is no strict requirement in this embodiment. In this embodiment, is taken as an example for description. If is greater than or equal to , then the license plate area is the license plate area of a new energy vehicle, and the license plate area is divided into 8 character areas according to the distribution of characters in the license plate area of the new energy vehicle. At the same time, index labels are assigned to each character area from left to right; when is less than , then the license plate area is the license plate area of an ordinary car, and the license plate area is divided into 7 character areas according to the distribution of characters in the license plate area of the ordinary car. At the same time, index labels are assigned to each character area from left to right. , then the license plate area is the license plate area of an ordinary car, and the license plate area is divided into 7 character areas according to the distribution of characters in the license plate area of the ordinary car. At the same time, index labels are assigned to each character area from left to right.

[0029] It should be noted that each character area of the video frame contains a complete character of the license plate; when the character area is affected by light, there will be a region with a large gray level in the character area, and the gray level values of the pixel points in this region will show a gradient feature of decreasing from the highest gray level to the surrounding areas. Therefore, the gray level gradient of the character area in the video frame can be obtained to provide a theoretical basis for subsequent evaluation of the quality of the character area in the video frame.

[0030] Preferably, in a specific embodiment of the present invention, for any character area in any video frame, the pixel point with the largest gray level value in the character area in the video frame is recorded as the reference pixel point, and the gray level gradient of the character area in the video frame is obtained according to the gray level difference and distance between each pixel point in the character area in the video frame and the reference pixel point. The specific calculation formula is: In the formula, represents the gray level gradient of the character area in the video frame; represents the number of pixel points in the character area in the video frame; represents the gray level value of the th pixel point in the character area in the video frame; represents the gray level value of the th pixel point in the character area in the video frame; represents the distance between the th pixel point in the character region within the video frame and the reference pixel point; represents the distance between the th pixel point in the character region within the video frame and the reference pixel point; represents the sign function.

[0031] It should be noted that the gray-scale gradation of the character region means that the gray-scale values of the pixel points in the character region will show a gradual change characteristic of decreasing from the highest gray-scale value to the surrounding areas. The larger the value, the greater the influence of light on the character region, and the reference pixel point is the pixel point with the largest gray-scale value in the character region; therefore, when the difference between the gray-scale value of the pixel point in the character region and the gray-scale value of the reference pixel point is larger, and at the same time, the distance between the pixel point in the character region and the reference pixel point is farther, the character region has more obvious gradation characteristics. When is positive, it indicates that the gray-scale value of the th pixel point in the character region is greater than the gray-scale value of the th pixel point. At this time, if the character region has a gradation characteristic, the distance between the th pixel point and the reference pixel point should be less than the distance between the th pixel point and the reference pixel point; and being positive means that the distance between the th pixel point and the reference pixel point should be less than the distance between the th pixel point and the reference pixel point. Therefore the larger the value of, the stronger the gray-scale gradation of the character region.

[0032] It should be further noted that when the vehicle drives into the unmanned parking lot, the influence of light on the character region in the license plate has a certain variation law. The variation law of the influence of light on the character region is: as the vehicle travels, the influence of light on the character region in the license plate gradually increases to the maximum value and then gradually decreases, or the influence of light on the character region gradually decreases, and there will be no situation where the influence of light on the character region is large and small; and because the larger the value of the gray-scale gradation of the character region, the more likely the character region is affected by light, so the quality of the character region can be obtained by analyzing the change of the gray-scale gradation of the character region within a local time.

[0033] Preferably, in a specific embodiment of the present invention, a local time range is preset. The specific value can be set according to the actual situation by itself, and this embodiment does not make a rigid requirement. In this embodiment, is taken as an example for description; for any character region in any video frame, the Video frames within seconds are used as local video frames of the video frames. The character regions in the local video frames of the video frames that have the same character region index label as the character region within the video frames are used as the corresponding character regions within the local video frames of the video frames. According to the gray-scale gradual change of the corresponding character regions within the local video frames of the video frames, combined with the gray-scale gradual change of the character regions within the video frames, the quality of the character regions within the video frames is obtained. The specific calculation formula is as follows: In the formula, represents the change in the gray-scale gradual change of the corresponding character region within the th local video frame of the video frame; represents the gray-scale gradual change of the corresponding character region within the th local video frame of the video frame; represents the gray-scale gradual change of the corresponding character region within the th local video frame of the video frame; represents the gray-scale gradual change of the character region within the video frame; represents the quality of the character region within the video frame; represents the number of local video frames of the video frame; represents the change in the gray-scale gradual change of the corresponding character region within the th local video frame of the video frame; represents the sign function; represents the absolute value function; represents the sigmoid function, which is used for normalization operation in this embodiment.

[0034] It should be noted that the character regions with the same index label are the character regions corresponding to the same character in the license plate in different video frames; A positive value of indicates an increase in the influence of light on the character region, and a negative value of indicates a decrease in the influence of light on the character region; therefore, when the value of gets closer to 0, it means that the gray-scale gradual change of the character regions corresponding to the same character in different video frames within the local time range is more regular and more likely to be affected by light, that is, the quality of the character region is lower; and the greater the gray-scale gradual change of the character region, the greater the degree of influence of light on the character region, and the lower the quality of the character region. Therefore, the smaller the value of , the lower the quality of the character region.

[0035] Thus, the quality of the character regions within the video frames is obtained.

[0036] Step S003: Group the character regions with the same index label in all video frames into the same character region group; obtain the character recognition results and the confidence levels of the recognition results for each character region in the character region group, and combine the quality of the character region to obtain the credibility of each type of character region in the character region group and the primary recognition result of the character region group; according to the recognition results and the credibility of each type of character region in the character region group, obtain the error degree of the primary recognition result of the character region group to distinguish the unrecognized group and the recognized group, and obtain the characters corresponding to the recognized group.

[0037] It should be noted that after obtaining the quality of each character region in all video frames through step S002, a calculation weight can be assigned to recognize each character region in each video frame according to the quality of each character region in each video frame, so as to avoid the interference caused by the character regions affected by light and improve the accuracy of character recognition.

[0038] Preferably, in a specific embodiment of the present invention, the character regions with the same index label in all video frames are grouped into the same character region group. For any character region group, the OCR (Optical Character Recognition) technology is used to recognize all the character regions in the character region group to obtain the character recognition results and the confidence levels of the recognition results for each character region in the character region group. Since the OCR technology is a well-known existing technology, it will not be elaborated in this embodiment; the character regions with the same character recognition results in the character region group are grouped into the same type of character region. According to the character recognition results and the confidence levels of the recognition results for all the character regions in each type of character region in the character region group, and combining the quality of all the character regions in each type of character region in the character region group, obtain the credibility of the character recognition results for each type of character region in the character region group. The specific calculation formula is: In the formula, represents the credibility of the character recognition result of the th type of character region in the character region group; represents the number of character regions in the th type of character region in the character region group; represents the confidence level of the recognition result of the th character region in the th type of character region in the character region group; represents the quality of the th character region in the th type of character region in the character region group.

[0039] It should be noted that the character regions in the character region group are the character regions of the same character in different video frames of the license plate; the greater the confidence of the character region recognition result and the greater the character quality, the less the recognition result is affected by light interference, and the more reliable the corresponding recognition result is. Therefore, the most reliable recognition result can be used as the primary recognition result of the character region group.

[0040] Specifically, for any character region group, obtain the confidence of the character recognition results of each character region in the character region group, and take the recognition result corresponding to the maximum confidence of the character recognition results of all character regions in the character region group as the primary recognition result of the character region group.

[0041] It should be noted that to ensure that the primary recognition result of the character region group is the character corresponding to the character region group, it is also necessary to further evaluate the accuracy of the primary recognition result. Based on the accuracy of the primary recognition result, the recognized character region group and the unrecognized character region group are obtained.

[0042] Preferably, in a specific embodiment of the present invention, for any character region group, according to the recognition results and the confidence of the recognition results corresponding to all character regions in the character region group, obtain the error degree of the primary recognition result of the character region group. The specific calculation formula is: In the formula, represents the error degree of the primary recognition result of the character region group; represents the number of all character regions in the character region group; represents the proportion of the character region in the th character region in the character region group; represents the th character region in the character region group, and represents the confidence of the character recognition result of the th character region;

[0043] It should be noted that is the existing information entropy calculation formula, It represents the degree of chaos of all character region recognition results in the character region group. The greater the degree of chaos, the more types of recognition results obtained by OCR technology for recognizing the character region group, and the more likely the recognition results obtained by OCR technology are incorrect. However, when calculating the information entropy, the credibility of each recognition result is not considered. Therefore, when evaluating the accuracy of the recognition results through information entropy, a calculation weight is assigned to the recognition results based on the credibility of the recognition results to constrain the impact of untrustworthy recognition results, so as to accurately evaluate the error degree of the recognition results. That is, according to the error degree of the primary recognition results of the character region group, it can be judged whether the primary recognition results are the characters corresponding to the character region group.

[0044] Specifically, a threshold of the error degree is preset , and the specific value of can be set according to the actual situation by itself, and this embodiment does not make a rigid requirement. In this embodiment, is taken as an example for description; for any character region group, if the error degree of the primary recognition result of the character region group is greater than or equal to , the character region group is recorded as an unrecognized group. If the error degree of the primary recognition result of the character region group is less than , the character region group is recorded as a recognized group, and the primary recognition result of the recognized group is used as the character corresponding to the recognized group.

[0045] It should be noted that the primary recognition result of the recognized group is the character corresponding to the recognized group; while the primary recognition result of the unrecognized group is not necessarily the character corresponding to the unrecognized group.

[0046] So far, several unrecognized groups, several recognized groups and the characters corresponding to the recognized groups are obtained.

[0047] Step S004: According to the distribution of the feature points and the skeleton pixel points of the motor vehicle license plate characters, determine the structure labels of the feature points, and construct an unrecognized group recognition model based on this; based on the distribution of the feature points of each character region in the unrecognized group, cluster to obtain feature point clusters, and according to the distribution of the structure labels and the cluster centers included in the feature point clusters, obtain the characters corresponding to the unrecognized group through the model.

[0048] It should be noted that the character regions corresponding to the unrecognized groups obtained in step S003 are always affected by light during the process of the vehicle driving into the unmanned parking lot, resulting in that the primary recognition results of the unrecognized groups are not necessarily the characters corresponding to the unrecognized groups. Therefore, it is necessary to further construct a character recognition model for recognizing the unrecognized groups to recognize the characters corresponding to the unrecognized groups.

[0049] Specifically, all vehicle license plate characters (the vehicle license plate characters are the characters that may appear on the license plate) are obtained. For any vehicle license plate character, the Guo-Hall algorithm is used to obtain the skeleton of the character, and the pixel points located on the skeleton are recorded as skeleton pixel points. Since the Guo-Hall algorithm and the SIFT algorithm are well-known existing technologies, they will not be elaborated in this embodiment; For any feature point of the character, the pixel point on the skeleton of the character that is closest to the feature point is recorded as the reference point, and a -sized local window is constructed with the reference point as the center. All the skeleton pixel points within the local window are used as the local pixel points of the reference point. The is the preset side length of the local window, The specific value of can be set according to the actual situation and is not strictly required in this embodiment. In this embodiment, it is described by taking as an example; if there are 3 or more skeleton pixel points in the eight-neighborhood of any local pixel point of the reference point, the structure label of the feature point is an intersection point; if there are no 3 or more skeleton pixel points in the eight-neighborhood of all the local pixel points of the reference point, the structure label of the feature point is an endpoint; the structure label of the feature point is obtained; Further, all the feature points and the structure labels of the feature points in each character that may appear on the license plate are used. All the feature points in each character that may appear on the license plate are used as each training sample, and the character corresponding to each training sample is used as the training label. All the training samples are input into a CNN (Convolutional Neural Network) model for training. The loss function used is the cross-entropy loss function. Since the specific training process of the CNN model is a well-known existing technology, it will not be elaborated in this embodiment, and an unrecognized group recognition model is obtained.

[0050] It should be noted that the structure label is a custom feature of the character feature points in this embodiment, and the training label is the correct answer marked for the training samples during the training of the model.

[0051] It should be further noted that although the character regions corresponding to the unrecognized groups are always affected by light during the process of the vehicle driving into the unmanned parking lot, as the vehicle moves, the positions affected by light in different character regions corresponding to the unrecognized groups are different. For example, in the first character region corresponding to the unrecognized group, the left side is severely affected by light, but the right side is not severely affected. In the last character region corresponding to the unrecognized group, the right side is severely affected by light, but the left side is not severely affected. By extracting the feature points of all character regions in the unrecognized group, the feature points of all character regions in the unrecognized group are mapped to the same space. According to the structure labels and distribution positions of all feature points in the space, several target points that can represent all feature points are obtained. The structure labels of the target points are assigned according to the structure labels of the corresponding feature points. The structure labels and distribution positions of the target points are input into the unrecognized group recognition model, so as to accurately obtain the characters corresponding to the unrecognized group; finally, the vehicle is recognized during the driving process of the vehicle.

[0052] Preferably, in a specific embodiment of the present invention, for any character region in any unrecognized group, all feature points of the character region and the structure labels of the feature points are obtained; the process of obtaining all feature points of the character region and the structure labels of the feature points is the same as that of obtaining the feature points and the structure labels of the characters that may appear on the license plate, so this embodiment will not be elaborated here; Furthermore, the last character region in the unrecognized group is used as the feature space. The Gaussian pyramid algorithm is used to map the feature points of all character regions in the unrecognized group to the feature space. The Euclidean distance between the feature points in the feature space is used as the metric distance. The DBSCAN clustering algorithm is used to cluster all feature points in the feature space to obtain several feature point cluster classes of the unrecognized group. Since the Gaussian pyramid algorithm and the DBSCAN clustering algorithm are both well-known existing technologies, they will not be elaborated in this embodiment; For any feature point cluster class of the unrecognized group, the cluster center of the feature point cluster class is denoted as the target point, and the result label corresponding to the feature point with the most structure labels in the feature point cluster class is used as the structure label of the target point, so as to obtain the structure labels of all target points of the unrecognized group. The structure labels of all target points of the unrecognized group are input into the unrecognized group recognition model to obtain the characters corresponding to the unrecognized group.

[0053] It should be noted that, before recognizing the characters in the license plate, the present application quantifies the degree of influence of illumination on each video frame, evaluates the quality of each character region in each video frame as the calculation weight when recognizing characters, so as to reduce the interference caused by illumination influence; at the same time, it checks whether the recognition result is accurate. For the characters with inaccurate recognition, the recognition model is trained to further recognize them, so as to avoid the influence of illumination received during the vehicle driving process and realize the recognition of the vehicle during the vehicle driving process.

[0054] So far, this embodiment is completed.

[0055] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A vehicle visual recognition method for an unmanned parking lot charging system, characterized in that: The method comprises the following steps: Collect vehicle driving videos and obtain the license plate area in each video frame in the video; The license plate area in each video frame is divided into several character areas and index labels are assigned to the character areas; the grayscale gradient of the character area in the video frame is obtained according to the grayscale distribution of the pixels in the character area in the video frame; the quality of the character area in the video frame is obtained according to the grayscale gradient of the character area with the same index label in the video frame and the previous local range; Classify the character regions with the same index label in all video frames into the same character region group; obtain the character recognition result and the confidence of the recognition result of each character region in the character region group, and obtain the credibility of each character region in the character region group and the primary recognition result of the character region group in combination with the quality of the character region; obtain the error degree of the primary recognition result of the character region group based on the recognition result and credibility of each character region in the character region group to distinguish between the unrecognized group and the recognized group, and obtain the characters corresponding to the recognized group; According to the distribution of feature points and skeleton pixels of the motor vehicle license plate characters, the structural labels of the feature points are determined, and an unrecognized group recognition model is constructed based on this. Based on the distribution of feature points in each character area in the unrecognized group, feature point clusters are obtained by clustering, and according to the structural labels included in the feature point clusters and the distribution of cluster centers, the characters corresponding to the unrecognized group are obtained through the model.

2. The vehicle visual recognition method for an unmanned parking lot charging system according to claim 1 is characterized in that: The specific method of dividing the license plate area in each video frame into a plurality of character areas and assigning character area index labels includes: For the license plate area in any video frame, the license plate area is divided into a foreground part and a background part using the Otsu method, and the channel value means of all pixels in the background part of the license plate area under the R, G, and B channels are obtained and recorded as , , ,according to , , The green degree of the background part of the license plate area is obtained, and the specific calculation formula is: In the formula, Indicates the green degree of the background part of the license plate area; represents the mean value of the channel values ​​of all pixels in the background part of the license plate area under the G channel; represents the mean value of the channel values ​​of all pixels in the background part of the license plate area under the R channel; represents the channel value mean of all pixels in the background part of the license plate area under the B channel; Represents the sigmoid function; Preset a green level threshold ,like Greater than or equal to , the license plate area is the license plate area of ​​the new energy vehicle, and the license plate area is divided into 8 character areas according to the distribution of characters in the license plate area of ​​the new energy vehicle, and an index label is assigned to each character area from left to right; when Less than , then the license plate area is the license plate area of ​​an ordinary car, and the license plate area is divided into 7 character areas according to the distribution of characters in the license plate area of ​​an ordinary car, and an index label is assigned to each character area from left to right.

3. The vehicle visual recognition method for an unmanned parking lot charging system according to claim 1 is characterized in that: The specific method of obtaining the grayscale gradient of the character area in the video frame according to the grayscale distribution of the pixels in the character area in the video frame is as follows: For any character area in any video frame, the pixel with the largest grayscale value in the character area in the video frame is recorded as the reference pixel. According to the grayscale difference and distance between each pixel in the character area in the video frame and the reference pixel, the grayscale gradient of the character area in the video frame is obtained. The specific calculation formula is: In the formula, Indicates the grayscale gradient of the character area in the video frame; Represents the number of pixels in the character area in the video frame; Indicates the first The gray value of each pixel; Indicates the first The gray value of each pixel; Indicates the first The distance between a pixel and a reference pixel; Indicates the first The distance between a pixel and a reference pixel; Represents the sign function.

4. The vehicle visual recognition method for an unmanned parking lot charging system according to claim 1 is characterized in that: The method of obtaining the quality of the character region in the video frame according to the grayscale gradient of the character region with the same index label in the video frame and the previous local range thereof includes the following specific methods: Preset a local time range ; For any character area in any video frame, the front of the video frame A video frame within seconds is used as a local video frame of the video frame, and a character region in the local video frame of the video frame with the same index label as the character region in the video frame is used as the corresponding character region in the local video frame of the video frame; according to the grayscale gradient of the corresponding character region in the local video frame of the video frame, combined with the grayscale gradient of the character region in the video frame, the quality of the character region in the video frame is obtained, and the specific calculation formula is: In the formula, Indicates the first The grayscale gradient change of the corresponding character area in a local video frame; Indicates the first The grayscale gradient of the corresponding character area in a local video frame; Indicates the first The grayscale gradient of the corresponding character area in a local video frame; Indicates the grayscale gradient of the character area in the video frame; Indicates the quality of the character area in the video frame; A number of local video frames representing the video frame; Indicates the first The grayscale gradient change of the corresponding character area in a local video frame; represents the sign-taking function; It represents the absolute value function; Represents the sigmoid function.

5. The vehicle visual recognition method for an unmanned parking lot charging system according to claim 1 is characterized in that: The method of obtaining the character recognition result and the confidence of the recognition result of each character region in the character region group, and obtaining the credibility of each character region in the character region group and the primary recognition result of the character region group in combination with the quality of the character region, includes the following specific methods: For any character area group, all character areas in the character area group are identified using OCR technology to obtain the recognition result of each character area in the character area group and the confidence of the recognition result; the character areas with the same recognition result in the character area group are classified as the same type of character areas, and the confidence of the character recognition result of each character area in the character area group is obtained according to the recognition results of all character areas in each character area in the character area group and the confidence of the recognition result, combined with the quality of all character areas in each character area in the character area group, and the specific calculation formula is: In the formula, Indicates the first character in the character area group The credibility of the character recognition results in the character area; Indicates the first character in the character area group The number of character regions in the character region; Indicates the first character in the character area group In the character area The confidence level of the character region recognition result; Indicates the first character in the character area group In the character area The quality of the character area; The credibility of the character recognition results of each character region in the character region group is obtained, and the recognition result corresponding to the maximum credibility of the character recognition results of all the character regions in the character region group is used as the primary recognition result of the character region group.

6. The vehicle visual recognition method for an unmanned parking lot charging system according to claim 5 is characterized in that: The specific method of obtaining the error degree of the primary recognition result of the character region group according to the recognition result and credibility of each character region in the character region group includes: For any character region group, the error degree of the primary recognition result of the character region group is obtained according to the recognition results corresponding to all the character regions in the character region group and the credibility of the recognition results. The specific calculation formula is: In the formula, Indicates the error degree of the primary recognition result of the character region group; Indicates the number of all types of character regions in the character region group; Indicates the first character in the character area group a proportion of character regions in the character region group; Indicates the first character in the character area group The credibility of the character recognition results in the character area; represents the logarithmic function with base 2; Represents the sigmoid function.

7. The vehicle visual recognition method for an unmanned parking lot charging system according to claim 1, characterized in that: The specific method for distinguishing the unrecognized group from the recognized group and obtaining the characters corresponding to the recognized group includes: Preset an error threshold For any character region group, if the error degree of the primary recognition result of the character region group is greater than or equal to , the character region group is recorded as an unrecognized group. If the error degree of the primary recognition result of the character region group is less than , the character area group is recorded as a recognized group, and the primary recognition result of the recognized group is used as the character corresponding to the recognized group.

8. The vehicle visual recognition method for an unmanned parking lot charging system according to claim 1, characterized in that: The specific method of constructing the unrecognized group recognition model includes: Obtain all the characters of the motor vehicle license plate, for any character of the motor vehicle license plate, use the SIFT algorithm to extract all the feature points of the character, use the Guo-Hall algorithm to obtain the skeleton of the character, and record the pixel points located on the skeleton as skeleton pixels; For any feature point of the character, the pixel point in the skeleton of the character that is closest to the feature point is recorded as the reference point, and a A local window of size is formed, and all skeleton pixels in the local window are used as local pixels of reference points. is the preset local window side length; if the number of skeleton pixels of any local pixel point of the reference point in the eight-neighborhood is greater than or equal to 3, the structural label of the feature point is the intersection point; if the number of skeleton pixels of all local pixels of the reference point in the eight-neighborhood is less than 3, the structural label of the feature point is the endpoint; the structural label of the feature point is obtained; The structural labels of all feature points in each motor vehicle license plate character are obtained, and the structural labels of all feature points in each motor vehicle license plate character are constructed into training samples of each motor vehicle license plate character. The character corresponding to each training sample is used as a training label, and all training samples are input into the CNN model for training, wherein the loss function used is the cross entropy loss function, and an unrecognized group recognition model is obtained.

9. The vehicle visual recognition method for an unmanned parking lot charging system according to claim 8, characterized in that: The specific method of obtaining the characters corresponding to the unrecognized group includes: For any character region in any unrecognized group, obtain all feature points and structural labels of the feature points in the character region; The last character region in the unrecognized group is used as a feature space, and the feature points of all the character regions in the unrecognized group are mapped to the feature space using a Gaussian pyramid algorithm; Obtaining a feature space of the unidentified group; The characters corresponding to the unrecognized group are obtained through the unrecognized group recognition model and the feature space of the unrecognized group.

10. The vehicle visual recognition method for an unmanned parking lot charging system according to claim 9, characterized in that: The method of obtaining characters corresponding to the unrecognized group through the unrecognized group recognition model and the feature space of the unrecognized group includes: For any character region in any unrecognized group, the Euclidean distance between feature points in the feature space is used as the metric distance, and all feature points in the feature space are clustered by the DBSCAN clustering algorithm to obtain several feature point clusters of the unrecognized group; The cluster center of any feature point cluster of the unrecognized group is recorded as the target point, and the structural label with the largest number of feature points in the feature point cluster is used as the structural label of the target point to obtain the structural labels of all target points of the unrecognized group. The structural labels of all target points of the unrecognized group are input into the unrecognized group recognition model to obtain the characters corresponding to the unrecognized group.

Images