License plate recognition method and device, electronic equipment and readable storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and provides a license plate recognition method and device, electronic equipment and a readable storage medium, and the method comprises the steps: obtaining a current video frame containing a target vehicle; inputting the current video frame into a license plate recognition model, and obtaining a current recognition result output by the license plate recognition model; the current recognition result is recognized license plate characters and recognition confidence of each license plate character; and updating a pre-stored historical identification result based on the current identification result to determine the license plate number of the target vehicle. After the recognition result of the current video frame is obtained, whether the recognition result is superior to the historical optimal recognition result or not is judged by combining the number of the recognized characters and the confidence coefficient of each character, the historical optimal recognition result is iteratively updated, and license plate recognition is performed according to the updated recognition result. And the license plate recognition precision is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of license plate recognition, and in particular to a license plate recognition method, device, electronic equipment and readable storage medium. Background Art

[0002] In places where people gather, such as residential communities, corporate parks, and shopping mall basements, it is usually necessary to manage the entry and exit of motor vehicles. By installing security cameras at the entrances and exits of the venues, using artificial intelligence-related technologies, the images of vehicles entering and leaving are automatically captured to extract key information, and finally this key information and the captured vehicle images are uploaded to the central control storage. This key information may include license plate number, vehicle color, vehicle type, and vehicle entry and exit information, but the license plate number is the most critical information because it can uniquely index the target vehicle.

[0003] Existing license plate recognition methods are generally based on template matching, manually designed feature extraction and classifiers, etc., to locate the license plate area in the vehicle image, then perform character segmentation, and then recognize the segmented characters one by one, convert them into corresponding text or numbers, and then splice them into the final recognized license plate number for direct output.

[0004] At present, in license plate recognition, only a randomly extracted vehicle image is analyzed, and in the analysis process, the characters are recognized one by one directly according to the characteristics of the character segmentation area. There are defects such as low detection accuracy and high probability of recognition error. Summary of the invention

[0005] The embodiments of the present invention provide a license plate recognition method, device, electronic device and readable storage medium to solve the defects of low accuracy and high error probability of license plate recognition results.

[0006] In a first aspect, the present invention provides a license plate recognition method, comprising:

[0007] Get the current video frame containing the target vehicle;

[0008] Input the current video frame into the license plate recognition model to obtain a current recognition result output by the license plate recognition model; the current recognition result is the recognized license plate characters and the recognition confidence of each license plate character;

[0009] The pre-stored historical recognition results are updated based on the current recognition results to determine the license plate number of the target vehicle.

[0010] According to a license plate recognition method provided by the present invention, the updating of the pre-stored historical recognition result based on the current recognition result comprises:

[0011] Comparing the current recognition result with the historical recognition result from two dimensions: the number of license plate characters and the recognition confidence of each license plate character;

[0012] The historical recognition result is updated according to the comparison result.

[0013] According to a license plate recognition method provided by the present invention, the current recognition result is compared with the historical recognition result from two dimensions: the number of license plate characters and the recognition confidence of each license plate character, including:

[0014] Obtaining a first comparison result of a first quantity and a second quantity, wherein the first quantity is the quantity of license plate characters included in the current recognition result, and the second quantity is the quantity of license plate characters included in the historical recognition result;

[0015] Obtaining a second comparison result between the first number and a preset character number threshold range;

[0016] Determine the current lowest confidence and the current confidence average among the recognition confidences of all license plate characters included in the current recognition result, and the historical lowest confidence and the historical confidence average among the recognition confidences of all license plate characters included in the historical recognition results;

[0017] Obtaining a third comparison result between the current lowest confidence and the historical lowest confidence, and a fourth comparison result between the current confidence mean and the historical confidence mean;

[0018] The comparison result is determined based on the first comparison result, the second comparison result, the third comparison result, and the fourth comparison result.

[0019] According to a license plate recognition method provided by the present invention, when the first comparison result is that the first number is less than the second number, determining the comparison result based on the first comparison result, the second comparison result, the third comparison result and the fourth comparison result includes:

[0020] If the second comparison result is that the first number is within the character number threshold range, and the current minimum confidence is greater than the first confidence threshold, then determining that the comparison result is that the current recognition result is better than the historical recognition result;

[0021] Otherwise, it is determined that the comparison result is that the current recognition result is not better than the historical recognition result.

[0022] According to a license plate recognition method provided by the present invention, when the first comparison result is that the first number is greater than or equal to the second number, determining the comparison result based on the first comparison result, the second comparison result, the third comparison result and the fourth comparison result includes:

[0023] If the second comparison result is that the first number is within the character number threshold range, the third comparison result is that the current lowest confidence is greater than the historical lowest confidence, and the fourth comparison result is that the current confidence mean is greater than the historical confidence mean, then the comparison result is determined to be that the current recognition result is better than the historical recognition result;

[0024] Otherwise, it is determined that the comparison result is that the current recognition result is not better than the historical recognition result.

[0025] According to a license plate recognition method provided by the present invention, the updating of the historical recognition result according to the comparison result includes:

[0026] When it is determined that the comparison result shows that the current recognition result is better than the historical recognition result, replacing the historical recognition result with the current recognition result;

[0027] Otherwise, the historical recognition result remains unchanged.

[0028] According to a license plate recognition method provided by the present invention, after replacing the historical recognition result with the current recognition result, determining the license plate number of the target vehicle includes:

[0029] In the case where it is determined that the current recognition result meets the preset upload condition, determining the license plate number of the target vehicle according to all license plate characters included in the current recognition result;

[0030] The preset upload condition is that the current minimum confidence level is greater than a second confidence level threshold, and the second confidence level threshold is greater than the first confidence level threshold.

[0031] According to a license plate recognition method provided by the present invention, when it is determined that the current recognition result does not meet the preset upload condition, the next video frame is re-acquired;

[0032] The next video frame is set as the current video frame, and the steps of inputting the current video frame into the license plate recognition model to determining the license plate number of the target vehicle based on the updated recognition result are iteratively performed until it is determined that the recognition result meets the preset upload condition or the next video frame is the last video frame;

[0033] The license plate number of the target vehicle is determined based on all license plate characters included in the recognition result.

[0034] According to a license plate recognition method provided by the present invention, when it is determined that the next video frame is the last video frame, the method further includes:

[0035] Inputting the next video frame into the license plate recognition model to obtain a final recognition result output by the license plate recognition model;

[0036] Using the final recognition result to update the historical recognition result;

[0037] If the lowest confidence of all the license plate characters included in the updated recognition result is less than or equal to the second confidence threshold and greater than the third confidence threshold, determining the license plate number of the target vehicle based on all the license plate characters included in the updated recognition result;

[0038] If the lowest confidence of all license plate characters included in the updated recognition result is less than or equal to the third confidence threshold, determining the license plate number of the target vehicle based on the best-scoring image;

[0039] The third confidence threshold is greater than the first confidence threshold; the best-scoring image is a video frame with the highest QoE score extracted from all video frames containing the target vehicle.

[0040] According to a license plate recognition method provided by the present invention, the best scoring image is extracted based on the following steps:

[0041] Inputting the current video frame into a QoE scoring model to obtain a current QoE score output by the QoE scoring model;

[0042] If the current QoE score is greater than the QoE score of the best-scoring image, the best-scoring image is replaced with the current video frame as a new best-scoring image;

[0043] Otherwise, keep the best-scoring image unchanged;

[0044] The QoE scoring model is trained based on image samples and the QoE label of each image sample.

[0045] According to a license plate recognition method provided by the present invention,

[0046] The license plate recognition model includes a target detection model and a license plate reading model. The current video frame is input into the license plate recognition model to obtain the current recognition result output by the license plate recognition model, including:

[0047] Inputting the current video frame into the target detection model to mark the vehicle detection frame coordinates and the license plate detection frame coordinates in the current video frame;

[0048] Cropping the current video frame based on the license plate detection frame coordinates to obtain a license plate image;

[0049] The license plate image is input into the license plate reading model to obtain the current recognition result output by the license plate reading model.

[0050] In a second aspect, the present invention provides a license plate recognition device, comprising:

[0051] An image acquisition unit, used for acquiring a current video frame containing a target vehicle;

[0052] A license plate recognition unit, used to input the current video frame into a license plate recognition model, and obtain a current recognition result output by the license plate recognition model; the current recognition result is the recognized license plate characters and the recognition confidence of each of the license plate characters;

[0053] The license plate determination unit updates the pre-stored historical recognition results based on the current recognition results to determine the license plate number of the target vehicle.

[0054] In a third aspect, the present invention provides an electronic device, comprising a processor and a memory storing a computer program, wherein the processor implements the license plate recognition method described in the first aspect when executing the program.

[0055] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the license plate recognition method described in any one of the above is implemented. .

[0056] The license plate recognition method, device, electronic device and readable storage medium provided by the embodiments of the present invention, after obtaining the recognition result of the current video frame, combine the two dimensions of the number of recognized characters and the confidence of each character to judge whether it is better than the best recognition result in history, so as to iteratively update the best recognition result in history, and perform license plate recognition according to the updated recognition result, thereby effectively improving the accuracy of license plate recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 It is one of the flow charts of the license plate recognition method provided by the present invention;

[0059] Figure 2 This is the second flow chart of the license plate recognition method provided by the present invention;

[0060] Figure 3 It is a structural schematic diagram of the license plate recognition device provided by the present invention;

[0061] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] It should be noted that in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the presence of other identical elements in the process, method, article or device including the elements. The orientation or positional relationship indicated by the terms "upper", "lower", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0064] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0065] Combine the following Figure 1-Figure 4The license plate recognition method and device provided by the embodiments of the present invention are described.

[0066] Figure 1 It is one of the flow charts of the license plate recognition method provided by the present invention, such as Figure 1 As shown, including but not limited to the following steps:

[0067] Step 101: Obtain a current video frame containing a target vehicle.

[0068] Select appropriate video acquisition equipment, such as surveillance cameras, driving recorders, etc., and collect real-time video streams by pointing the camera of the video acquisition equipment at the vehicle entrance and exit.

[0069] The captured video stream can be preprocessed as necessary, such as removing noise, adjusting brightness and contrast, etc.

[0070] Computer vision technology can be used to perform target detection on the preprocessed video frames to determine the portion of the video frames containing the target vehicle. The target detection method may include, but is not limited to: object detection algorithms based on deep learning (such as YOLO, Faster R-CNN, etc.), background subtraction based on traditional image processing, frame difference method, etc., which are not specifically limited in the present invention.

[0071] Furthermore, according to the target detection result, the video frame containing the target vehicle can be clipped out and output as image data in a specified format as the current video frame.

[0072] Step 102: Input the current video frame into a license plate recognition model to obtain a current recognition result output by the license plate recognition model.

[0073] The current recognition result is the recognized license plate characters and the recognition confidence of each license plate character.

[0074] First, a suitable license plate recognition model is selected and pre-trained using a historical sample set, wherein each sample in the historical sample set includes a historical video frame and a recognition result label corresponding to the historical video frame.

[0075] The current video frame may be preprocessed, including but not limited to image resizing, color space conversion, denoising, etc., to improve the accuracy of subsequent license plate number recognition.

[0076] Furthermore, a target detection method or an image processing method may be used to locate the license plate area in the current video frame. The target detection method may be a network model such as YOLO or Faster R-CNN, and the image processing method may be an edge detection method or a morphological operation method.

[0077] The located license plate area is cropped out from the current video frame and input into the trained license plate recognition model for recognition. The license plate recognition model recognizes the license plate characters contained in the license plate area to obtain a recognition result.

[0078] The format of the recognition result may be in text form, and the format of each line in the text content is a license plate character + the recognition confidence of the license plate character.

[0079] For example, if the detected license plate area contains 7 license plate characters, the corresponding recognition results are shown in Table 1:

[0080] Table 1. Examples of recognition results

[0081] serial number License Plate Characters Recognition confidence 1 Guangdong 93％ 2 A 93％ 3 1 95％ 4 2 99％ 5 T 93％ 6 0 97％ 7 2 91％

[0082] It should be noted that the number of license plate characters of a general license plate is certain, but the number of license plate characters in different regions and different models is different. For example, the number of license plate characters of a general fuel vehicle is 7, while the number of license plate characters of a new energy vehicle is 8. In view of this, the present invention can pre-set a character quantity threshold range: N1-N2, where N1 is the lower limit of the character quantity threshold range, and N2 is the upper limit of the character quantity threshold range. If the number of license plate characters of the current recognition result is within the character quantity threshold range, the current recognition result is considered to be valid, otherwise it is considered to be invalid data, and a new video frame needs to be re-captured for license plate character recognition.

[0083] Step 103: Based on the current recognition result, the pre-stored historical recognition result is updated to determine the license plate number of the target vehicle.

[0084] The present invention establishes a cache in a processor for implementing the license plate recognition method to store historical recognition results of license plate numbers, which can be stored in a database, file or memory.

[0085] It should be noted that there is generally only one historical recognition result in the cache, which is to recognize multiple video frames collected historically, and store the optimal recognition result in the cache. When a new recognition result is obtained subsequently, it only needs to be compared with the optimal recognition result stored in the cache, without having to compare with the recognition results of all historical video frames, which can effectively avoid the waste of computing resources.

[0086] After obtaining the current recognition result corresponding to the current video frame, it is compared with the historical recognition result in the cache, that is, the optimal recognition result.

[0087] When determining the comparison result, the present invention comprehensively evaluates the quality of the recognition result of the license plate of the target vehicle according to a preset comparison logic, in combination with the number of license plate characters and the confidence level of each license plate character.

[0088] As an alternative embodiment, updating the pre-stored historical recognition result based on the current recognition result includes:

[0089] Comparing the current recognition result with the historical recognition result from two dimensions: the number of license plate characters and the recognition confidence level of each license plate character;

[0090] Updating the historical recognition result according to the comparison result.

[0091] Specifically, if the comparison result shows that the current recognition result is better than the historical recognition result in the cache, then replace the historical recognition result in the cache with the current recognition result as the new optimal recognition result.

[0092] If the comparison result shows that the current recognition result is worse than the historical recognition result in the cache, then keep the historical recognition result in the cache unchanged.

[0093] After comparing the current recognition result corresponding to the current video frame with the historical recognition result in the cache and completing the update of the recognition result stored in the cache according to the comparison result, it is possible to further determine whether the recognition result in the updated cache can finally determine the license plate number of the target vehicle.

[0094] For example, the license plate number of the target vehicle can be determined by the number of license plate characters in the recognition result in the cache and the recognition confidence level of each license plate character. When the number of license plate characters is within the range of the character number threshold and it can be confirmed that the lowest confidence level among the recognition confidence levels of all license plate characters is greater than a certain preset threshold, then it can be considered that the recognition result in the cache is credible, and the license plate number of the target vehicle can be composed of all the license plate characters in this recognition result.

[0095] Suppose all the license plate characters in the recognition result and the recognition execution level of each license plate character are as shown in Table 1, and the above preset threshold is 90%, then the license plate number of the target vehicle can be determined as Yue A12T02.

[0096] Suppose the above preset threshold is 95%, then since the recognition confidence levels of some license plate characters are less than this preset threshold, it means that it is not credible to directly determine the license plate number of the target vehicle from the updated recognition result in the cache. At this time, the next video frame can be re-acquired, and the above steps can be iterated until the update of the recognition result in the cache is completed, so that the updated recognition result is more credible for predicting the license plate number of the target vehicle.

[0097] The license plate recognition method provided by the present invention, after obtaining the recognition result of the current video frame, combines the two dimensions of the number of recognized characters and the confidence of each character to determine whether it is better than the historical best recognition result in the cache, so as to iteratively update the content in the cache of the historical best recognition result, ensure that the cache contains the best recognition result, and perform license plate recognition based on the updated recognition result, thereby effectively improving the accuracy of license plate recognition.

[0098] Based on the content of the above embodiment, as an optional embodiment, comparing the current recognition result with the historical recognition result from two dimensions, namely, the number of license plate characters and the recognition confidence of each license plate character, includes:

[0099] Obtaining a first comparison result of a first quantity and a second quantity, wherein the first quantity is the quantity of license plate characters included in the current recognition result, and the second quantity is the quantity of license plate characters included in the historical recognition result;

[0100] Obtaining a second comparison result between the first number and a preset character number threshold range;

[0101] Determine the current lowest confidence and the current confidence average among the recognition confidences of all license plate characters included in the current recognition result, and the historical lowest confidence and the historical confidence average among the recognition confidences of all license plate characters included in the historical recognition results;

[0102] Obtaining a third comparison result between the current lowest confidence and the historical lowest confidence, and a fourth comparison result between the current confidence mean and the historical confidence mean;

[0103] The comparison result is determined based on the first comparison result, the second comparison result, the third comparison result, and the fourth comparison result.

[0104] The present invention provides a specific implementation method of comparing a current recognition result with a historical recognition result in a cache, which mainly includes but is not limited to the following steps:

[0105] Step 1: compare a first number of license plate characters included in a current recognition result with a second number of license plate characters included in a historical recognition result.

[0106] Since the historical recognition results stored in the cache are the results of recognizing historical video frames, and the current recognition results are the results of recognizing the current video frames, different video frames will cause differences in recognition results to a certain extent due to changes in the video shooting environment, shooting angles caused by vehicle movement, and changes in obstacles such as cover. The most intuitive manifestation is the recognition of the number of license plate characters. Therefore, when comparing the current recognition result with the historical recognition result, the present invention focuses on the first comparison result of the number of license plate characters.

[0107] Step 2, compare the first number of license plate characters contained in the current recognition result with the character number threshold range, because it can directly determine whether the number of license plate characters contained in the current recognition result is reasonable. Assuming that the character number threshold range is 6-8, if the first number is 5, it means that some license plate characters must not be recognized. However, the historical recognition result in the cache is the optimal recognition result obtained by recognizing the historical video frame, and the second number of license plate characters contained therein must be within the character number threshold range. At this time, it can be directly determined that the current recognition result is worse than the historical recognition result.

[0108] Step 3, respectively determine the minimum confidence and confidence mean of the current recognition result, and the minimum confidence and confidence mean of the historical recognition results.

[0109] The current minimum confidence corresponding to the current recognition result refers to the lowest confidence value among the recognition confidences of all license plate characters in the current recognition result.

[0110] The current confidence mean corresponding to the current recognition result refers to the average value of the recognition confidence of all license plate characters in the current recognition result.

[0111] The historical lowest confidence corresponding to the historical recognition results refers to the lowest confidence value among the recognition confidences of all license plate characters in the historical recognition results.

[0112] The historical confidence mean corresponding to the historical recognition results refers to the average value of the recognition confidence of all license plate characters in the historical recognition results.

[0113] Step 4: compare the current lowest confidence with the historical lowest confidence, and compare the current confidence mean with the historical confidence mean, to obtain a third comparison result and a fourth comparison result respectively.

[0114] Generally speaking, the higher the minimum confidence and the larger the mean confidence, the better the corresponding recognition result.

[0115] Step 5: According to the differences among the first comparison result, the second comparison result, the third comparison result and the fourth comparison result, the superiority or inferiority between the current recognition result and the historical recognition result can be determined.

[0116] The license plate recognition method provided by the present invention utilizes the number of license plate characters and recognition confidence to establish recognition evaluation rules, wherein the relationship between the number of license plate characters in the current recognition result and the number of license plate characters in the cache is considered in the rules, and then a recognition result comparison relationship is established respectively by the recognition confidence, a new optimal recognition result is determined, and the optimal recognition result is cached, and the advantages and disadvantages between the two are measured from multiple dimensions, thereby improving the recognition accuracy of the finally recognized license plate number.

[0117] Based on the content of the above embodiment, as an optional embodiment, when it is determined that the first comparison result is that the first number is less than the second number, the present invention provides a specific implementation method of determining the comparison result based on the first comparison result, the second comparison result, the third comparison result and the fourth comparison result, including but not limited to:

[0118] If the second comparison result is that the first number is within the character number threshold range, and the current minimum confidence is greater than the first confidence threshold, then determining that the comparison result is that the current recognition result is better than the historical recognition result;

[0119] Otherwise, it is determined that the comparison result is that the current recognition result is not better than the historical recognition result.

[0120] When it is determined that the first comparison result is that the first number is smaller than the second number, it means that the historical recognition results stored in the cache may have missed the recognition of license plate characters. Then the number of license plate characters in the correct recognition result must be greater than the second number (but must be less than the upper limit value N2 of the character number threshold range).

[0121] Furthermore, in order to ensure that the multiple recognized license plate characters in the current recognition result are correctly recognized rather than incorrectly recognized, the present invention determines whether the minimum confidence of the current recognition result (i.e., the current minimum confidence) is greater than a preset first confidence threshold (which can be assumed to be 90%). If the minimum confidence is greater than the first confidence threshold, it is considered that the multiple recognized license plate characters are correctly recognized, and ultimately it can be confirmed that the current recognition result is better than the historical recognition results in the cache.

[0122] Otherwise, if the first number is not within the character number threshold or the lowest confidence of all license plate characters in the current recognition result is less than or equal to the first confidence threshold, the current recognition result is considered to be worse than the historical recognition result.

[0123] It should be noted that when the first comparison result is that the first number is less than the second number, since the advantages and disadvantages between the current recognition result and the historical recognition result can be judged based on the first comparison result and the second comparison result, there is no need to combine the third comparison result and the fourth comparison result for reference comparison of the advantages and disadvantages of the two.

[0124] For example, if the first number of license plate characters in the current recognition result is 9, and the character number threshold range is 6-8, then it can be determined that there are incorrectly recognized license plate characters in the current recognition result, and the historical recognition results are better than the current recognition results at least in terms of the number of license plate characters, then it can be considered that the current recognition result is worse than the historical recognition results.

[0125] For another example, if the lowest confidence among all license plate characters in the current recognition result is 80%, which is less than the first confidence threshold of 90%, it means that the recognition accuracy of at least one license plate character in the current recognition result is too low. Since the lowest confidence of the historical recognition results in the cache is greater than the first confidence threshold of 90%, it can be considered that the current recognition result is worse than the historical recognition result.

[0126] As another optional embodiment, when the first comparison result is that the first number is greater than or equal to the second number, determining the final comparison result based on the first comparison result, the second comparison result, the third comparison result, and the fourth comparison result may include but is not limited to the following steps:

[0127] If the second comparison result is that the first number is within the character number threshold range, the third comparison result is that the current lowest confidence is greater than the historical lowest confidence, and the fourth comparison result is that the current confidence mean is greater than the historical confidence mean, then the comparison result is determined to be that the current recognition result is better than the historical recognition result;

[0128] Otherwise, it is determined that the comparison result is that the current recognition result is not better than the historical recognition result.

[0129] In the case where the first comparison result is that the first quantity is less than the second quantity, it is also necessary to first determine whether the number of license plate characters in the current recognition result is within the character quantity threshold range. Since the second quantity is less than the upper limit of the character quantity threshold range, the first quantity must be less than the upper limit of the character quantity threshold range at this time, so it is only necessary to determine whether the first quantity is greater than the lower limit of the character quantity threshold range. If the first quantity is greater than the lower limit of the character quantity threshold range, it can be considered that the first quantity is within the character quantity threshold range.

[0130] Furthermore, by obtaining the third comparison result between the current lowest confidence and the historical lowest confidence, and obtaining the fourth comparison result between the current confidence average and the historical confidence average, the pros and cons of the current recognition result and the historical recognition result can be judged.

[0131] When the third comparison result and the fourth comparison result show that the current recognition result is better, it means that the current recognition result is better than the historical recognition result.

[0132] On the contrary, if the third comparison result is that the current lowest confidence is less than or equal to the historical lowest confidence, or the fourth comparison result is that the current confidence mean is less than or equal to the historical confidence mean, it cannot be said that the current recognition result is better than the historical recognition result.

[0133] Meanwhile, if the first number is not within the character number threshold, it means that some license plate characters must not be recognized in the current recognition result, and it cannot be said that the current recognition result is better than the historical recognition result.

[0134] As another optional embodiment, when the first comparison result is equal to that the first number is less than the second number, since the second number is within the character number threshold range, there is no need to determine whether the first number is within the character number threshold range.

[0135] At this time, it is necessary to obtain the third comparison result and the fourth comparison result to determine the pros and cons between the current recognition result and the historical recognition result according to the two comparison results.

[0136] If the third comparison result and the fourth comparison result show that the current recognition result is better, it means that the current recognition result is better than the historical recognition result. Otherwise, it does not mean that the current recognition result is better than the historical recognition result.

[0137] The license plate recognition method provided by the present invention compares the advantages and disadvantages of the current recognition result and the historical recognition results based on the number of license plate characters in the current recognition result and the historical recognition results in the cache and the recognition confidence of a single license plate character. It can accurately cover the current license plate numbers of different lengths, realize recognition quality judgment in two dimensions of the number of license plate characters and the recognition confidence of the characters, and improve the recognition accuracy.

[0138] Based on the content of the above embodiment, as an optional embodiment, updating the historical recognition result according to the comparison result includes:

[0139] When it is determined that the comparison result shows that the current recognition result is better than the historical recognition result, replacing the historical recognition result in the cache with the current recognition result;

[0140] Otherwise, the historical recognition result in the cache is kept unchanged.

[0141] Specifically, when the comparison result shows that the current recognition result is better than the historical recognition result, the current recognition result can be used to replace the historical recognition result in the cache as the optimal recognition result.

[0142] On the contrary, if the comparison result is that the current recognition result is not better than the historical recognition result, it means that there is a problem with the acquisition of the current video frame, or the recognition processing of the current video frame is incorrect. There is no need to use the current recognition result to replace the historical recognition result in the cache, but to re-capture the next video frame to re-determine the license plate number of the target vehicle based on the recognition result of the next video frame combined with the optimal historical recognition result in the current cache.

[0143] Based on the content of the above embodiment, as an optional embodiment, after replacing the historical recognition result in the cache with the current recognition result, determining the license plate number of the target vehicle includes:

[0144] In the case where it is determined that the current recognition result meets the preset upload condition, determining the license plate number of the target vehicle according to all license plate characters included in the current recognition result;

[0145] The preset upload condition is that the current minimum confidence level is greater than a second confidence level threshold, and the second confidence level threshold is greater than the first confidence level threshold.

[0146] The license plate recognition method provided by the present invention ultimately aims to accurately recognize the license plate number of a target vehicle by iteratively comparing and iteratively obtaining an optimal recognition result which is stored in a cache.

[0147] After each video frame is recognized and a new recognition result is obtained, if the recognition result is used to update the historical recognition results in the cache, it means that the obtained recognition result is better than all historical recognition results. At this time, it is necessary to determine whether the recognition result meets the upload conditions for determining the license plate number of the target vehicle. If it meets the conditions, the recognition result is uploaded to the output unit of the processor to generate and output the license plate number of the target vehicle based on the recognition result; if it does not meet the upload conditions, continue to collect the next video frame.

[0148] It should be noted that the present invention has a higher requirement for judging whether the current recognition result meets the preset upload condition than for judging whether the current recognition result is stored in the cache. That is to say, the judgment of whether the preset upload condition is met is to compare the current minimum confidence of the current recognition result with the second confidence threshold, and when judging whether the current recognition result will be used to replace the historical recognition result in the cache, it is only necessary to compare whether the current minimum confidence is greater than the first confidence threshold. Obviously, the second execution threshold is required to be greater than the first confidence threshold.

[0149] If the current minimum confidence is greater than the second confidence threshold, it means that the current recognition result is qualified and the license plate number of the target vehicle can be accurately determined based on all the license plate characters it contains. At this time, all subsequent operations for license plate number recognition can be directly terminated.

[0150] On the contrary, if the current minimum confidence is less than or equal to the second confidence threshold, it means that the recognition accuracy of the current recognition result still cannot meet the requirement of accurately determining the target vehicle license plate number.

[0151] The license plate recognition method provided by the present invention utilizes the two dimensions of license plate character recognition and recognition confidence to jointly establish a termination mechanism. If the current recognition result meets the upload conditions, the subsequent processing of the license plate number of the target vehicle is stopped. In this way, during the entire recognition process, once a result that meets the upload conditions is obtained, the multi-frame recognition operation is immediately stopped, saving limited computing resources on the terminal side.

[0152] As another optional embodiment, when it is determined that the current recognition result does not meet the preset upload condition, the next video frame is re-acquired;

[0153] The next video frame is set as the current video frame, and the steps of inputting the current video frame into the license plate recognition model to determining the license plate number of the target vehicle based on the updated recognition result are iteratively performed until it is determined that the recognition result in the cache meets the preset upload condition or the next video frame is the last video frame;

[0154] The license plate number of the target vehicle is determined based on all license plate characters included in the recognition result in the cache.

[0155] The problem to be solved by the present invention is how to accurately identify the license plate number of the target vehicle. However, if the current recognition result obtained according to the current video frame does not meet the preset upload conditions, it is necessary to continue to capture the next video frame.

[0156] By using the method provided in the above embodiment, the next video frame can be preprocessed, the located license plate area can be cropped from the next video frame, and input into the trained license plate recognition model for recognition. The license plate recognition model will recognize the license plate characters contained in the license plate area to obtain a new recognition result.

[0157] Then, the new recognition result is compared with the historical recognition result stored in the cache to update the recognition result in the cache.

[0158] Finally, it is re-determined whether the updated recognition result in the cache meets the preset upload condition, that is, whether the minimum confidence of the updated recognition result is greater than the second confidence threshold.

[0159] If so, the license plate number of the target vehicle can be determined based on all the license plate characters included in the updated recognition result.

[0160] If not, continue to capture the next video frame and iterate all the above processes until a recognition result that meets the preset upload conditions is obtained.

[0161] The license plate recognition method provided by the present invention establishes a termination mechanism using the number of license plate characters and recognition confidence. If the recognition result meets the preset upload conditions, the subsequent processing of the target vehicle license plate is stopped. In this way, during the processing, once the recognition result that meets the preset upload conditions is obtained, the multi-frame recognition operation is immediately stopped, saving limited computing resources on the terminal side.

[0162] Figure 2 This is a second flow chart of the license plate recognition method provided by the present invention, as an optional embodiment, such as Figure 2 As shown, in the case where it is determined that the next video frame is the last video frame, it also includes:

[0163] Inputting the next video frame into the license plate recognition model to obtain a final recognition result output by the license plate recognition model;

[0164] Using the final recognition result to update the historical recognition results in the cache;

[0165] If the lowest confidence of all the license plate characters included in the updated recognition result is less than or equal to the second confidence threshold and greater than the third confidence threshold, determining the license plate number of the target vehicle based on all the license plate characters included in the updated recognition result;

[0166] If the lowest confidence of all license plate characters included in the updated recognition result is less than or equal to the third confidence threshold, determining the license plate number of the target vehicle based on the best-scoring image in the cache;

[0167] The third confidence threshold is greater than the first confidence threshold; the best-scoring image is a video frame with the highest QoE score extracted from all video frames containing the target vehicle.

[0168] Among them, the situation of determining that the next video frame is the last video frame mainly occurs when, during the process of identifying the license plate number of the target vehicle, if the identification result that meets the preset upload conditions has not been obtained, the target vehicle has disappeared from the monitorable area of ​​the video acquisition device.

[0169] At this time, there may already be a historical recognition result in the cache, which is also the best recognition result up to now. The final recognition result obtained after processing the last video frame is compared with the historical recognition result in the cache.

[0170] If the final recognition result is better than the historical recognition result, the historical recognition result is updated with the final recognition result as the updated recognition result in the cache.

[0171] If the final recognition result is not better than the historical recognition result, the historical recognition result continues to be used as the updated recognition result in the cache.

[0172] Since the updated recognition result is already the best recognition result that can be obtained, we first determine whether it meets the preset upload conditions, that is, the minimum confidence corresponding to the updated recognition result is greater than the second confidence threshold. Then the updated recognition result can be uploaded and the license plate number of the target vehicle can be determined based on it.

[0173] If the updated recognition result does not meet the preset upload conditions, that is, its corresponding minimum confidence is less than or equal to the second confidence threshold, you can consider appropriately lowering the recognition accuracy, a second confidence threshold that is greater than the first confidence threshold but less than the third confidence threshold. If its corresponding minimum confidence is greater than the third confidence threshold, continue to use this updated recognition result to recognize the license plate number.

[0174] However, if the minimum confidence of the updated recognition result is less than or equal to the third confidence threshold, it means that the recognition accuracy of the license plate number cannot meet the minimum recognition accuracy requirement. In this case, other means are needed to assist in the recognition of the license plate number.

[0175] The present invention also provides a method for assisting license plate number recognition by using QoE (Quality of Experience) score. When extracting video frames, the video frames with the highest QoE score are also stored in the cache, which are called the best-scoring images. When acquiring each new video frame, the best-scoring images already stored in the cache are updated according to the QoE score of the new video frame.

[0176] Among them, QoE score is a quantitative indicator or score that measures the user's perception and satisfaction with a specific product, service or application from the user's perspective. QoE score can be based on various factors, such as image clarity, detail expression, color accuracy, etc.

[0177] The present invention constructs a QoE scoring model and pre-trains it using image samples and the QoE label of each image sample, which may include but is not limited to the following steps:

[0178] (1) Data collection: Collect image samples under various scenes and conditions, such as different lighting conditions, different shooting angles, and different backgrounds. At the same time, annotate each image sample with its corresponding QoE score label, which can be achieved through manual subjective evaluation or objective evaluation methods.

[0179] (2) Model construction: Select a suitable basic model and build a QoE scoring model. This can be achieved by using machine learning algorithms, such as Support Vector Regression (SVR) and Random Forest. In this step, it is necessary to select a suitable machine learning algorithm based on specific needs and scenarios, and adjust and optimize it to improve the prediction accuracy and generalization ability of the QoE scoring model.

[0180] (3) Pre-training: Use the collected image samples and their corresponding QoE score labels to pre-train the constructed QoE score model. This can be achieved by using a supervised learning algorithm, such as support vector regression or random forest algorithm for training. During the pre-training process, the data set needs to be divided into a training set and a validation set, and the model needs to be cross-validated and adjusted to improve the generalization ability and prediction accuracy of the model.

[0181] (4) Model testing and evaluation: Test and evaluate the pre-trained QoE scoring model. This can be done by using the test set to evaluate the model and calculating its prediction accuracy, recall rate, F1 value and other indicators. At the same time, cross-validation can be used to further verify the generalization ability and stability of the model.

[0182] (5) Model optimization: Based on the test and evaluation results, the QoE scoring model is optimized and improved. This can be achieved by adjusting model parameters, increasing the number of features, increasing data samples, etc. During the model optimization process, continuous testing and evaluation are required to ensure that the model's prediction accuracy and generalization ability are improved.

[0183] Continuing with the content of the above embodiment, if the updated lowest confidence of the recognition result is less than or equal to the third confidence threshold, the best-scoring image stored in the cache can be extracted to read the license plate number of the target vehicle based on the image.

[0184] Generally speaking, since the best-scoring image is better in terms of image clarity, detail expression, color accuracy, etc., the best-scoring image can be directly output to the user, and the user can identify the license plate number therein by himself.

[0185] Of course, image processing and computer vision technology can also be used for license plate detection and character recognition, including a combination of Canny edge detection, contour extraction, color segmentation and optical character recognition (OCR).

[0186] Alternatively, a license plate detection and character recognition method based on deep learning includes using a convolutional neural network combined with object detection and segmentation algorithms to detect the license plate area, and realizing character recognition through models such as RNN and LSTM.

[0187] The license plate recognition method provided by the present invention uses a QoE scoring model to score the image quality of the collected video frames, saves the video frame belonging to the license plate with the highest QoE score to a cache, and obtains the best-scored image that is the clearest and easiest for the human eye to discern the license plate number subjectively. When uncertainty occurs in the process of automatically recognizing the license plate characters based on the recognition results of the video frames, the license plate number can be recognized with assistance of the best-scoring image in the cache. A recognition method with a relatively objective recognition confidence of the license plate characters is used as the main recognition basis, and a recognition method with a relatively subjective auxiliary QoE score is used as an auxiliary recognition basis. The combination of the two can effectively ensure the accurate recognition of the target vehicle license plate number.

[0188] As an optional embodiment, the present invention provides a method for updating the best-scoring image in a cache, which mainly includes but is not limited to the following steps:

[0189] Inputting the current video frame into a QoE scoring model to obtain a current QoE score output by the QoE scoring model;

[0190] If the current QoE score is greater than the QoE score of the best-scoring image in the cache, the best-scoring image in the cache is replaced with the current video frame as a new best-scoring image;

[0191] Otherwise, the image with the best score in the cache is kept unchanged.

[0192] In the process of license plate number recognition, tracking the target vehicle can help to more accurately determine the location of the license plate and better adapt to different scenarios, thereby improving recognition accuracy and robustness.

[0193] In view of this, the license plate recognition model provided by the present invention mainly includes a target detection model and a license plate reading model.

[0194] The above-mentioned inputting the current video frame into the license plate recognition model and obtaining the current recognition result output by the license plate recognition model mainly includes:

[0195] Inputting the current video frame into the target detection model to mark the vehicle detection frame coordinates and the license plate detection frame coordinates in the current video frame;

[0196] Cropping the current video frame based on the license plate detection frame coordinates to obtain a license plate image;

[0197] The license plate image is input into the license plate reading model to obtain the current recognition result output by the license plate reading model.

[0198] The current video frame can be converted into an image first and then input into the object detection model to detect whether there is a vehicle in the image. Typically, existing object detection models or detectors such as Faster R-CNN, YOLO, SSD, etc. can be used for vehicle detection.

[0199] If the target detection model or detector can automatically mark the coordinates of the license plate, you can use this information to mark the vehicle detection frame coordinates and the license plate detection frame coordinates. Otherwise, you need to use other technologies such as OCR, color-based license plate recognition, etc. to detect the license plate and obtain the license plate coordinates.

[0200] Then, the license plate image can be cropped from the current video frame according to the marked license plate detection frame coordinates. Usually, the image cropping algorithm can be used for cropping.

[0201] After preprocessing the current video frame, the cropped license plate image can be input into the trained license plate reading model to recognize the license plate character sequence. The construction of the license plate reading model can use various deep learning models as initial models, such as Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN).

[0202] Finally, the license plate reading model will output multiple license plate characters, and all license plate characters will be output in a sequence. The goal of license plate number recognition can be achieved based on the license plate character sequence.

[0203] Furthermore, after marking the coordinates of the vehicle detection frame in the current video frame, the following steps may also be included:

[0204] The motion trajectory of the target vehicle is determined according to an intersection-and-union ratio of the vehicle detection frame coordinates in the current video frame and the adjacent video frames of the current video frame.

[0205] After obtaining the coordinates of the vehicle detection frame and the license plate detection frame in the current video frame, the present invention can track the running trajectory of the target vehicle according to the two coordinate information. The main principle is to track the target vehicle by using the motion trajectory and the IoU relationship of the vehicle detection frames of the previous and next frames of the video, and assign a unique ID to the target vehicle. The main steps include:

[0206] Step 1, trajectory initialization: In the initial video frame, the target vehicle’s position is detected using the target detection model and used as the initial vehicle detection frame to initialize the trajectory.

[0207] Step 2, motion prediction: Based on the coordinates of the vehicle detection frame in the previous video frame, the approximate position of the target vehicle in the next video frame is predicted by the motion model. Commonly used motion models include linear models, Kalman filters, etc.

[0208] Step 3, vehicle detection frame matching: In the current video frame, the target detection model is used to detect the position of the target vehicle, and the IoU relationship between all vehicle detection frames in the current video frame and the predicted vehicle detection frame is calculated.

[0209] Step 4, IoU threshold screening: According to the set IoU threshold, filter out the vehicle detection boxes whose IoU values ​​with the predicted vehicle detection boxes are higher than the IoU threshold.

[0210] Step 5, trajectory update: match the filtered vehicle detection frame with the previous trajectory to update or create a new trajectory.

[0211] Step 6, track management: Maintain a track set that contains all currently valid tracks. For unmatched vehicle detection frames, they can be regarded as new vehicles and new tracks can be created for tracking. At the same time, tracks that have not been updated for a long time need to be managed, such as deleted or reinitialized.

[0212] Step 7, trajectory output: Based on the tracking results, relevant information of each motion trajectory can be output, such as vehicle ID, coordinate trajectory, etc.

[0213] It should be noted that the tracking method of vehicle motion trajectory and IoU relationship has certain defects and challenges. For example, there are situations such as occlusion, target deformation, and trajectory intersection, which may lead to inaccurate tracking. The algorithm can be improved and optimized according to the specific situation, such as introducing appearance features, updating the motion model, etc., to improve the accuracy and robustness of tracking.

[0214] After determining the movement trajectory of the target vehicle, it can be determined that the license plate belongs to the tracked target vehicle, mainly including:

[0215] 1) Determine the vehicle detection frame and the license plate detection frame: From the current video frame, use the license plate detection algorithm or the license plate recognition algorithm to extract the license plate detection frame, and use the target detection or tracking algorithm to extract the vehicle detection frame of the target vehicle.

[0216] 2) Calculate the IoU value between the vehicle detection frame and the license plate detection frame: According to the coordinates of the vehicle detection frame and the license plate detection frame, calculate their intersection area and union area, and then calculate the IoU value.

[0217] 3) Set IoU threshold: Set a new IoU threshold according to the specific application scenario and requirements. Usually, when the calculated IoU value is greater than or equal to the IoU threshold, the recognized license plate is considered to belong to the tracked target vehicle.

[0218] 4) Determine the ownership of the license plate: The calculated IoU value and the set new IoU threshold can be used to determine whether the license plate area belongs to a tracked motor vehicle.

[0219] The last step can be achieved by following these steps:

[0220] 4.1) Maintain a list or dictionary of license plate detection frames for each tracked target vehicle, which contains the license plate detection frame information of each target vehicle.

[0221] 4.2) When a new license plate detection frame is extracted from the current video frame, the IoU value between the license plate detection frame and the license plate detection frame of each tracked target vehicle is calculated.

[0222] 4.3) For each tracked target vehicle, if the IoU value between its license plate detection frame and the new license plate detection frame is greater than or equal to the set IoU threshold, the license plate is considered to belong to the target vehicle.

[0223] 4.4) If there are multiple tracked target vehicles that meet the IoU threshold condition, further screening can be performed based on the size of the IoU value, and the target vehicle with the largest IoU value can be selected as the target vehicle.

[0224] 4.5) If no tracked target vehicle meets the IoU threshold condition, the license plate is considered to belong to a new target vehicle and a new tracking object is created for it.

[0225] The present invention can determine whether the target vehicle leaves the shooting area according to the above-mentioned motion trajectory tracking. If it is determined to leave, the video frame at the time of leaving is used as the last video frame. The method provided by the above-mentioned embodiment is adopted, and the recognition confidence of the license plate characters is relatively objective as the main recognition basis, and the auxiliary QoE score is relatively subjective as the auxiliary recognition basis, so as to ensure the accurate recognition of the license plate number of the target vehicle.

[0226] Figure 3 Schematic diagram of the structure of the license plate recognition device provided by the present invention. Figure 3 As shown, the present invention also provides a license plate recognition device, which mainly includes:

[0227] An image acquisition unit 31 is used to acquire a current video frame including a target vehicle;

[0228] The license plate recognition unit 32 is used to input the current video frame into the license plate recognition model to obtain the current recognition result output by the license plate recognition model; the current recognition result is the recognized license plate characters and the recognition confidence of each of the license plate characters;

[0229] The license plate determination unit 33 is used to update the pre-stored historical recognition result based on the current recognition result to determine the license plate number of the target vehicle.

[0230] It should be noted that the license plate recognition device provided in the embodiment of the present invention can execute the license plate recognition method described in any of the above embodiments during specific operation, which will not be described in detail in this embodiment.

[0231] The license plate recognition device provided by the present invention, after obtaining the recognition result of the current video frame, combines the two dimensions of the number of recognized characters and the confidence of each character to determine whether it is better than the historical best recognition result in the cache, so as to iteratively update the content in the cache of the historical best recognition result, ensure that the cache contains the best recognition result, and perform license plate recognition based on the updated recognition result, thereby effectively improving the accuracy of license plate recognition.

[0232] Figure 4 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the license plate recognition method, which includes: obtaining a current video frame containing a target vehicle; inputting the current video frame into a license plate recognition model to obtain a current recognition result output by the license plate recognition model; the current recognition result includes the license plate characters obtained after recognizing the license plate number of the target vehicle and the recognition confidence of each license plate character; based on the number of license plate characters and the recognition confidence of each license plate character, the current recognition result is compared with the historical recognition result in the cache; the historical recognition result is updated according to the comparison result; and the license plate number of the target vehicle is determined based on the updated recognition result.

[0233] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0234] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the license plate recognition method provided by the above-mentioned embodiments, the method including: obtaining a current video frame containing a target vehicle; inputting the current video frame into a license plate recognition model to obtain a current recognition result output by the license plate recognition model; the current recognition result includes license plate characters obtained after recognizing the license plate number of the target vehicle and the recognition confidence of each license plate character; based on the number of license plate characters and the recognition confidence of each license plate character, comparing the current recognition result with the historical recognition results in the cache; updating the historical recognition results according to the comparison results; and determining the license plate number of the target vehicle based on the updated recognition result.

[0235] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the license plate recognition method provided by the above-mentioned embodiments, the method comprising: obtaining a current video frame containing a target vehicle; inputting the current video frame into a license plate recognition model to obtain a current recognition result output by the license plate recognition model; the current recognition result comprises license plate characters obtained after recognizing the license plate number of the target vehicle and a recognition confidence of each license plate character; based on the number of license plate characters and the recognition confidence of each license plate character, comparing the current recognition result with historical recognition results in a cache; updating the historical recognition results according to the comparison result; and determining the license plate number of the target vehicle based on the updated recognition result.

[0236] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0237] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0238] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A license plate recognition method, characterized in that: include: Get the current video frame containing the target vehicle; Input the current video frame into the license plate recognition model to obtain a current recognition result output by the license plate recognition model; the current recognition result is the recognized license plate characters and the recognition confidence of each license plate character; The pre-stored historical recognition results are updated based on the current recognition results to determine the license plate number of the target vehicle.

2. The license plate recognition method according to claim 1, characterized in that: The updating of the pre-stored historical recognition results based on the current recognition results includes: Comparing the current recognition result with the historical recognition result from two dimensions: the number of license plate characters and the recognition confidence of each license plate character; The historical recognition result is updated according to the comparison result.

3. The license plate recognition method according to claim 2, characterized in that: The comparing the current recognition result with the historical recognition result from two dimensions, namely, the number of license plate characters and the recognition confidence of each license plate character, includes: Obtaining a first comparison result of a first quantity and a second quantity, wherein the first quantity is the quantity of license plate characters included in the current recognition result, and the second quantity is the quantity of license plate characters included in the historical recognition result; Obtaining a second comparison result between the first number and a preset character number threshold range; Determine the current lowest confidence and the current confidence average among the recognition confidences of all license plate characters included in the current recognition result, and the historical lowest confidence and the historical confidence average among the recognition confidences of all license plate characters included in the historical recognition results; Obtaining a third comparison result between the current lowest confidence and the historical lowest confidence, and a fourth comparison result between the current confidence mean and the historical confidence mean; The comparison result is determined based on the first comparison result, the second comparison result, the third comparison result, and the fourth comparison result.

4. The license plate recognition method according to claim 3, characterized in that: When the first comparison result is that the first number is less than the second number, determining the comparison result based on the first comparison result, the second comparison result, the third comparison result, and the fourth comparison result includes: If the second comparison result is that the first number is within the character number threshold range, and the current minimum confidence is greater than the first confidence threshold, then determining that the comparison result is that the current recognition result is better than the historical recognition result; Otherwise, it is determined that the comparison result is that the current recognition result is not better than the historical recognition result.

5. The license plate recognition method according to claim 3, characterized in that: When the first comparison result is that the first number is greater than or equal to the second number, determining the comparison result based on the first comparison result, the second comparison result, the third comparison result, and the fourth comparison result includes: If the second comparison result is that the first number is within the character number threshold range, the third comparison result is that the current lowest confidence is greater than the historical lowest confidence, and the fourth comparison result is that the current confidence mean is greater than the historical confidence mean, then the comparison result is determined to be that the current recognition result is better than the historical recognition result; Otherwise, it is determined that the comparison result is that the current recognition result is not better than the historical recognition result.

6. The license plate recognition method according to any one of claims 4 to 5, characterized in that: The updating of the historical recognition result according to the comparison result includes: When it is determined that the comparison result shows that the current recognition result is better than the historical recognition result, replacing the historical recognition result with the current recognition result; Otherwise, the historical recognition result remains unchanged.

7. The license plate recognition method according to claim 6, characterized in that: After replacing the historical recognition result with the current recognition result, determining the license plate number of the target vehicle includes: In the case where it is determined that the current recognition result meets the preset upload condition, determining the license plate number of the target vehicle according to all license plate characters included in the current recognition result; The preset upload condition is that the current minimum confidence level is greater than a second confidence level threshold, and the second confidence level threshold is greater than the first confidence level threshold.

8. The license plate recognition method according to claim 7, characterized in that: If it is determined that the current recognition result does not meet the preset upload condition, reacquiring the next video frame; The next video frame is set as the current video frame, and the step of inputting the current video frame into the license plate recognition model to determine the license plate number of the target vehicle is iteratively performed until it is determined that the recognition result meets the preset upload condition or the next video frame is the last video frame; The license plate number of the target vehicle is determined based on all license plate characters included in the recognition result.

9. The license plate recognition method according to claim 8, characterized in that: In the case where it is determined that the next video frame is the last video frame, the method further includes: Inputting the next video frame into the license plate recognition model to obtain a final recognition result output by the license plate recognition model; Using the final recognition result to update the historical recognition result; If the lowest confidence of all the license plate characters included in the updated recognition result is less than or equal to the second confidence threshold and greater than the third confidence threshold, determining the license plate number of the target vehicle based on all the license plate characters included in the updated recognition result; If the lowest confidence of all license plate characters included in the updated recognition result is less than or equal to the third confidence threshold, determining the license plate number of the target vehicle based on the best-scoring image; The third confidence threshold is greater than the first confidence threshold; the best-scoring image is a video frame with the highest QoE score extracted from all video frames containing the target vehicle.

10. The license plate recognition method according to claim 9, characterized in that: The best scoring image is extracted based on the following steps: Inputting the current video frame into a QoE scoring model to obtain a current QoE score output by the QoE scoring model; If the current QoE score is greater than the QoE score of the best-scoring image, the best-scoring image is replaced with the current video frame as a new best-scoring image; Otherwise, keep the best-scoring image unchanged; The QoE scoring model is trained based on image samples and the QoE label of each image sample.

11. The license plate recognition method according to claim 1, characterized in that: The license plate recognition model includes a target detection model and a license plate reading model. The current video frame is input into the license plate recognition model to obtain the current recognition result output by the license plate recognition model, including: Inputting the current video frame into the target detection model to mark the vehicle detection frame coordinates and the license plate detection frame coordinates in the current video frame; Cropping the current video frame based on the license plate detection frame coordinates to obtain a license plate image; The license plate image is input into the license plate reading model to obtain the current recognition result output by the license plate reading model.

12. A license plate recognition device, characterized in that: include: An image acquisition unit, used for acquiring a current video frame containing a target vehicle; A license plate recognition unit, used to input the current video frame into a license plate recognition model, and obtain a current recognition result output by the license plate recognition model; the current recognition result is the recognized license plate characters and the recognition confidence of each of the license plate characters; The license plate determination unit is used to update the pre-stored historical recognition result based on the current recognition result to determine the license plate number of the target vehicle.

13. An electronic device comprising a processor and a memory storing a computer program, characterized in that: When the processor executes the computer program, the license plate recognition method according to any one of claims 1 to 11 is implemented.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the license plate recognition method according to any one of claims 1 to 11 is implemented.

Images