A license plate number recognition method and system, and a storage medium

Abstract

The application provides a license plate number recognition method, system and storage medium, and relates to a license plate number recognition method, wherein an edge device first extracts a vehicle image containing a running vehicle in a monitoring area image, and determines a target vehicle corresponding to the image; then, the edge device detects a license plate image in the vehicle image, and caches the license plate image to a cache area corresponding to the target vehicle after optimizing the license plate image; further, the monitoring area image is marked with a plurality of specified positions, and the edge device only performs license plate number recognition on the license plate image of the target vehicle when determining that the target vehicle reaches one of the specified positions, thereby effectively saving the computing resources of the edge device; finally, the edge device performs another round of optimization on the license plate number recognition results of the target vehicle detected at each specified position, thereby improving the accuracy of the license plate number recognition results on the premise of saving the computing resources of the edge device.

Description

Technical Field

[0001] This invention relates to the field of license plate number recognition, and in particular to a license plate number recognition method, system and storage medium. Background Technology

[0002] License plates are crucial vehicle identifiers. In elevated roadside parking management systems, license plates are typically used to match vehicle entry and exit events to form complete orders. Therefore, improving license plate recognition accuracy is essential. In related technologies, license plate recognition is usually processed by edge devices. However, edge devices have limited performance and can only capture images of poor quality, severely impacting the reliability of license plate recognition. Summary of the Invention

[0003] The purpose of this invention is to provide a license plate recognition method, system, and storage medium, wherein the edge device can perform license plate recognition on the optimized license plate image of the target vehicle when it determines that the target vehicle has driven to a designated location in the monitoring area, thereby improving the accuracy of the license plate recognition results while saving the computing resources of the edge device.

[0004] To solve the above-mentioned technical problems, the present invention provides a license plate number recognition method, comprising:

[0005] Edge devices acquire images of the monitored area in real time, detect vehicle images containing moving vehicles in the monitored area images, and use tracking algorithms to determine the target vehicle to which the vehicle image belongs; multiple designated locations are pre-marked in the monitored area images;

[0006] When a license plate image containing a license plate is detected from the vehicle image and it is determined that the license plate image meets the preset preferred conditions, the license plate image is cached in the cache area corresponding to the target vehicle;

[0007] When it is determined that the target vehicle has arrived at a designated location in the monitored area image, license plate number recognition is performed on all license plate images in the cache area, and the recognition result with the highest confidence is taken as the license plate number recognition result of the target vehicle for the current recognition.

[0008] When it is determined that the target vehicle has passed all the specified locations and has the license plate number recognition result, the license plate number recognition result with the highest confidence is selected as the edge side license plate number recognition result of the target vehicle.

[0009] Optionally, after determining the target vehicle to which the vehicle image belongs using a tracking algorithm, the method further includes:

[0010] When the edge device does not detect the license plate image from the vehicle image and determines that the vehicle image meets the preset vehicle image preference conditions, it caches the vehicle image in the cache area.

[0011] Accordingly, before caching the license plate image to the cache area corresponding to the target vehicle, the method further includes:

[0012] Clear the vehicle images in the cache area, and after clearing, perform the step of caching the license plate image to the cache area corresponding to the target vehicle.

[0013] Optionally, determining whether the vehicle image meets preset vehicle image preference conditions includes:

[0014] The positions of multiple specified chassis key points on the target vehicle in the vehicle image are determined sequentially, and the tilt angle of the vehicle image is determined using all the chassis key points.

[0015] When the tilt angle is determined to be less than a preset threshold, the vehicle image is determined to meet the preset vehicle image preference conditions.

[0016] Optionally, after caching the license plate image to the cache area corresponding to the target vehicle, the method further includes:

[0017] When the edge device determines that the target vehicle has passed all the specified locations and does not have the license plate number recognition result, it sends all vehicle images in the cache area to the cloud device.

[0018] The cloud device determines whether the received vehicle image contains the license plate image;

[0019] If so, license plate number recognition is performed on all detected license plate images, the recognition result with the highest confidence is set as the cloud license plate number recognition result, and the cloud license plate number recognition result is sent to the edge device;

[0020] If not, then the target vehicle is determined not to have the license plate.

[0021] Optionally, determining whether the license plate image meets preset preferred conditions includes:

[0022] The edge device calculates the size quality value of the license plate image using the size of the license plate image;

[0023] The positions of multiple specified chassis key points on the target vehicle in the vehicle image are determined sequentially, the tilt angle of the target vehicle is determined using all the chassis key points, and the tilt angle quality value of the license plate image is calculated using the tilt angle.

[0024] The license plate image is input into a neural network model for processing, so that the neural network model can determine the sharpness quality value of the license plate image;

[0025] When the size quality value, the tilt angle quality value, and the sharpness quality value are all greater than the corresponding preset quality thresholds, the license plate image is determined to meet the preset preferred conditions.

[0026] Optionally, before inputting the license plate image into the neural network model for processing, the method further includes:

[0027] The edge device acquires multiple license plate training images; each license plate training image is labeled with a corresponding manual score for clarity.

[0028] The Laplacian operator is used to calculate the value for each license plate training image, and the variance of each license plate training image is calculated using all the calculated values ​​to obtain the quantitative score corresponding to each license plate training image.

[0029] The comprehensive score of each license plate training image is determined by the manual score of clarity and the quantitative score, and the neural network model is trained using each license plate training image and its corresponding comprehensive score.

[0030] The step of inputting the license plate image into the neural network model for processing is performed using the trained neural network model.

[0031] Optionally, caching the license plate image to the cache area corresponding to the target vehicle includes:

[0032] The edge device performs a weighted calculation on the size quality value, tilt angle quality value, and sharpness quality value of the license plate image to obtain the overall quality value of the license plate image;

[0033] Determine whether the number of cached license plate images in the cache area is less than a preset threshold;

[0034] If so, the license plate image is cached in the cache area;

[0035] If not, when it is determined that there is a target cached license plate image with a smaller overall quality value than the license plate image, the target cached license plate image with the smallest overall quality value is removed from the cache area, and the license plate image is cached in the cache area.

[0036] Optionally, after selecting the license plate number recognition result with the highest confidence level as the edge-side license plate number recognition result of the target vehicle, the method further includes:

[0037] The edge device sorts the license plate images in the cache area in descending order of the overall quality value;

[0038] The sorted license plate images of the first preset number and the license plate number recognition results on the edge side are sent to the cloud device;

[0039] The cloud device performs license plate number recognition on the received license plate image to obtain an initial recognition result, determines the proportion of identical initial recognition results among all initial recognition results, and sets the initial recognition result with the highest proportion as the cloud license plate number recognition result;

[0040] When it is determined that the cloud-based license plate recognition result is different from the edge-side license plate recognition result, the cloud-based license plate recognition result is sent to the edge device so as to correct the edge-side license plate recognition result using the cloud-based license plate recognition result.

[0041] The present invention also provides a license plate recognition system, comprising: an edge device, wherein,

[0042] The edge device is used to acquire images of a monitored area in real time, detect vehicle images containing moving vehicles in the monitored area images, and determine the target vehicle to which the vehicle image belongs using a tracking algorithm. Multiple designated locations are pre-marked in the monitored area images. When a license plate image containing a license plate is detected from the vehicle images and it is determined that the license plate image meets preset preferred conditions, the license plate image is cached in the cache area corresponding to the target vehicle. When it is determined that the target vehicle has reached one of the designated locations in the monitored area images, license plate number recognition is performed on all license plate images in the cache area, and the recognition result with the highest confidence is used as the current license plate number recognition result for the target vehicle. When it is determined that the target vehicle has passed all the designated locations and has the license plate number recognition result, the license plate number recognition result with the highest confidence is selected as the edge-side license plate number recognition result for the target vehicle.

[0043] The present invention also provides a storage medium storing computer-executable instructions, which, when loaded and executed by a processor, implement the license plate number recognition method described above.

[0044] This invention provides a license plate recognition method, comprising: an edge device acquiring a monitored area image in real time, detecting vehicle images containing moving vehicles in the monitored area image, and using a tracking algorithm to determine the target vehicle to which the vehicle image belongs; multiple designated locations are pre-marked in the monitored area image; when a license plate image containing a license plate is detected from the vehicle image and it is determined that the license plate image meets preset preferred conditions, the license plate image is cached in a cache area corresponding to the target vehicle; when it is determined that the target vehicle has reached one of the designated locations in the monitored area image, license plate recognition is performed on all license plate images in the cache area, and the recognition result with the highest confidence is taken as the license plate recognition result of the target vehicle for the current recognition; when it is determined that the target vehicle has passed all the designated locations and has the license plate recognition result, the license plate recognition result with the highest confidence is selected as the edge-side license plate recognition result of the target vehicle.

[0045] As can be seen, the edge device in this invention first performs vehicle detection in the monitored area image, extracts vehicle images containing moving vehicles, and uses a tracking algorithm to determine the target vehicle to which the vehicle image belongs. Subsequently, the edge device detects license plate images in the vehicle images, and upon extraction, determines whether the license plate image meets preset optimization conditions. Only when the license plate image meets the preset optimization conditions is the image saved to the cache area corresponding to the target vehicle, thus optimizing the license plate image to improve the accuracy of license plate number recognition. Furthermore, the monitored area image is marked with multiple designated locations. When the target vehicle is determined to have reached a designated location, the edge device will optimize the target vehicle... The selected license plate image is used for license plate recognition. This means the edge device does not perform license plate recognition in real time, but only when the target vehicle reaches a designated location within the monitored area. This effectively saves the edge device's computing resources. Finally, after confirming that the target vehicle has passed all designated locations, the edge device selects the result with the highest confidence from all license plate recognition results for the target vehicle as the edge-side license plate recognition result. This further optimizes the license plate recognition result obtained from the selected license plate image, thereby improving the accuracy of the license plate recognition result while saving edge device computing resources. This invention also provides a license plate recognition system and storage medium, which have the above-mentioned beneficial effects. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0047] Figure 1 This is a flowchart of a license plate number recognition method provided in an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of an early warning wireframe provided in an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of key points of a chassis provided in an embodiment of the present invention;

[0050] Figure 4 This is a structural block diagram of a license plate recognition system provided in an embodiment of the present invention;

[0051] Figure 5 A flowchart of another license plate recognition method provided in an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] License plates are crucial vehicle identifiers. In elevated roadside parking management systems, license plates are typically used to match vehicle entry and exit events to form complete orders. Therefore, improving license plate recognition accuracy is essential. In related technologies, license plate recognition is usually processed by edge devices. However, edge devices have limited performance and can only capture images of poor quality, severely impacting the reliability of license plate recognition. In view of this, the present invention provides a license plate recognition method in which the edge device performs license plate recognition on a optimized image of the target vehicle only after determining that the target vehicle has reached a designated location within the monitored area. This improves the accuracy of license plate recognition results while saving the computational resources of the edge device. Please refer to [reference needed]. Figure 1 , Figure 1 This is a flowchart of a license plate number recognition method provided in an embodiment of the present invention. The method may include:

[0054] S101. The edge device acquires images of the monitored area in real time, detects vehicle images containing moving vehicles in the monitored area images, and uses a tracking algorithm to determine the target vehicle to which the vehicle image belongs; multiple designated locations are pre-marked in the monitored area images.

[0055] In this embodiment of the invention, the edge device first acquires an image of the monitored area and detects whether the image contains a vehicle image corresponding to a moving vehicle. If it does, the device further determines the target vehicle to which the vehicle image belongs, so that the image to be processed and the processing result can be subsequently bound to the target vehicle. It is understood that vehicle image detection needs to be completed by an object detection model (such as a vehicle detection model). Specifically, the image refers to the image enclosed by the detection box generated by the object detection model in the monitored area image, and the model is specifically a neural network model. This embodiment of the invention does not limit the specific object detection model; for example, it can be SSD (Single Shot MultiBox Detector), YOLO (You only look once) series, etc. Furthermore, it is worth noting that this embodiment of the invention mainly performs license plate detection and license plate number recognition on vehicles in motion. In other words, when the edge device determines that a vehicle detected in the monitored area is entering or leaving a parking space, it will automatically perform license plate detection and license plate number recognition on that vehicle. It should be noted that vehicle driving status detection can generally be achieved through tracking algorithms. These algorithms track the same target detected in multiple consecutive frames to determine its motion state. The basic principle of this algorithm is as follows: the vehicle detection box extracted from the initial image is used as the initial target for tracking. Then, the vehicle detection box obtained in the next frame is compared with all vehicle detection boxes obtained in the previous frame. If the overlap is greater than a set threshold, it is considered to be the same vehicle; otherwise, the remaining vehicle detection boxes are traversed. If a vehicle detection box ultimately does not match any target, it is considered a new target. If no match is found for a preset number of consecutive frames (e.g., 10 frames), the vehicle is considered lost. This embodiment of the invention does not limit the specific method of using tracking algorithms to detect vehicles entering and leaving parking spaces; relevant technologies for vehicle driving detection can be referenced. This embodiment of the invention also does not limit the specific tracking algorithm used; relevant technologies can be referenced and selected as needed.

[0056] Furthermore, it should be noted that multiple designated locations are pre-marked in the monitored area image. These designated locations are used to trigger the edge device to perform license plate recognition. Specifically, when it is determined that the target vehicle detected in the monitored area image is located at the aforementioned designated location, the edge device will perform license plate recognition on the target vehicle. In other words, the edge device in this embodiment of the invention does not perform license plate recognition on the target vehicle in real time, but only when it is determined that the target vehicle has driven to the designated location in the monitored area. This can significantly reduce the consumption of computing resources on the edge device, thereby improving the operating efficiency of the device. It should be noted that this embodiment of the invention does not limit the specific setting method of the aforementioned designated locations, and can be set according to actual application needs. In one possible case, considering that the vehicle will drive into or out of the parking space, the aforementioned designated locations can be set based on the parking space area. For example, the parking space area is marked in the monitored area image, and a warning line area is obtained by extending the parking space area outward by a preset distance, such as... Figure 2 As shown. Subsequently, the frame line of the warning line area can be set to the specified position mentioned above. When the edge device determines that a target vehicle has come into contact with any frame line of the warning line area, it will identify the license plate number of the target vehicle. Of course, the warning line area can be further flexibly set. For example, a first warning line area can be obtained by expanding the parking space area by a preset distance, a second warning line area can be obtained by expanding the first warning line area by a preset distance, and so on, with the frame line of each warning line area set to the specified position mentioned above. It should be noted that the embodiments of the present invention do not limit the specific preset distance value on which the expansion processing is based, and can be set according to actual application requirements.

[0057] S102. When a license plate image containing a license plate is detected from a vehicle image and it is determined that the license plate image meets the preset preferred conditions, the license plate image is cached in the cache area corresponding to the target vehicle.

[0058] It should be specifically pointed out that "license plate detection" and "license plate number recognition" in this embodiment of the invention are two different operations. "License plate detection" specifically detects the "license plate" object, and its detection method is consistent with that of objects such as "vehicles," i.e., it uses an object detection model for detection. "License plate number recognition," on the other hand, specifically identifies the license plate number characters in the license plate image. It can be understood that the license plate image refers to the image enclosed by the detection box generated by the object detection model in the vehicle image. Obviously, the size of the license plate image will be much smaller than the vehicle image and the monitored area image. Therefore, the quality of the license plate image is easily affected by external factors such as distance and shooting angle, which can easily interfere with the license plate number recognition effect. Therefore, this embodiment of the invention also optimizes the license plate image to ensure that only the optimized license plate image is used for license plate number recognition, thereby guaranteeing the recognition effect. Specifically, when a license plate image containing a license plate is detected from the vehicle image, this embodiment of the invention also needs to determine whether the license plate image meets preset optimization conditions, and only caches it in the cache area corresponding to the target vehicle if the conditions are met.

[0059] It should be noted that the embodiments of the present invention do not limit specific preset preferred conditions, which can be set according to actual application needs. For example, the size of the license plate image, the tilt angle of the license plate in the image, and the image clarity are all important factors affecting the license plate recognition effect. Therefore, corresponding size quality values, tilt angle quality values, and clarity quality values ​​can be calculated for the license plate image, and corresponding preset thresholds can be set for the above quality values. Only when the size quality value, tilt angle quality value, and clarity quality value of a license plate image are all greater than the corresponding preset thresholds is the license plate image determined to meet the preset preferred conditions. The embodiments of the present invention do not limit the specific values ​​of each preset threshold, which can be set according to actual application needs.

[0060] Furthermore, it is understood that the size and tilt angle of the license plate image are easily quantifiable and analyzed. The corresponding quality value can be calculated based on the specific size and tilt angle, and the specific calculation method can be set according to actual application requirements. This embodiment of the invention does not limit the specific method for determining the license plate tilt angle. For example, the angle between the license plate border and the horizontal or vertical direction can be determined and used as the tilt angle; alternatively, the tilt angle of the vehicle in the vehicle image can be determined first, and then used as the tilt angle of the license plate. Considering that the vehicle image is relatively large and contains many features, determining the vehicle's tilt angle in the vehicle image is relatively easy. Therefore, this embodiment of the invention can use the vehicle tilt angle as the license plate tilt angle. Specifically, when performing vehicle image detection, a vehicle detection model can also be used to sequentially determine the positions of multiple chassis key points on the vehicle chassis, and the vehicle tilt angle can be determined based on these chassis key points. For example, please refer to... Figure 3 , Figure 3This is a schematic diagram of key chassis points provided in an embodiment of the present invention, where A, B, C, and D represent the left rear wheel, left front wheel, right front wheel, and right rear wheel of the vehicle, respectively. A straight line passing through points A and D (i.e., the two rear wheels) can be determined, and the angle between this straight line and the horizontal frame of the vehicle detection frame is taken as the vehicle's tilt angle. Of course, the vehicle's tilt angle can also be determined based on other key chassis points, and can be set according to actual application requirements.

[0061] Furthermore, considering the difficulty in quantifying and analyzing sharpness, this embodiment of the invention employs a neural network model to calculate the corresponding sharpness quality value for each license plate image. Specifically, a neural network model can be trained using license plate training images with different degrees of blurriness and labeled with scores. This allows the model to learn the correlation between various blurriness features and scores. The trained neural network model is then used to calculate the sharpness quality value for each license plate image, and the quality of the license plate image is determined by combining these sharpness quality values. It should be noted that this embodiment of the invention is not limited to the specific method of labeling scores on license plate training images. For example, scores can be manually labeled based on subjective feelings and given scoring standards. Alternatively, the scores for each license plate training image can be determined based on the feature differences between blurry and clear images, combined with specific operators. For instance, blurry images have less boundary information, while clear images have more boundary information. The Laplacian operator has second-order differentiability, which allows for the calculation of regions with rapid density changes in the image. Therefore, the Laplacian operator is calculated for the image, and the variance is calculated. The resulting score is then used as the score for the license plate training image. Of course, the scores given manually and those calculated using the Laplacian operator can be combined to evaluate the clarity of the license plate training images. The manual scores can be used as qualitative scores, and the scores calculated using the Laplacian operator can be used as quantitative scores.

[0062] In one possible scenario, determining whether a license plate image meets preset preferred conditions includes:

[0063] Step 11: The edge device calculates the size quality value of the license plate image using its dimensions;

[0064] Step 12: Sequentially determine the positions of multiple specified chassis key points on the target vehicle in the vehicle image, use all chassis key points to determine the tilt angle of the target vehicle, and use the tilt angle to calculate the tilt angle quality value of the license plate image.

[0065] Step 13: Input the license plate image into the neural network model for processing so that the neural network model can determine the sharpness quality value of the license plate image;

[0066] Step 14: When the size quality value, tilt angle quality value and sharpness quality value are all greater than the corresponding preset quality threshold, the license plate image is determined to meet the preset preferred conditions.

[0067] In one possible scenario, before inputting the license plate image into the neural network model for processing, the following may also be included:

[0068] Step 21: The edge device acquires multiple license plate training images; each license plate training image is labeled with a corresponding manual score for clarity;

[0069] Step 22: Perform Laplacian operator calculations on each license plate training image to obtain the calculated value, and use all the calculated values ​​to calculate the variance of each license plate training image to obtain the quantitative score corresponding to each license plate training image.

[0070] Step 23: Determine the comprehensive score of each license plate training image using the manual score and quantitative score, and train the neural network model using each license plate training image and its corresponding comprehensive score;

[0071] Step 24: Use the trained neural network model to perform the step of inputting the license plate image into the neural network model for processing.

[0072] It should be noted that the embodiments of the present invention do not limit the scoring process of the manual definition score. For example, a threshold range of 1 to 5 can be set, with a larger number indicating a clearer image. The annotator can select an appropriate value within the threshold range based on subjective judgment to annotate the license plate training image. Of course, other methods can also be used to annotate the manual definition score. Furthermore, the quantitative score obtained by the Laplacian operator can be calculated using the following formula:

[0073] D(f)=∑ y ∑ x |f(x+2,y)-f(x,y)| 2

[0074] Where f(x,y) represents the gray value of the pixel (x,y) corresponding to image f, and D(f) is the result of the image sharpness calculation.

[0075] Furthermore, after determining that the license plate image meets the preset optimization conditions, the image can be cached in the cache area corresponding to the target vehicle in the edge device. Of course, to further reduce performance loss on the edge device and further optimize the license plate image, only a limited number of license plate images are stored in the cache area, and only the highest quality license plate images are stored. Considering that the above embodiments calculate the corresponding size quality value, tilt angle quality value, and sharpness quality value for each license plate image, these three values ​​can be weighted to obtain the comprehensive quality value corresponding to the license plate image. Subsequently, when it is determined that the cache area of ​​the target vehicle is full, the comprehensive quality value of the license plate image can be compared with the comprehensive quality values ​​of each cached license plate image. If the comprehensive quality value of the license plate image is lower than the comprehensive quality value of each cached license plate image, the license plate image can be directly discarded; conversely, if there is a target cached license plate image with a comprehensive quality value lower than the target cached license plate image, the target cached license plate image can be moved out of the cache area, and the current license plate image can be cached, thereby achieving a further optimization effect.

[0076] In one possible scenario, caching the license plate image to the cache area corresponding to the target vehicle may include:

[0077] Step 31: The edge device performs a weighted calculation on the size quality value, tilt angle quality value, and sharpness quality value of the license plate image to obtain the overall quality value of the license plate image;

[0078] Step 32: Determine whether the number of cached license plate images in the cache area is less than a preset threshold; if yes, proceed to step 33; if no, proceed to step 34.

[0079] Step 33: Cache the license plate image to the cache area;

[0080] Step 34: When it is determined that there is a target cached license plate image with a smaller overall quality value than the license plate image, the target cached license plate image with the smallest overall quality value is removed from the cache area, and the license plate image is cached in the cache area.

[0081] It should be noted that the embodiments of the present invention do not limit the maximum number of license plate images that can be cached in the cache area of ​​the target vehicle, and can be set according to actual application needs.

[0082] S103. When it is determined that the target vehicle has arrived at a specified location in the monitored area image, the license plate number is identified in all the license plate images in the cache area, and the identification result with the highest confidence is taken as the license plate number identification result of the target vehicle for the current identification.

[0083] As described above, when it is determined that the target vehicle has reached a designated location within the monitored area, such as having touched a warning zone boundary, license plate recognition can be performed on all license plate images within the vehicle's buffer area. It is understood that these license plate recognition results may not be identical. Therefore, in this embodiment of the invention, the recognition result with the highest confidence level can be selected as the current license plate recognition result for that vehicle, thus choosing the most feasible result from several options. It should be noted that this embodiment of the invention does not limit the specific process of license plate recognition; reference can be made to related technologies for text recognition.

[0084] S104. When it is determined that the target vehicle has passed all the specified locations and has license plate recognition results, select the license plate recognition result with the highest confidence as the edge side license plate recognition result of the target vehicle.

[0085] Assuming the license plate of the target vehicle can be continuously identified, since the license plate number is recognized every time the vehicle reaches a designated location, there will be multiple license plate number recognition results after the vehicle has passed all designated locations. At this point, the embodiment of the present invention can perform another round of optimization on these results, selecting the license plate number recognition result with the highest confidence as the final recognition result of the target vehicle at the edge device (i.e., the edge-side license plate number recognition result). Furthermore, after multiple rounds of optimization, the edge device in the embodiment of the present invention can not only generate high-quality license plate number recognition results, but also save a significant amount of computing resources.

[0086] Of course, considering that the models deployed in edge devices are usually lightweight models, which have lower accuracy compared to complex models, in order to ensure the validity of the license plate recognition results, in this embodiment of the invention, after generating the edge-side license plate recognition result, the edge device can also send this result and the previously cached license plate images to the cloud device. The cloud device then uses a more complex model to verify this result, thereby avoiding the adverse effects of incorrect recognition results generated by the edge device on parking management. It should be noted that this embodiment of the invention does not limit whether the edge device needs to send all license plate images of the target vehicle to the cloud device, or only send a preset number of license plate images of the best quality to the cloud device. To avoid consuming a large amount of network transmission resources, in this embodiment of the invention, the edge device can send a preset number of license plate images of the best quality to the cloud device. Specifically, the comprehensive quality value calculated in the above embodiment can be used to sort all license plate images in the target vehicle's cache area in descending order, and the first preset number of license plate images in the sorted sequence can be selected and sent to the cloud device.

[0087] Furthermore, it's understandable that the cloud device also performs license plate recognition on the received license plate image, generating a cloud-based license plate recognition result. This result is then compared with the received edge-side license plate recognition result. If a difference is found, the cloud device can send the cloud-based license plate recognition result to the edge device to correct the edge-side recognition result. Similarly, it's understandable that the initial recognition results generated by the cloud device for each license plate image may differ. To ensure the reliability and effectiveness of the cloud-based license plate recognition result, the cloud device can calculate the percentage of identical initial recognition results among all results and use the initial recognition result with the highest percentage as the cloud-based license plate recognition result.

[0088] In one possible scenario, after selecting the license plate number recognition result with the highest confidence level as the edge-side license plate number recognition result of the target vehicle, it may also include:

[0089] Step 51: The edge device sorts the license plate images in the cache area in descending order of their overall quality scores;

[0090] Step 52: Send the sorted first preset number of license plate images and the edge side license plate number recognition results to the cloud device;

[0091] Step 53: The cloud device performs license plate number recognition on the received license plate image to obtain the initial recognition result, determines the proportion of the same initial recognition result among all initial recognition results, and sets the initial recognition result with the highest proportion as the cloud license plate number recognition result;

[0092] Step 54: When it is determined that the license plate recognition result in the cloud is different from the license plate recognition result on the edge, the license plate recognition result in the cloud is sent to the edge device so as to correct the license plate recognition result on the edge using the license plate recognition result in the cloud.

[0093] Based on the above embodiments, the edge device in this invention first performs vehicle detection in the monitored area image, extracts vehicle images containing moving vehicles, and uses a tracking algorithm to determine the target vehicle to which the vehicle image belongs. Subsequently, the edge device detects license plate images in the vehicle images, and upon extraction, determines whether the license plate image meets preset optimization conditions. Only when the license plate image meets the preset optimization conditions is the image saved to the cache area corresponding to the target vehicle, thus optimizing the license plate image and improving the accuracy of license plate number recognition. Furthermore, the monitored area image is marked with multiple designated locations. When the target vehicle is determined to have reached a designated location, the edge device will track the target vehicle... The edge device performs license plate recognition on the optimized license plate image. This means that the edge device does not perform license plate recognition in real time, but only when it determines that the target vehicle has reached a designated location in the monitoring area. This effectively saves the computing resources of the edge device. Finally, after determining that the target vehicle has passed all designated locations, the edge device will select the result with the highest confidence from all the license plate recognition results of the target vehicle as the edge-side license plate recognition result of the target vehicle. This further optimizes the license plate recognition result obtained from the optimized license plate image, thereby improving the accuracy of the license plate recognition result while saving the computing resources of the edge device.

[0094] Based on the above embodiments, considering factors such as occlusion and the absence of license plates, the edge device may be unable to detect the license plate, and thus unable to identify the license plate number of the target vehicle. In this case, to avoid the impact of the inability to identify the license plate number on parking management, the edge device also needs to cache the vehicle image corresponding to the target vehicle, so as to use the vehicle image as a credential for subsequent management. The following details the processing method for the edge device when it cannot identify the license plate number. In one possible scenario, after determining the target vehicle to which the vehicle image belongs using a tracking algorithm, the following may also be included:

[0095] S201. When the edge device does not detect a license plate image from the vehicle image and determines that the vehicle image meets the preset vehicle image preference conditions, it caches the vehicle image to the cache area.

[0096] In this embodiment of the invention, if the edge device fails to detect the license plate image from the vehicle image, the vehicle image also needs to be cached to serve as a credential for parking management. Similar to caching the license plate image, the vehicle image can only be cached if it meets preset preferred conditions. This embodiment of the invention does not limit the specific preset vehicle image preferred conditions; they can be set according to actual application needs. Considering the large size of the vehicle image and the need to ensure sufficient clarity, in this embodiment of the invention, only the tilt angle of the vehicle in the vehicle image is used as an indicator to judge the quality of the vehicle image. Consistent with the method for determining the license plate tilt angle, this embodiment of the invention also determines the vehicle's tilt angle based on multiple specified chassis key points on the target vehicle (such as key points located on each wheel).

[0097] In one possible scenario, determining whether a vehicle image meets preset vehicle image preference criteria may include:

[0098] Step 61: Sequentially determine the positions of multiple specified chassis key points on the target vehicle in the vehicle image, and use all chassis key points to determine the tilt angle of the vehicle image.

[0099] Step 62: When the tilt angle is less than the preset threshold, determine that the vehicle image meets the preset vehicle image preference conditions.

[0100] It should be noted that the embodiments of the present invention do not limit the specific value of the preset threshold, and can be set according to actual application needs. Of course, a limited number of vehicle images can also be stored in the cache area of ​​the target vehicle, and only the vehicle image with the smallest tilt angle can be retained in the cache area for management purposes.

[0101] Accordingly, before caching the license plate image to the cache area corresponding to the target vehicle, the following steps are also included:

[0102] S202. Clear the vehicle images in the cache area, and after clearing, perform the step of caching the license plate image to the cache area corresponding to the target vehicle.

[0103] Understandably, if the license plate image of the target vehicle can be identified again, the edge device can continue processing according to the license plate recognition procedure without retaining the previously cached vehicle image, and can clear the vehicle image in the cache area. Of course, if the edge device fails to detect the license plate of the target vehicle, the vehicle image can be retained.

[0104] Furthermore, considering that the inability to detect license plates may be due to the weak detection capabilities of the edge device, the edge device can send the cached vehicle images to the cloud device for verification after determining that the target vehicle has passed all designated locations. This embodiment of the invention does not limit whether the edge device needs to send all vehicle images of the target vehicle to the cloud device, or only a preset number of the best-quality vehicle images; the choice can be made according to actual application needs.

[0105] Furthermore, it is understandable that the cloud device will also perform license plate image detection on the received vehicle images. If the cloud device determines that a license plate image exists in the vehicle image, it will further perform license plate number recognition, set the recognition result with the highest confidence as the cloud license plate number recognition result, and send the cloud license plate number recognition result to the edge device for correction; if the cloud device determines that a license plate image does not exist in the vehicle image, it can directly determine that the target vehicle is a vehicle without a license plate.

[0106] In one possible scenario, after caching the license plate image to the cache area corresponding to the target vehicle, the following may also be included:

[0107] Step 71: When the edge device determines that the target vehicle has passed all the specified locations and there is no license plate recognition result, it sends all vehicle images in the cache area to the cloud device.

[0108] Step 72: The cloud device determines whether the received vehicle image contains a license plate image; if yes, proceed to step 73; if no, proceed to step 74.

[0109] Step 73: Perform license plate number recognition on all detected license plate images, set the recognition result with the highest confidence as the cloud license plate number recognition result, and send the cloud license plate number recognition result to the edge device;

[0110] Step 74: Determine that the target vehicle does not have a license plate.

[0111] The license plate recognition system and storage medium provided in the embodiments of the present invention are described below. The license plate recognition system and storage medium described below can be referred to in correspondence with the license plate recognition method described above.

[0112] Please refer to Figure 4 , Figure 4 This is a structural block diagram of a license plate recognition system provided in an embodiment of the present invention. The system may first include:

[0113] Edge device 401 is used to acquire images of a monitored area in real time, detect vehicle images containing moving vehicles in the monitored area images, and use a tracking algorithm to determine the target vehicle to which the vehicle image belongs. Multiple designated locations are pre-marked in the monitored area images. When a license plate image containing a license plate is detected from a vehicle image and it is determined that the license plate image meets the preset preferred conditions, the license plate image is cached in the cache area corresponding to the target vehicle. When it is determined that the target vehicle has reached a designated location in the monitored area images, license plate number recognition is performed on all license plate images in the cache area, and the recognition result with the highest confidence is used as the license plate number recognition result of the target vehicle for the current recognition. When it is determined that the target vehicle has passed all designated locations and has license plate number recognition results, the license plate number recognition result with the highest confidence is selected as the edge-side license plate number recognition result of the target vehicle.

[0114] Optionally, the edge device 401 is further configured to cache the vehicle image in a cache area after determining the target vehicle to which the vehicle image belongs using a tracking algorithm, when no license plate image is detected from the vehicle image and the vehicle image is determined to meet a preset vehicle image preference condition.

[0115] Correspondingly, the edge device 401 is also used to clear the vehicle image in the cache area before caching the license plate image to the cache area corresponding to the target vehicle, and to perform the step of caching the license plate image to the cache area corresponding to the target vehicle after clearing.

[0116] Optionally, the edge device 401 is further configured to sequentially determine the positions of multiple specified chassis key points on the target vehicle in the vehicle image, and to determine the tilt angle of the vehicle image using all chassis key points; when the tilt angle is determined to be less than a preset threshold, the vehicle image is determined to meet preset vehicle image preference conditions.

[0117] Optionally, the system may also include a cloud device 402, wherein,

[0118] Edge device 401 is also used to send all vehicle images in the cache area to cloud device 402 after caching the license plate image to the cache area corresponding to the target vehicle and determining that the target vehicle has passed all the specified locations and there is no license plate recognition result.

[0119] The cloud device 402 is used to determine whether the received vehicle image contains a license plate image; if so, it performs license plate number recognition on all detected license plate images, sets the recognition result with the highest confidence as the cloud license plate number recognition result, and sends the cloud license plate number recognition result to the edge device 401; if not, it determines that the target vehicle does not have a license plate.

[0120] Optionally, the edge device 401 is further configured to calculate the size quality value of the license plate image using the size of the license plate image; sequentially determine the positions of multiple specified chassis key points on the target vehicle in the vehicle image, determine the tilt angle of the target vehicle using all chassis key points, and calculate the tilt angle quality value of the license plate image using the tilt angle; input the license plate image into a neural network model for processing, so that the neural network model determines the sharpness quality value of the license plate image; when the size quality value, tilt angle quality value, and sharpness quality value are all greater than the corresponding preset quality thresholds, the license plate image is determined to meet the preset preferred conditions.

[0121] Optionally, the edge device 401 is further configured to acquire multiple license plate training images before inputting the license plate image into the neural network model for processing; each license plate training image is labeled with a corresponding manual score for clarity; a calculated value is obtained by performing a Laplacian operator calculation on each license plate training image, and the variance of each license plate training image is calculated using all the calculated values ​​to obtain a quantitative score corresponding to each license plate training image; a comprehensive score for each license plate training image is determined using the manual score for clarity and the quantitative score, and a neural network model is trained using each license plate training image and its corresponding comprehensive score; and the trained neural network model is used to perform the step of inputting the license plate image into the neural network model for processing.

[0122] Optionally, the edge device 401 is further configured to perform weighted calculations on the size quality value, tilt angle quality value, and sharpness quality value of the license plate image to obtain the comprehensive quality value of the license plate image; determine whether the number of cached license plate images in the cache area is less than a preset threshold; if so, cache the license plate image to the cache area; if not, when it is determined that there is a target cached license plate image with a smaller comprehensive quality value than the license plate image, remove the target cached license plate image with the smallest comprehensive quality value from the cache area and cache the license plate image to the cache area.

[0123] Optionally, the edge device 401 is further configured to, after selecting the license plate recognition result with the highest confidence as the edge-side license plate recognition result of the target vehicle, sort the license plate images in the cache area in descending order of comprehensive quality value; and send the sorted first preset number of license plate images and the edge-side license plate recognition results to the cloud device 402.

[0124] The cloud device 402 is also used to perform license plate number recognition on the received license plate image to obtain an initial recognition result, determine the proportion of the same initial recognition result among all initial recognition results, and set the initial recognition result with the highest proportion as the cloud license plate number recognition result; when it is determined that the cloud license plate number recognition result is different from the edge-side license plate number recognition result, the cloud license plate number recognition result is sent to the edge device 401 so as to use the cloud license plate number recognition result to correct the edge-side license plate number recognition result.

[0125] For easier understanding, please refer to Figure 5 , Figure 5 for Figure 5 This is a flowchart of another license plate recognition method provided by an embodiment of the present invention. The functions and interaction content of the edge device and the cloud device are marked in the figure.

[0126] This invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the license plate number recognition method described in any of the above embodiments.

[0127] Since the embodiments of the storage medium section correspond to the embodiments of the license plate number recognition method section, please refer to the description of the embodiments of the license plate number recognition method section for the embodiments of the storage medium section, and will not be repeated here.

[0128] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0129] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0130] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0131] The present invention has provided a detailed description of a license plate recognition method, system, and storage medium. Specific examples have been used to illustrate the principles and implementation methods of the invention. The descriptions of these embodiments are merely illustrative and are intended to aid in understanding the method and its core concepts. It should be noted that those skilled in the art can make various improvements and modifications to the invention without departing from its principles, and these improvements and modifications also fall within the scope of protection of the claims.

Claims

1. A method for recognizing vehicle license plates, characterized in that, include: Edge devices acquire images of the monitored area in real time, detect vehicle images containing moving vehicles in the monitored area images, and use tracking algorithms to determine the target vehicle to which the vehicle image belongs. The monitored area image has multiple designated locations pre-marked; When a license plate image containing a license plate is detected from the vehicle image and it is determined that the license plate image meets the preset preferred conditions, the license plate image is cached in the cache area corresponding to the target vehicle; When it is determined that the target vehicle has arrived at a designated location in the monitored area image, license plate number recognition is performed on all license plate images in the cache area, and the recognition result with the highest confidence is taken as the license plate number recognition result of the target vehicle for the current recognition. When it is determined that the target vehicle has passed all the specified locations and has the license plate number recognition result, the license plate number recognition result with the highest confidence is selected as the edge side license plate number recognition result of the target vehicle. Determining whether the license plate image meets preset preferred conditions includes: The edge device calculates the size quality value of the license plate image using the size of the license plate image; The positions of multiple specified chassis key points on the target vehicle in the vehicle image are determined sequentially, the tilt angle of the target vehicle is determined using all the chassis key points, and the tilt angle quality value of the license plate image is calculated using the tilt angle. The license plate image is input into a neural network model for processing, so that the neural network model can determine the sharpness quality value of the license plate image; wherein, the neural network model is trained using license plate training images with different degrees of blur and labeled with sharpness scores, so that the neural network model can learn the correlation between various blur features and sharpness scores; When the size quality value, the tilt angle quality value, and the sharpness quality value are all greater than the corresponding preset quality thresholds, the license plate image is determined to meet the preset preferred conditions.

2. The license plate recognition method according to claim 1, characterized in that, After determining the target vehicle to which the vehicle image belongs using a tracking algorithm, the process further includes: When the edge device does not detect the license plate image from the vehicle image and determines that the vehicle image meets the preset vehicle image preference conditions, it caches the vehicle image in the cache area. Accordingly, before caching the license plate image to the cache area corresponding to the target vehicle, the method further includes: Clear the vehicle images in the cache area, and after clearing, perform the step of caching the license plate image to the cache area corresponding to the target vehicle.

3. The license plate recognition method according to claim 2, characterized in that, Determining whether the vehicle image meets preset vehicle image preference conditions includes: The positions of multiple specified chassis key points on the target vehicle in the vehicle image are determined sequentially, and the tilt angle of the vehicle image is determined using all the chassis key points. When the tilt angle is determined to be less than a preset threshold, the vehicle image is determined to meet the preset vehicle image preference conditions.

4. The license plate recognition method according to claim 2, characterized in that, After caching the license plate image to the cache area corresponding to the target vehicle, the method further includes: When the edge device determines that the target vehicle has passed all the specified locations and does not have the license plate number recognition result, it sends all vehicle images in the cache area to the cloud device. The cloud device determines whether the received vehicle image contains the license plate image; If so, license plate number recognition is performed on all detected license plate images, the recognition result with the highest confidence is set as the cloud license plate number recognition result, and the cloud license plate number recognition result is sent to the edge device; If not, then the target vehicle is determined not to have the license plate.

5. The license plate recognition method according to claim 1, characterized in that, Before inputting the license plate image into the neural network model for processing, the process also includes: The edge device acquires multiple license plate training images; each license plate training image is labeled with a corresponding manual score for clarity. The Laplacian operator is used to calculate the value for each license plate training image, and the variance of each license plate training image is calculated using all the calculated values ​​to obtain the quantitative score corresponding to each license plate training image. The comprehensive score of each license plate training image is determined by the manual score of clarity and the quantitative score, and the neural network model is trained using each license plate training image and its corresponding comprehensive score. The step of inputting the license plate image into the neural network model for processing is performed using the trained neural network model.

6. The license plate recognition method according to claim 1, characterized in that, The step of caching the license plate image to the cache area corresponding to the target vehicle includes: The edge device performs a weighted calculation on the size quality value, tilt angle quality value, and sharpness quality value of the license plate image to obtain the overall quality value of the license plate image; Determine whether the number of cached license plate images in the cache area is less than a preset threshold; If so, the license plate image is cached in the cache area; If not, when it is determined that there is a target cached license plate image with a smaller overall quality value than the license plate image, the target cached license plate image with the smallest overall quality value is removed from the cache area, and the license plate image is cached in the cache area.

7. The license plate recognition method according to claim 6, characterized in that, After selecting the license plate number recognition result with the highest confidence as the edge-side license plate number recognition result of the target vehicle, the following steps are also included: The edge device sorts the license plate images in the cache area in descending order of the overall quality value; The sorted license plate images of the first preset number and the license plate number recognition results on the edge side are sent to the cloud device; The cloud device performs license plate number recognition on the received license plate image to obtain an initial recognition result, determines the proportion of identical initial recognition results among all initial recognition results, and sets the initial recognition result with the highest proportion as the cloud license plate number recognition result; When it is determined that the cloud-based license plate recognition result is different from the edge-side license plate recognition result, the cloud-based license plate recognition result is sent to the edge device so as to correct the edge-side license plate recognition result using the cloud-based license plate recognition result.

8. A license plate recognition system, characterized in that, include: Edge devices, among which, The edge device is used to acquire images of a monitored area in real time, detect vehicle images containing moving vehicles in the monitored area images, and determine the target vehicle to which the vehicle image belongs using a tracking algorithm. Multiple designated locations are pre-marked in the monitored area images. When a license plate image containing a license plate is detected from the vehicle images and it is determined that the license plate image meets preset preferred conditions, the license plate image is cached in the cache area corresponding to the target vehicle. When it is determined that the target vehicle has reached one of the designated locations in the monitored area images, license plate number recognition is performed on all license plate images in the cache area, and the recognition result with the highest confidence is used as the current license plate number recognition result for the target vehicle. When it is determined that the target vehicle has passed all the designated locations and has the license plate number recognition result, the license plate number recognition result with the highest confidence is selected as the edge-side license plate number recognition result for the target vehicle. The edge device is further configured to: calculate the size quality value of the license plate image using its dimensions; sequentially determine the positions of multiple specified chassis key points on the target vehicle in the vehicle image; determine the tilt angle of the target vehicle using all the chassis key points; and calculate the tilt angle quality value of the license plate image using the tilt angle. The license plate image is then input into a neural network model for processing, so that the neural network model determines the sharpness quality value of the license plate image. Specifically, the neural network model is trained using license plate training images labeled with sharpness scores and varying degrees of blur, so that the neural network model learns the correlation between various blur features and sharpness scores. When the size quality value, the tilt angle quality value, and the sharpness quality value are all greater than the corresponding preset quality thresholds, the license plate image is determined to meet the preset preferred conditions.

9. A storage medium, characterized in that, The storage medium stores computer-executable instructions, which, when loaded and executed by a processor, implement the license plate number recognition method as described in any one of claims 1 to 7.

Images