A license plate recognition method, device, equipment and computer storage medium

Abstract

The application provides a license plate recognition method, device and equipment and a computer storage medium. The method comprises the following steps: performing vehicle rough recognition on a current image frame, performing vehicle tracking according to a recognition result, determining an effective vehicle appearing in the current frame according to a vehicle tracking result, a vehicle rough recognition result and an effective vehicle screening condition, selecting at least one effective vehicle matching a to-be-scheduled resource as a to-be-scheduled vehicle according to a vehicle scheduling priority of the effective vehicle appearing in the current frame and the to-be-scheduled resource, performing license plate frame detection on the to-be-scheduled vehicle again, performing license plate recognition on the license plate frame detected again, associating the license plate frame detected again with the recognized license plate and the corresponding to-be-scheduled vehicle, and obtaining a license plate recognition result, determining a target vehicle according to the license plate recognition result, and scheduling the target vehicle by using the to-be-scheduled resource. The application can guarantee that the accuracy of license plate recognition is improved in a natural scene, and important technical support is provided for tracking vehicles.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a license plate recognition method and device, equipment and a computer storage medium. BACKGROUND

[0002] In a video monitoring scene, a license plate recognition technology of a vehicle is an important technical means for tracing a vehicle, can recognize and save the content in the license plate, and provides important technical support for a technical personnel when tracing the vehicle; however, in a natural scene, due to factors such as a vehicle angle, illumination, and device performance, there is a high probability that the recognition is inaccurate, which affects the work, especially when a law enforcement personnel traces a vehicle illegal behavior, the vehicle may be wrongly or unrecognized, which greatly affects the work. Based on the above problems, a license plate recognition method is urgently needed to recognize the content in the license plate as much as possible in a natural environment. SUMMARY

[0003] The present application provides a license plate recognition method, device, equipment and computer storage medium, which can ensure that the accuracy of license plate recognition is improved in a natural scene, and important technical support is provided for tracing a vehicle.

[0004] In a first aspect, the present application provides a license plate recognition method, comprising:

[0005] obtaining a current image frame from a video stream, performing vehicle coarse recognition on the current image frame, performing vehicle tracking according to a recognition result, the vehicle coarse recognition comprising vehicle frame detection and license plate frame detection;

[0006] determining an effective vehicle appearing in a current frame according to a vehicle tracking result, a vehicle coarse recognition result and an effective vehicle screening condition;

[0007] selecting at least one effective vehicle matching the to-be-scheduled resource as a to-be-scheduled vehicle according to a vehicle scheduling priority of the effective vehicle appearing in the current frame and the to-be-scheduled resource;

[0008] performing license plate frame detection on a position region of the vehicle frame of the to-be-scheduled vehicle again, performing license plate recognition on the license plate frame detected again, associating the license plate frame detected again with the recognized license plate and the corresponding to-be-scheduled vehicle, and obtaining a license plate recognition result;

[0009] determining a target vehicle according to the license plate recognition result, and scheduling the target vehicle by using the to-be-scheduled resource.

[0010] In one or more possible embodiments, the vehicle coarse recognition on the current image frame comprises:

[0011] The current image frame is detected by using a deep learning detection method to obtain a vehicle frame and a license plate frame.

[0012] According to the position area of the vehicle frame and the license plate frame detected in the current frame, the correlation degree of the detected vehicle frame and the license plate frame is calculated.

[0013] According to the calculated correlation degree, the associated vehicle frame and license plate frame belonging to the same vehicle are determined, the vehicle frame without the associated license plate frame and the license plate frame without the associated vehicle frame are obtained, and a recognition result is obtained.

[0014] In one or more possible embodiments, vehicle tracking is performed according to the recognition result, including:

[0015] The detected vehicle frame and license plate frame are obtained.

[0016] When a new vehicle frame or license plate frame is recognized by using a tracking algorithm, an identity corresponding to the new vehicle frame is added, and the vehicle frame and the license plate frame of the new vehicle with an association relationship adopt the same identity.

[0017] When a tracked vehicle frame or license plate frame is recognized by using a tracking algorithm, the identity of the current recognized vehicle frame or license plate frame is determined according to the identity of the tracked vehicle frame or license plate frame, and the vehicle frame and the license plate frame of the tracked vehicle with an association relationship adopt the same identity.

[0018] In one or more possible embodiments, according to the target tracking result, the vehicle rough recognition result, and the effective target screening condition, an effective vehicle appearing in the current frame is determined, including:

[0019] The vehicles in the current target cache pool are obtained.

[0020] According to the target tracking result, the vehicle rough recognition result, and the effective target screening condition, the vehicles in the current target cache pool are updated, and the vehicles in the updated target cache pool are the effective vehicles appearing in the current frame.

[0021] In one or more possible embodiments, according to the target tracking result, the vehicle rough recognition result, and the effective target screening condition, the vehicles in the current target cache pool are updated, including at least one of the following steps:

[0022] According to the target tracking result, the vehicle rough recognition result, and the effective target screening condition, when any effective vehicle appearing in the current frame is not in the target cache pool, the effective vehicle is put into the target cache pool.

[0023] According to the target tracking result, when any vehicle in the target cache pool does not appear in the image frame for more than a set time length, the vehicle is deleted from the target cache pool.

[0024] According to the vehicle rough identification result, when it is determined that the size of the position area of the detected vehicle frame does not meet the requirement, the vehicle corresponding to the detected vehicle frame is determined as an invalid vehicle, and if the invalid vehicle is in the target cache pool, the invalid vehicle is deleted from the target cache pool;

[0025] According to the vehicle rough identification result, when it is determined that the confidence level of the detected vehicle frame or license plate frame is lower than a set threshold, the vehicle corresponding to the detected vehicle frame or license plate frame is determined as an invalid vehicle, and if the invalid vehicle is in the target cache pool, the invalid vehicle is deleted from the target cache pool;

[0026] According to the vehicle rough identification result, when it is determined that the detected vehicle frame or license plate frame is located outside the set detection area, the vehicle corresponding to the detected vehicle frame or license plate frame is determined as an invalid vehicle, and if the invalid vehicle is in the target cache pool, the invalid vehicle is deleted from the target cache pool;

[0027] According to the vehicle rough identification result and the target tracking result, the displacement of the vehicle frame or license plate frame is determined, and according to the displacement, the vehicle in a static state is determined as an invalid vehicle, and if the invalid vehicle is in the target cache pool, the invalid vehicle is deleted from the target cache pool;

[0028] According to the vehicle rough identification result, the vehicle existing in the cache pool and not appearing in the current frame is determined as an invalid vehicle, and the invalid vehicle is deleted from the target cache pool.

[0029] In one or more possible embodiments, the vehicle scheduling priority of the valid vehicle appearing in the current frame is determined in the following manner:

[0030] If the valid vehicle is a newly appearing vehicle in the current frame, the vehicle scheduling priority of the valid vehicle is determined as a set default value;

[0031] If the valid vehicle is a vehicle not scheduled in the last frame, the scheduling priority of the valid vehicle is increased by one level;

[0032] If the valid vehicle is a valid vehicle scheduled in the last frame, the scheduling priority of the valid vehicle is determined as the lowest priority.

[0033] In one or more possible embodiments, the re-detected license plate frame is associated with the recognized license plate and the corresponding vehicle to be scheduled, which includes:

[0034] For the same vehicle to be scheduled, if there is a license plate frame associated with the detection frame in the vehicle rough identification, and the associated license plate frame is inconsistent with the re-detected license plate frame, the re-detected license plate frame is used to replace the associated license plate frame;

[0035] If there is no license plate frame associated with the detection frame in the vehicle rough identification, the re-detected license plate frame is associated with the detection frame of the vehicle to be scheduled.

[0036] In one or more possible embodiments, the target vehicle is determined according to the license plate recognition result, and the target vehicle is dispatched using the to-be-dispatched resource.

[0037] For the same to-be-dispatched vehicle, license plate recognition results of recent frames are obtained and counted.

[0038] If the to-be-dispatched vehicle has different license plate recognition results in multiple frames, the vehicle corresponding to the license plate recognition result with the largest number of frames is determined as the target vehicle according to the number of frames with the same license plate recognition result.

[0039] If there are multiple license plate recognition results with the largest number of frames, the vehicle corresponding to the license plate recognition result with the highest confidence is determined as the target vehicle.

[0040] In a second aspect, the present application provides a license plate recognition device, the device comprising: a vehicle rough identification module, configured to obtain a current image frame from a video stream, perform vehicle rough identification on the current image frame, and perform vehicle tracking according to the identification result, wherein the vehicle rough identification comprises vehicle bounding box detection and license plate bounding box detection;

[0041] an effective vehicle determination module, configured to determine an effective vehicle appearing in a current frame according to a vehicle tracking result, a vehicle rough identification result, and an effective vehicle screening condition;

[0042] a to-be-dispatched vehicle determination module, configured to select at least one effective vehicle matching the to-be-dispatched resource as a to-be-dispatched vehicle according to a vehicle dispatch priority of the effective vehicle appearing in the current frame and a current to-be-dispatched resource;

[0043] a license plate recognition module, configured to perform license plate bounding box detection again on a position region of a vehicle bounding box of the to-be-dispatched vehicle, perform license plate recognition on the license plate bounding box detected again, and associate the license plate bounding box detected again with the recognized license plate and the corresponding to-be-dispatched vehicle to obtain a license plate recognition result;

[0044] a license plate result determination module, configured to determine a target vehicle according to the license plate recognition result, and dispatch the target vehicle using the to-be-dispatched resource.

[0045] In a third aspect, the present application provides a license plate recognition device, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the license plate recognition method according to any one of the first aspect.

[0046] In a fourth aspect, the present application provides a computer storage medium storing a computer program for causing a computer to execute the license plate recognition method according to any one of the first aspect.

[0047] The present application provides a license plate recognition method, device, equipment and computer storage medium, which can guarantee the accuracy of license plate recognition in a natural scene, and provide important technical support for tracking vehicles. The present application illustrates how to poll the license plate of a motor vehicle tracking target through a resource scheduling strategy in a resource-limited situation, so as to obtain a more accurate license plate recognition result through multi-frame license plate recognition voting while controlling the peak time consumption of a single frame. BRIEF DESCRIPTION OF DRAWINGS

[0048] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application and, do not limit the application.

[0049] Figure 1 A flowchart according to an embodiment of the present application is provided;

[0050] Figure 2 A flowchart according to an embodiment of the present application is provided;

[0051] Figure 3 A flowchart according to an embodiment of the present application is provided;

[0052] Figure 4 A device schematic diagram according to an embodiment of the present application is provided;

[0053] Figure 5 A device schematic diagram according to an embodiment of the present application is provided;

[0054] Figure 6 A computer readable storage medium according to an embodiment of the present application is provided. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0056] In a video monitoring scene, vehicle license plate recognition technology is an important technical means for tracing vehicles, which can identify and save the content in the license plate, and provide important technical support for technical personnel when tracing vehicles. However, in a natural scene, due to factors such as vehicle angle, lighting, and device performance, there is a high probability that the recognition will not be accurate, which will affect the work, especially when law enforcement personnel trace the illegal behavior of vehicles, the vehicle may be incorrectly or unrecognized, which will have a huge impact on the work. Therefore, based on the above problems, the application provides a license plate recognition method, which can ensure that the accuracy of license plate recognition is improved in a natural scene, and important technical support is provided for tracking vehicles.

[0057] The application provides a license plate recognition method, as shown in Figure 1 The application provides a license plate recognition method, as shown in

[0058] Step 101, acquiring a current image frame from a video stream, performing vehicle rough recognition on the current image frame, and performing vehicle tracking according to the recognition result, wherein the vehicle rough recognition includes vehicle frame detection and license plate frame detection.

[0059] In one or more possible embodiments, first, image frame data in a video stream is acquired, each image frame is processed, vehicle rough recognition on the current frame includes vehicle frame detection and license plate frame detection, vehicle frame detection information D v ={d i ,i=1,...M} is obtained, indicating that there are M vehicles in the current frame, license plate frame detection information D p ={d i ,i=1,...N} is obtained, indicating that N license plates are detected in the current frame, and vehicle tracking is performed according to the above M vehicles and N license plates.

[0060] Step 102, determining an effective vehicle appearing in the current frame according to the vehicle tracking result, the vehicle rough recognition result, and an effective vehicle screening condition.

[0061] In one or more possible embodiments, according to the vehicle tracking result, the vehicle rough recognition result, and the effective vehicle screening condition, the M vehicles and the N license plates identified in the current frame are selected as the effective vehicle of the current frame; the vehicle tracking result includes new vehicles identified in the current frame, vehicles in a lost state, vehicles in a deletion state exceeding a life cycle, and vehicles being tracked; the life cycle refers to a time period from the first frame appearing to the last frame disappearing to a specified time period, and a vehicle exceeding the life cycle is defined as a vehicle in a deletion state.

[0062] Step 103, according to the vehicle scheduling priority of the effective vehicle appearing in the current frame and the current to-be-scheduled resource, selecting at least one effective vehicle matching the to-be-scheduled resource as a to-be-scheduled vehicle.

[0063] In one or more possible embodiments, the scheduling priority of the effective vehicle is a default value in the vehicle scheduling priority of the newly appearing current frame, and the scheduling priority of the effective vehicle is related to the time when the vehicle is not to be scheduled, that is, the longer the time when the vehicle is not to be scheduled, the higher the vehicle scheduling priority; the effective vehicles can be sorted according to the vehicle scheduling priority, and a set number of effective vehicles are selected as to-be-scheduled vehicles according to the vehicle sorting and the to-be-scheduled resource for the next operation.

[0064] Step 104, performing license plate frame detection on the position area of the vehicle frame of the to-be-scheduled vehicle again, and performing license plate recognition on the re-detected license plate frame, associating the re-detected license plate frame with the recognized license plate and the corresponding to-be-scheduled vehicle to obtain a license plate recognition result.

[0065] In one or more possible embodiments, after the to-be-scheduled vehicle is output, the to-be-scheduled vehicle is re-detected for vehicle detection and license plate frame detection, and license plate recognition is performed, and the re-detected license plate frame is associated with the corresponding to-be-scheduled vehicle, and then the to-be-scheduled vehicle can obtain the corresponding license plate recognition result.

[0066] Step 105, determining a target vehicle according to the license plate recognition result, and scheduling the target vehicle using the to-be-scheduled resource.

[0067] In one or more possible embodiments, the target vehicle is determined from the to-be-scheduled vehicle, and the to-be-scheduled resource is used to schedule the target vehicle, and other required work is performed.

[0068] According to the license plate recognition method provided in the present application, the accuracy of license plate recognition can be improved in a natural scene; in the case of resource limitation, the license plate recognition strategy of polling the vehicle tracking target through the resource scheduling strategy can control the peak time of a single frame while obtaining more accurate license plate recognition results through multi-frame license plate recognition voting, providing important technical support for tracking vehicles; and according to the two associations of the vehicle and the license plate, the false license plates can be filtered, and the accuracy of the recognized license plates can be improved.

[0069] In one or more possible embodiments, the current image frame is subjected to coarse vehicle recognition, as shown in FIG. 2A, which includes: Figure 2

[0070] Step 201, performing vehicle frame detection and license plate frame detection on the current image frame by using a deep learning detection method;

[0071] ​In one or more possible embodiments, the deep learning detection algorithm described above can be a YOLO series or a regional Faster RCNN series deep learning detection method with CNN features, and the current image frame is subjected to vehicle frame detection and license plate frame detection.

[0072] Step 202, calculating the correlation degree of the detected vehicle frame and license plate frame according to the position area of the detected vehicle frame and license plate frame of the current frame;

[0073] Step 203, determining the associated vehicle frame and license plate frame according to the calculated correlation degree, the vehicle frame without the associated license plate frame, the license plate frame without the associated vehicle frame, and obtaining the recognition result.

[0074] In one or more possible embodiments, the correlation degree of the detected vehicle frame and license plate frame is calculated by the Hungarian algorithm according to the position area of the detected vehicle frame and license plate frame, and the specific correlation degree calculation formula is:

[0075]

[0076] Wherein, x v and y v are the horizontal and vertical coordinates of the center point of the vehicle frame; x p and y p are the horizontal and vertical coordinates of the center point of the license plate frame; w frame and h frame are the width and height of the image of the current frame; according to the calculation of the correlation degree, the threshold value can be set to determine whether there is an association relationship between the vehicle and the license plate; for example, there are M vehicles and N license plates, the correlation degree of the M vehicles and the N license plates is calculated, a M*N matrix is obtained, and the threshold value is set to determine whether there is an association relationship between the vehicle and the license plate. When the threshold value is exceeded, the corresponding license plate and vehicle have no association relationship, and according to the calculation of the two-way matching result of the vehicle and the license plate and less than the set threshold value, the corresponding vehicle and license plate are associated. Therefore, after the above association calculation, there are three situations, respectively, the associated vehicle frame and license plate frame, the vehicle frame without the associated license plate frame, and the license plate frame without the associated vehicle frame, and three recognition results are obtained.

[0077] In one or more possible embodiments, vehicle tracking is performed according to the recognition result, as shown in Figure 3 , including:

[0078] Step 301, obtaining the detected vehicle frame and license plate frame;

[0079] In step 302, when the tracking algorithm determines that a new vehicle frame or license plate frame is recognized, the identity corresponding to the new vehicle frame is increased, and the new vehicle frame and the license plate frame having the association relationship adopt the same identity.

[0080] In step 303, when the tracking algorithm determines that a tracking vehicle frame or license plate frame is recognized, the identity of the currently recognized frame or license plate frame is determined according to the identity of the tracking vehicle frame or license plate frame, and the tracking vehicle frame and the license plate frame having the association relationship adopt the same identity. Wherein, when the tracking algorithm determines that a vehicle frame tracked in a historical frame is recognized according to the detected vehicle frame, it is determined that the vehicle frame tracked in the historical frame adopts the identity of the vehicle frame, and if it is determined that the vehicle frame has an associated license plate frame, it is determined that the identity of the license plate frame is the same as that of the vehicle frame; when the tracking algorithm determines that a new vehicle license plate frame is recognized according to the detected license plate frame, the identity corresponding to the new vehicle license plate frame is increased, and if it is determined that the license plate frame has an associated vehicle frame, it is determined that the identity of the vehicle frame is the same as that of the license plate frame; when the tracking algorithm determines that a license plate frame tracked in a historical frame is recognized according to the detected license plate frame, it is determined that the license plate frame tracked in the historical frame adopts the identity of the license plate frame, and if it is determined that the license plate frame has an associated vehicle frame, it is determined that the identity of the vehicle frame is the same as that of the license plate frame.

[0081] In one or more possible embodiments, the above tracking algorithm can be Sort, DeepSort, etc. to track vehicles; when the above tracking algorithm is used for vehicle tracking, if a new vehicle frame or license plate frame is recognized in the current frame, a new identity is set for the new vehicle frame or license plate frame, and if the new vehicle frame and the license plate frame have an association relationship, the same identity can be adopted, for example, (T v ,T p )={(t v ,t p ) i ,i=1,...P} indicates that the vehicle and the license plate are successfully associated, where t v and t p represent the successfully associated vehicle and license plate, and P represents the number of successfully associated vehicles and license plates; (T v )={(t v ) i ,i=1,...Q} indicates that there are Q vehicles in total; (T p )={(t p ) iR represents that there are R individual license plates; when a vehicle box or a license plate box of a tracked vehicle is recognized in other frames, the identity of the vehicle box or the license plate box recognized in the current frame is determined according to the identity of the vehicle box or the license plate box of the tracked vehicle, and the vehicle box and the license plate box of the tracked vehicle having the association relationship adopt the same identity; therefore, the identities corresponding to the associated vehicle box and the license plate box, the vehicle box without the associated license plate box, and the license plate box without the associated vehicle box can be obtained.

[0082] In one or more possible embodiments, the effective vehicle appearing in the current frame is determined according to the target tracking result, the vehicle rough recognition result and the effective target screening condition, and includes at least one of the following steps:

[0083] 1) obtaining the vehicle in the current target cache pool; the current target cache pool includes the effective vehicle determined in the last frame;

[0084] 2) updating the vehicle in the current target cache pool according to the target tracking result, the vehicle rough recognition result and the effective target screening condition; the vehicle in the updated target cache pool is the effective vehicle appearing in the current frame; since new effective vehicles, lost vehicles, vehicles exceeding the life cycle and vehicles with poor recognition result accuracy may appear in the current frame, the vehicle in the current target cache pool needs to be updated to obtain the latest target cache pool.

[0085] 3) the target cache pool can be established before data collection of the image frame of the video stream, and is updated step by step according to the current frame; the target cache pool can be directly called when the data of the current frame is used; and the three different identities are stored in the target cache pool, which can be more convenient and faster when called.

[0086] In one or more possible embodiments, the effective vehicle appearing in the current frame is determined according to the target tracking result, the vehicle rough recognition result and the effective target screening condition, and includes at least one of the following steps:

[0087] 1) when any effective vehicle appearing in the current frame is not in the target cache pool according to the target tracking result, the vehicle rough recognition result and the effective target screening condition, the effective vehicle is put into the target cache pool; the effective vehicle appearing in the current frame is put into the target cache pool as a new effective vehicle for updating.

[0088] 2) when any vehicle in the target cache pool does not appear in the image frame for more than a set time according to the target tracking result, the vehicle is deleted from the target cache pool; whether the number of frames from the last frame when the vehicle appears to the current frame exceeds the set threshold or the vehicle does not appear for more than a set time, the vehicle needs to be deleted from the target cache pool, and at this time, the vehicle is in a deleted state.

[0089] 3) According to the rough identification result of the vehicle, if the size of the position area of the detected vehicle frame does not meet the requirement, the vehicle corresponding to the detected vehicle frame is determined as an invalid vehicle, and if the invalid vehicle is in the target cache pool, the invalid vehicle is deleted from the target cache pool. When the height of the vehicle frame in the current frame is less than the set height threshold and the width is less than the set width threshold, it is determined that the size of the position area of the detected vehicle frame does not meet the requirement, and the vehicle corresponding to the vehicle frame is defined as an invalid vehicle and is deleted from the target cache pool.

[0090] 4) According to the rough identification result of the vehicle, if the vehicle corresponding to the vehicle frame or the license plate frame with a detection confidence lower than a set threshold is determined as an invalid vehicle, and if the invalid vehicle is in the target cache pool, the invalid vehicle is deleted from the target cache pool. In the vehicle detection process, not only the vehicle frame and the license plate frame are detected, but also the confidence of the detection result is obtained. If the confidence is lower than the set threshold, it indicates that the accuracy of the detection result is poor, and the corresponding vehicle is defined as an invalid vehicle and is deleted from the target cache pool.

[0091] 5) According to the rough identification result of the vehicle, the vehicle frame or the license plate frame located in the set detection area is determined as an invalid vehicle, and if the invalid vehicle is in the target cache pool, the invalid vehicle is deleted from the target cache pool. For example, the set detection area of the license plate frame is generally the lower half of the vehicle frame, and the upper half of the vehicle frame is outside the set detection area. If a license plate frame is identified outside the set detection area, it is directly determined that an invalid license plate is identified, the corresponding vehicle is defined as an invalid vehicle, and then the invalid vehicle is deleted from the target cache pool.

[0092] 6) According to the rough identification result of the vehicle and the target tracking result, the displacement of the vehicle frame or the license plate frame is determined, and the vehicle in a stationary state is determined as an invalid vehicle according to the displacement. If the invalid vehicle is in the target cache pool, the invalid vehicle is deleted from the target cache pool. The stationary state can be calculated by the displacement of the center point of the vehicle frame. If the horizontal coordinate displacement is less than half of the width of the vehicle frame and the vertical coordinate displacement is less than half of the height of the vehicle frame, the motor vehicle is in a stationary state.

[0093] In one or more possible embodiments, the vehicle scheduling priority of the effective vehicle appearing in the current frame is determined in the following manner: if the effective vehicle is a newly appearing vehicle in the current frame, the vehicle scheduling priority of the effective vehicle is determined as a set default value; if the effective vehicle is an unscheduled effective vehicle in the last frame, the scheduling priority of the effective vehicle is increased by one level; if the effective vehicle is a scheduled vehicle in the last frame, the scheduling priority of the effective vehicle is determined as the lowest priority. In this way, the vehicle scheduling priority of a newly appearing vehicle in the current frame can be set as a default value and added to the above-mentioned target cache pool. If the effective vehicle in the above-mentioned target cache pool is also an effective vehicle in the last frame but is not scheduled, the scheduling priority of the corresponding effective vehicle is increased by one level, so that the vehicle can be scheduled as soon as possible in the next frame. If the effective vehicle has been scheduled in the last frame, the scheduling priority of the corresponding vehicle is set as the lowest priority, so that the vehicle will not be repeatedly scheduled in the next frame. For example, the effective vehicle is a newly added vehicle in the current frame, and the vehicle scheduling priority of the vehicle is a default value priority d . If the effective vehicle in the above-mentioned target cache pool is also an effective vehicle in the last frame but is not scheduled, the scheduling priority of the effective vehicle is updated, and the original scheduling priority p is updated to p = p + 1. If the effective vehicle has been scheduled in the last frame, the scheduling priority of the corresponding vehicle is set as the lowest priority 0, so that the effective vehicle will not be repeatedly scheduled.

[0094] In one or more possible embodiments, the re-detected license plate frame is associated with the recognized license plate and the corresponding vehicle to be dispatched, including: for the same vehicle to be dispatched, if there is a license plate frame associated with the detection frame in the vehicle rough identification, and the associated license plate frame is inconsistent with the re-detected license plate frame, the re-detected license plate frame is used to replace the associated license plate frame; if there is no license plate frame associated with the detection frame in the vehicle rough identification, the re-detected license plate frame is associated with the detection frame of the vehicle to be dispatched. Since the first rough identification process is based on the whole frame image for identification, there is a high possibility that the association between the vehicle and the license plate will be incorrect, or that the detection will be missed, so the second identification of the vehicle to be dispatched is a single detection of a small number of vehicles, which can effectively improve the accuracy of the detection; after the corresponding vehicle to be dispatched is output, the vehicle frame detection and the license plate frame detection are performed again, the license plate recognition is performed according to the new license plate frame detection, and the license plate recognition result is more accurate; when the license plate of the above vehicle to be dispatched is associated, if there is a license plate frame associated with the detection frame in the vehicle rough identification, but the associated license plate frame is inconsistent with the re-detected license plate frame due to different detection roughness, the re-detected license plate frame needs to be used to replace the associated license plate frame and the above vehicle to be dispatched is associated again; if there is no license plate frame associated with the detection frame in the vehicle rough identification, the re-detected license plate frame is associated with the detection frame of the vehicle to be dispatched.

[0095] In one or more possible embodiments, a target vehicle is determined according to a license plate recognition result, and the target vehicle is dispatched using a to-be-dispatched resource, including: for the same vehicle to be dispatched, license plate recognition results of recent frames are obtained and counted; if the vehicle to be dispatched is different in multiple frames of license plate recognition results, for example, there are 10 frames of license plate recognition results, 8 frames are "Sud 3Q521" and the other two frames of recognition results are "Sud 30521", the vehicle corresponding to the license plate recognition result with the most frames is determined as the target vehicle according to the number of frames with the same license plate recognition result, so the license plate of the target vehicle should be selected as the final recognition result "Sud 3Q521"; if there are multiple frames with the most different license plate recognition results, for example, there are 10 frames of license plate recognition results, 5 frames are "Sud 3Q521" and the other 5 frames of recognition results are "Sud 30521", but the confidence of the algorithm with the recognition result "Sud 3Q521" is relatively high, the vehicle corresponding to the license plate recognition result with the highest confidence is determined as the target vehicle, so the license plate of the target vehicle should be selected as the final recognition result "Sud 3Q521". In this way, the license plate recognition result corresponding to the vehicle to be dispatched can be obtained; the license plate recognition result with the most frames and the license plate recognition result with the highest confidence are used as the final license plate recognition result, which can effectively improve the accuracy of the license plate recognition result.

[0096] According to the license plate recognition method provided in the application, the accuracy of license plate recognition can be improved in a natural scene, and important technical support is provided for tracking vehicles. The scheme illustrates how to poll the license plate recognition strategy of the motor vehicle tracking target through the resource scheduling strategy under the condition of limited resources, and more accurate license plate recognition results can be obtained through multi-frame license plate recognition voting while controlling the peak time consumption of a single frame.

[0097] Based on the same inventive concept, the application also provides a license plate recognition device, as shown in the description. Figure 4 The device includes:

[0098] A vehicle rough identification module 401 is configured to acquire a current image frame from a video stream, perform vehicle rough identification on the current image frame, and perform vehicle tracking according to the identification result. The vehicle rough identification includes vehicle frame detection and license plate frame detection.

[0099] An effective vehicle determination module 402 is configured to determine an effective vehicle appearing in a current frame according to a vehicle tracking result, a vehicle rough identification result, and an effective vehicle screening condition.

[0100] A vehicle to be scheduled determination module 403 is configured to select at least one effective vehicle matching the to-be-scheduled resource as a vehicle to be scheduled according to a vehicle scheduling priority of the effective vehicle appearing in the current frame and the current to-be-scheduled resource.

[0101] A license plate recognition module 404 is configured to perform license plate frame detection on a position region of a vehicle frame of the vehicle to be scheduled again, perform license plate recognition on the license plate frame detected again, and associate the license plate frame detected again with the recognized license plate and the corresponding vehicle to be scheduled to obtain a license plate recognition result.

[0102] A license plate result determination module 405 is configured to determine a target vehicle according to the license plate recognition result, and schedule the target vehicle using the to-be-scheduled resource.

[0103] Based on the same inventive concept, the application also provides a license plate recognition device, including at least one processor and a memory in communication connection with the at least one processor. The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the license plate recognition method described above.

[0104] As shown in the description, Figure 5 The device includes a processor 501, a memory 502, a communication interface 503, and a bus 504. The processor 501, the memory 502, and the communication interface 503 are connected to each other through the bus 504.

[0105] The processor 501 is configured to read and execute instructions in the memory 502, so as to enable the at least one processor to perform the license plate recognition method provided in the above embodiments.

[0106] The memory 502 is configured to store various instructions and programs of the license plate recognition method provided in the above embodiments.

[0107] The bus 504 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the figure to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0108] The processor 501 can be a central processing unit (CPU), a network processor (NP), a graphic processing unit (GPU), or any combination of the CPU, the NP and the GPU. It can also be a hardware chip. The hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0109] In addition, the present application also provides a computer readable storage medium, such as a compact disc (CD) 505, as shown in the figure. Figure 6 The computer storage medium stores a computer program, and the computer program is used to enable the computer to execute any one of the methods in the above embodiments.

[0110] The memory can include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 1321 and / or a cache memory 1322, and can further include a read-only memory (ROM) 1323.

[0111] The storage can also include a program / utility 1325 having a set (at least one) of program modules 1324, including but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which

[0112] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, apparatus, or computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including but not limited to disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0113] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing system or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart Figure 1 one or more functions specified in the flowchart block or blocks. Figure 1 one or more functions specified in the flowchart block or blocks.

[0114] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more functions specified in the flowchart block or blocks. Figure 1 one or more functions specified in the flowchart block or blocks.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more functions specified in the flowchart block or blocks. Figure 1 one or more functions specified in the flowchart block or blocks.

[0116] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A license plate recognition method, characterized in that, include: The current image frame is obtained from the video stream, and vehicle coarse recognition is performed on the current image frame. Vehicle tracking is performed based on the recognition results. The vehicle coarse recognition includes vehicle frame detection and license plate frame detection. Based on the vehicle tracking results, the vehicle coarse identification results, and the effective vehicle screening criteria, determine the effective vehicles appearing in the current frame; Based on the vehicle scheduling priority of the valid vehicles appearing in the current frame and the currently scheduled resources, at least one valid vehicle matching the scheduled resources is selected as the vehicle to be scheduled. The vehicle scheduling priority of the valid vehicles appearing in the current frame is determined as follows: if the valid vehicle is a newly appearing vehicle in the current frame, its vehicle scheduling priority is set to the default value; if the valid vehicle is a valid vehicle that was not scheduled in the previous frame, its scheduling priority is increased by one level; if the valid vehicle is a valid vehicle that was scheduled in the previous frame, its scheduling priority is set to the lowest priority. The license plate frame is detected again in the location area of ​​the vehicle frame of the vehicle to be dispatched, and the license plate frame is recognized again. The detected license plate frame is associated with the recognized license plate and the corresponding vehicle to be dispatched to obtain the license plate recognition result. The target vehicle is determined based on the license plate recognition results, and the target vehicle is then scheduled using the resources to be scheduled.

2. The method according to claim 1, characterized in that, Performing coarse vehicle identification on the current image frame includes: Using deep learning detection methods, vehicle frame detection and license plate frame detection are performed on the current image frame; Based on the location regions of the detected vehicle frame and license plate frame in the current frame, calculate the correlation between the detected vehicle frame and license plate frame; Based on the calculated correlation, the associated vehicle frames and license plate frames belonging to the same vehicle are determined, and the vehicle frames without associated license plate frames and the license plate frames without associated vehicle frames are identified, thus obtaining the recognition result.

3. The method according to claim 2, characterized in that, Vehicle tracking is performed based on the identification results, including: Obtain the detected vehicle frame and license plate frame; When a new vehicle's frame or license plate frame is identified using a tracking algorithm, an identity identifier corresponding to the new vehicle's frame is added, and the same identity identifier is used for the frames and license plate frames of new vehicles that are related. When the tracking algorithm identifies the frame or license plate frame of the tracked vehicle, it determines the identity of the currently identified frame or license plate frame based on the identity identifier of the frame or license plate frame of the tracked vehicle. The frame and license plate frames of tracked vehicles that are related use the same identity identifier.

4. The method according to any one of claims 1 to 3, characterized in that, Based on the target tracking results, vehicle coarse identification results, and valid target selection criteria, the valid vehicles appearing in the current frame are determined, including: Retrieve vehicles from the current target cache pool; Based on the target tracking results, vehicle coarse identification results, and effective target selection criteria, the vehicles in the current target cache pool are updated, and the vehicles in the updated target cache pool are the effective vehicles appearing in the current frame.

5. The method according to claim 4, characterized in that, Based on the target tracking results, vehicle coarse identification results, and effective target selection criteria, the vehicles in the current target cache pool are updated, including at least one of the following steps: Based on the target tracking results, vehicle coarse identification results, and effective target screening conditions, if any effective vehicle appearing in the current frame is not in the target cache pool, the effective vehicle is added to the target cache pool. Based on the target tracking results, if any vehicle in the target cache pool does not appear in an image frame for a set time period, the vehicle will be deleted from the target cache pool. Based on the vehicle coarse recognition results, if the size of the detected vehicle frame location area does not meet the requirements, the vehicle corresponding to the detected vehicle frame is identified as an invalid vehicle. If the invalid vehicle is in the target cache pool, the invalid vehicle is deleted from the target cache pool. Based on the vehicle coarse recognition results, vehicles corresponding to vehicle frames or license plate frames with a detection confidence level lower than a set threshold are identified as invalid vehicles. If the invalid vehicle is in the target cache pool, the invalid vehicle is deleted from the target cache pool. Based on the vehicle coarse recognition results, vehicle frames or license plate frames located outside the set detection area are identified as invalid vehicles. If the invalid vehicle is in the target cache pool, the invalid vehicle is deleted from the target cache pool. Based on the vehicle coarse identification results and target tracking results, the displacement of the vehicle frame or license plate frame is determined. Based on the displacement, vehicles in a stationary state are determined to be invalid vehicles. If the invalid vehicle is in the target cache pool, the invalid vehicle is deleted from the target cache pool. Based on the vehicle coarse identification results, vehicles that exist in the cache pool but do not appear in the current frame are identified as invalid vehicles, and these invalid vehicles are deleted from the target cache pool.

6. The method according to claim 1, characterized in that, The license plate frame will be detected again and associated with the identified license plate and the corresponding vehicle to be dispatched, including: For the same vehicle to be dispatched, if there is a license plate frame associated with the detection frame in the vehicle coarse identification, and the associated license plate frame is inconsistent with the license plate frame detected again, the associated license plate frame shall be replaced by the license plate frame detected again. If no license plate frame is associated with the detection frame in the vehicle coarse identification, the license plate frame that is detected again will be associated with the detection frame of the vehicle to be dispatched.

7. The method according to any one of claims 1 to 3, characterized in that, The process of identifying the target vehicle based on the license plate recognition result and scheduling the target vehicle using the resources to be scheduled includes: For the same vehicle to be dispatched, obtain the license plate recognition results of the most recent few frames and perform statistics; If the license plate recognition results of the vehicle to be dispatched are different in multiple frames, the vehicle corresponding to the license plate recognition result with the most frames with the same license plate recognition result shall be determined as the target vehicle. If there are multiple license plate recognition results with the most frames, the vehicle corresponding to the license plate recognition result with the highest confidence level is determined as the target vehicle.

8. A license plate recognition device, characterized in that, The device includes: The vehicle coarse recognition module is used to acquire the current image frame from the video stream, perform vehicle coarse recognition on the current image frame, and track the vehicle based on the recognition result. The vehicle coarse recognition includes vehicle frame detection and license plate frame detection. The effective vehicle determination module is used to determine the effective vehicles appearing in the current frame based on vehicle tracking results, vehicle coarse identification results, and effective vehicle filtering conditions. The vehicle to be dispatched module is used to select at least one valid vehicle that matches the resource to be dispatched as the vehicle to be dispatched based on the vehicle dispatch priority of the valid vehicles appearing in the current frame and the current resource to be dispatched. The vehicle dispatch priority of the valid vehicles appearing in the current frame is determined as follows: if the valid vehicle is a newly appearing vehicle in the current frame, its vehicle dispatch priority is set to the default value; if the valid vehicle is a valid vehicle that was not dispatched in the previous frame, its dispatch priority is increased by one level; if the valid vehicle is a valid vehicle that was dispatched in the previous frame, its dispatch priority is set to the lowest priority. The license plate recognition module is used to re-detect the license plate frame of the vehicle frame of the vehicle to be dispatched, and to recognize the license plate frame of the re-detected license plate frame. The re-detected license plate frame is associated with the recognized license plate and the corresponding vehicle to be dispatched to obtain the license plate recognition result. The license plate result determination module is used to determine the target vehicle based on the license plate recognition result and to schedule the target vehicle using the resources to be scheduled.

9. A license plate recognition device, characterized in that, The device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the license plate recognition method as described in any one of claims 1-7.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer program that enables the computer to perform the license plate recognition method as described in any one of claims 1-7.

Images