Vehicle license plate matching method, system, medium and device

Abstract

The application discloses a vehicle license plate matching method, system, medium and equipment, and the method comprises the following steps: traversing and calculating the candidate matching vehicle table corresponding to each license plate in the license plate frame according to a preset threshold matching rule in the vehicle frame set, and performing initial matching of the vehicle license plate according to the candidate matching vehicle table corresponding to each license plate, the vehicle license plate confidence matching result and the priority matching principle; performing multiple cycle matching on each license plate and the corresponding candidate matching vehicle table left after the initial matching according to the cycle matching principle, and stopping the cycle matching until the number of vehicles left after the matched vehicles are deleted in the candidate matching vehicle table corresponding to each license plate left after the cycle matching is greater than or equal to the number threshold; and performing final matching on each license plate and the corresponding candidate matching vehicle table left after the cycle matching according to the preset area matching rule, so as to accurately match all vehicles and license plates one by one.

Description

Technical Field

[0001] This invention relates to the field of license plate matching technology, and in particular to a vehicle license plate matching method, system, medium and device. Background Technology

[0002] Video structuring technology can obtain structured information from surveillance videos, such as various attribute information of pedestrians and vehicles, providing information support for decision-making. However, for vehicle structuring, detection can only obtain all vehicles and license plates in the surveillance image, but cannot directly obtain the correlation between vehicles and license plates. Since the license plate is one of the vehicle's attributes, matching the two is necessary to obtain structured information.

[0003] When the license plate is on only one vehicle, it can be matched directly. However, when the license plate is on multiple vehicles, it typically occurs when a large vehicle is turning next to a smaller vehicle, as shown in the attached diagram. Figure 1 As shown, at this point, the license plate cannot be matched with the vehicle; an additional strategy is required for matching. Summary of the Invention

[0004] The present invention provides a vehicle license plate matching method, system, medium and device for accurately matching all vehicles with their license plates one-to-one.

[0005] Firstly, a vehicle license plate matching method is provided, which specifically includes the following steps:

[0006] Obtain the vehicle frame set corresponding to multiple vehicles and the license plate frame set corresponding to multiple license plates;

[0007] Calculate the license plate confidence score corresponding to each license plate frame within each vehicle frame, and perform vehicle license plate confidence score matching based on the license plate confidence score.

[0008] In the vehicle frame set, the candidate matching vehicle table corresponding to each license plate in the license plate frame is calculated by traversing the vehicle frame according to the preset threshold matching rule, and the initial matching of vehicle license plates is performed according to the candidate matching vehicle table corresponding to each license plate and the vehicle license plate confidence matching result, based on the priority matching principle.

[0009] The cyclic matching principle is used to perform multiple cyclic matchings on each unmatched license plate and its corresponding candidate matching vehicle table after the initial matching, until the number of vehicles in the candidate matching vehicle table corresponding to each unmatched license plate after the cyclic matching is greater than or equal to the number threshold is detected.

[0010] According to the preset area matching rules, the remaining unmatched license plates after the cyclic matching and their corresponding candidate matching vehicle tables are finally matched.

[0011] According to the first aspect, in a first possible implementation of the first aspect, the step of "calculating the license plate confidence score corresponding to each of the multiple license plate frames existing within each vehicle frame, and performing vehicle license plate confidence score matching based on the license plate confidence score" specifically includes the following steps:

[0012] When multiple license plate frames are detected within a vehicle frame, the license plate confidence score for each frame is calculated, and the license plate with the highest confidence score is selected to match one of the detected vehicles.

[0013] According to the first aspect, in a second possible implementation of the first aspect, the step of "traversing and calculating the candidate matching vehicle table corresponding to each license plate in the license plate frame according to a preset threshold matching rule in the vehicle frame set" specifically includes the following steps:

[0014] The area ratio A corresponding to each license plate i in the license plate frame is calculated by traversing the vehicle frame set as follows:

[0015] A = Area of ​​intersection between license plate i and vehicle j / Area of ​​license plate i;

[0016] When the area ratio corresponding to vehicle j is detected to be greater than the area ratio threshold, vehicle j is added to the candidate matching vehicle table of license plate i.

[0017] According to the first aspect, in the third possible implementation of the first aspect, the step of "performing initial matching of vehicle license plates according to the candidate matching vehicle table and vehicle license plate confidence matching results, and according to the priority matching principle" specifically includes the following steps:

[0018] Get the number of vehicles in the candidate matching vehicle table corresponding to each license plate;

[0019] When the number of vehicles in the candidate matching vehicle table corresponding to a license plate is equal to 1, and the only vehicle in the candidate matching vehicle table fails to match the license plate with the confidence level, then the detected license plate is matched with the only vehicle in the corresponding candidate matching vehicle table.

[0020] When the number of vehicles in the candidate matching vehicle table corresponding to a license plate is equal to 1, and the only vehicle in the candidate matching vehicle table has been matched with the license plate through confidence, the license plate confidence of the detected license plate is calculated and compared with the license plate confidence of the only vehicle that has been matched with the license plate through confidence. The license plate corresponding to the license plate with the highest license plate confidence is selected to update the matching.

[0021] If the number of vehicles in the candidate matching vehicle table corresponding to a license plate is equal to 0, then there is no matching vehicle for the detected license plate.

[0022] According to the first aspect, in the third possible implementation of the first aspect, the step of "performing multiple cyclic matchings on each unmatched license plate and its corresponding candidate matching vehicle table after the initial matching according to the cyclic matching principle, until the number of vehicles in the candidate matching vehicle table corresponding to each unmatched license plate after cyclic matching is greater than or equal to the number threshold" specifically includes the following steps:

[0023] After the initial matching, delete the successfully matched vehicles from the candidate matching vehicle table of the remaining unmatched license plates, and then match the remaining vehicle license plates again according to the priority matching principle.

[0024] After the second matching, delete the successfully matched vehicles from the candidate matching vehicle table of the remaining unmatched license plates, and then match the remaining vehicle license plates again according to the priority matching principle.

[0025] The matching process is repeated until the number of vehicles in the candidate matching vehicle table corresponding to each unmatched license plate after the cyclic matching is detected is greater than or equal to the number threshold.

[0026] According to the third possible implementation of the first aspect, in the fourth possible implementation of the first aspect, the step of "deleting the successfully matched vehicles from the candidate matching vehicle table of the remaining unmatched license plates after the initial matching, and re-matching the remaining vehicle license plates according to the priority matching principle" specifically includes the following steps:

[0027] Get the remaining unmatched license plates when the number of vehicles in the candidate matching vehicle table is greater than or equal to the number threshold;

[0028] Check if there are any vehicles in the candidate matching vehicle table for the remaining unmatched license plates that were successfully matched in the initial matching;

[0029] If there are vehicles that have been successfully matched in the initial matching, then delete the corresponding matched vehicles from the candidate matching vehicle table for each license plate.

[0030] If the number of vehicles in the candidate matching vehicle table corresponding to a license plate is equal to 1 after deleting the matched vehicles, then the detected license plate is matched with the unique vehicle in the corresponding candidate matching vehicle table.

[0031] If the number of vehicles in the candidate matching vehicle table corresponding to a license plate is equal to 0 after deleting the matched vehicles, then there are no matched vehicles for the detected license plate.

[0032] According to the first aspect, in the fifth possible implementation of the first aspect, the step of "performing a final match on each unmatched license plate and its corresponding candidate matching vehicle table after cyclic matching according to a preset area matching rule" specifically includes the following steps:

[0033] Get the remaining unmatched license plates after the loop matching and their corresponding candidate matching vehicle table, and delete the successfully matched vehicles in the candidate matching vehicle table of the remaining unmatched license plates.

[0034] Select the vehicle with the smallest vehicle frame area from the candidate matching vehicle table for each of the remaining unmatched license plates, and match each of the remaining unmatched license plates one by one.

[0035] Secondly, a vehicle license plate matching system is also provided, including:

[0036] The dataset module is used to obtain the vehicle frame set corresponding to multiple vehicles and the license plate frame set corresponding to multiple license plates;

[0037] The confidence matching module is connected in communication with the dataset module. It is used to calculate the confidence of the license plates corresponding to the multiple license plate frames in each vehicle frame, and to perform vehicle license plate confidence matching based on the license plate confidence.

[0038] The initial matching module is communicatively connected to the dataset module. It is used to traverse and calculate the candidate matching vehicle table corresponding to each license plate in the license plate frame according to the preset threshold matching rule in the vehicle frame set, and perform initial matching of vehicle license plates according to the candidate matching vehicle table corresponding to each license plate and the priority matching principle.

[0039] The initial matching module is communicatively connected to the dataset module and the confidence matching module. It is used to traverse and calculate the candidate matching vehicle table corresponding to each license plate in the license plate frame according to the preset threshold matching rule in the vehicle frame set, and perform initial matching of vehicle license plates according to the candidate matching vehicle table corresponding to each license plate and the vehicle license plate confidence matching result, and according to the priority matching principle.

[0040] A cyclic matching module, communicatively connected to the initial matching module, is used to perform multiple cyclic matching operations on each unmatched license plate remaining after the initial matching and its corresponding candidate matching vehicle table, according to the cyclic matching principle, until the number of vehicles in the candidate matching vehicle table corresponding to each unmatched license plate remaining after cyclic matching, after deleting matched vehicles, is greater than or equal to a number threshold, at which point the cyclic matching stops; and,

[0041] The area matching module is communicatively connected to the cyclic matching module and is used to perform final matching on each unmatched license plate and its corresponding candidate matching vehicle table after cyclic matching according to a preset area matching rule.

[0042] Thirdly, a storage medium is also provided, on which a computer program is stored, characterized in that, when the computer program is executed by a processor, it implements the vehicle license plate matching method as described above.

[0043] Fourthly, an electronic device is also provided, including a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that, when the processor executes the computer program, it implements the vehicle license plate matching method as described above.

[0044] Compared with the prior art, the advantages of the present invention are as follows: First, a candidate matching vehicle table corresponding to each license plate is obtained. Then, based on the candidate matching vehicle table corresponding to each license plate, initial matching of vehicles and license plates is performed according to the priority matching principle. Multiple cyclic matchings are performed on the remaining unmatched license plates and their corresponding candidate matching vehicle tables after the initial matching according to the cyclic matching principle. Finally, a final matching is performed on the remaining unmatched license plates and their corresponding candidate matching vehicle tables after the cyclic matching according to the preset area matching rule, so as to achieve a one-to-one accurate matching of all vehicles and license plates. Attached Figure Description

[0045] Figure 1 This is a schematic diagram showing the positional relationship between the license plate frame and the vehicle frame;

[0046] Figure 2 This is another schematic diagram showing the positional relationship between the license plate frame and the vehicle frame;

[0047] Figure 3 This is a flowchart illustrating an embodiment of a vehicle license plate matching method according to the present invention;

[0048] Figure 4 This is a schematic diagram of the structure of a vehicle license plate matching system according to the present invention. Detailed Implementation

[0049] Referring now to specific embodiments of the invention, examples of which are illustrated in the accompanying drawings. Although the invention will be described in conjunction with specific embodiments, it will be understood that it is not intended to limit the invention to the described embodiments. Rather, it is intended to cover variations, modifications, and equivalents included within the spirit and scope of the invention as defined by the appended claims. It should be noted that the method steps described herein can be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of both.

[0050] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] Note: The examples described below are merely specific examples and are not intended to limit the embodiments of the present invention to the specific steps, values, conditions, data, order, etc. Those skilled in the art can utilize the concept of the present invention to construct more embodiments not mentioned herein by reading this specification.

[0052] See Figure 3 As shown, this embodiment of the invention provides a vehicle license plate matching method, which specifically includes the following steps:

[0053] S100: Obtain the vehicle frame set corresponding to multiple vehicles and the license plate frame set corresponding to multiple license plates;

[0054] See Figure 1 As shown, in the video structure data, the obtained vehicle structure data is the vehicle detection box selected by the rectangular box, and the license plate also exists in the corresponding license plate box.

[0055] The initial set of vehicle frames and license plate frames were not matched, meaning that no license plate was matched for each vehicle.

[0056] S200, calculate the license plate confidence score corresponding to each of the multiple license plate frames in each vehicle frame, and perform vehicle license plate confidence score matching based on the license plate confidence score.

[0057] S300, in the vehicle frame set, the candidate matching vehicle table corresponding to each license plate in the license plate frame is calculated by traversing the vehicle frame set according to the preset threshold matching rule, and the vehicle license plate is initially matched according to the candidate matching vehicle table corresponding to each license plate and the priority matching principle.

[0058] S400: Perform multiple cyclic matchings on each unmatched license plate and its corresponding candidate matching vehicle table after the initial matching according to the cyclic matching principle, until the number of vehicles in the candidate matching vehicle table corresponding to each unmatched license plate after the cyclic matching is greater than or equal to the number threshold.

[0059] S500 performs a final match on each unmatched license plate and its corresponding candidate vehicle table after the cyclic matching according to the preset area matching rules.

[0060] Specifically, in this embodiment, for vehicle structuring, detection can only obtain all vehicles and license plates in the monitored image, and cannot directly obtain the association between vehicles and license plates. When a license plate is on only one vehicle, direct matching is possible. When a license plate is on multiple vehicles, this typically occurs when a large vehicle is next to a smaller vehicle while turning, as shown in the attached diagram. Figure 1As shown, at this point, license plates cannot be directly matched with vehicles, and an additional strategy is required for matching. Therefore, this invention first obtains a candidate matching vehicle table for each license plate, then, based on the candidate matching vehicle table and the vehicle-license plate confidence matching results, performs initial vehicle-license plate matching according to the priority matching principle, performs multiple cyclic matchings on the remaining unmatched license plates and their corresponding candidate matching vehicle tables according to the cyclic matching principle, and finally performs a final matching on the remaining unmatched license plates and their corresponding candidate matching vehicle tables according to the preset area matching rule, so as to achieve a one-to-one accurate match between all vehicles and license plates.

[0061] Preferably, in another embodiment of this application, the step "S200, calculating the license plate confidence score corresponding to each of the multiple license plate frames within each vehicle frame, and performing vehicle license plate confidence score matching based on the license plate confidence score" specifically includes the following steps:

[0062] When multiple license plate frames are detected within a vehicle frame, the license plate confidence score for each frame is calculated, and the license plate with the highest confidence score is selected to match one of the detected vehicles.

[0063] Specifically, in this embodiment, before applying the priority matching principle in the initial matching, the low-quality license plate filtering principle needs to be considered, see details below. Figure 1 In cases where a vehicle has multiple license plates, the license plate with the higher confidence score is selected as the matching license plate for that vehicle. Specifically, the license plate confidence score is calculated by multiplying the confidence score of the general license plate detection algorithm by the confidence score of the license plate character recognition algorithm. This can accurately exclude false detections of license plates or multiple license plates (for example, some trucks may have a license plate printed on the body), and only the one with the higher confidence score is selected.

[0064] It should be noted that general license plate detection algorithms, such as the YOLO series, the latest being YOLOv8, will predict the score of the detection box.

[0065] License plate character recognition algorithms, such as the PPOCR series, the latest being PPOCR_v4, all predict character probabilities.

[0066] Preferably, in another embodiment of this application, the step "S300, traversing and calculating the candidate matching vehicle table corresponding to each license plate in the license plate frame according to a preset threshold matching rule in the vehicle frame set" specifically includes the following steps:

[0067] The area ratio A corresponding to each license plate i in the license plate frame is calculated by traversing the vehicle frame set as follows:

[0068] A = Area of ​​intersection between license plate i and vehicle j / Area of ​​license plate i;

[0069] When the area ratio corresponding to vehicle j is detected to be greater than the area ratio threshold, vehicle j is added to the candidate matching vehicle table of license plate i.

[0070] Specifically, in this embodiment, based on a preset threshold matching, it is determined whether the intersection area of ​​license plate i and vehicle j / the area of ​​license plate i is greater than an area ratio threshold (the area ratio threshold can be set to any value between 0 and 1, such as 0.7, 0.8, 0.9, etc.). If it is greater than the area ratio threshold, vehicle j is added to the candidate matching vehicle list of license plate i. The specific implementation is as follows:

[0071] Preferably, in another embodiment of this application, the step "S300, performing initial matching of vehicle license plates according to the candidate matching vehicle table corresponding to each license plate and the priority matching principle" specifically includes the following steps:

[0072] Get the number of vehicles in the candidate matching vehicle table corresponding to each license plate;

[0073] When the number of vehicles in the candidate matching vehicle table corresponding to a license plate is equal to 1, it is determined whether the unique vehicle in the candidate matching vehicle table has already been matched. If not, this license plate is matched with the unique vehicle in the corresponding candidate matching vehicle table; if the unique vehicle has already been matched (see reference...). Figure 1 It's possible that license plate 0 matches vehicle 0, and when iterating to license plate 1, it also only matches vehicle 0. We need to determine whether the confidence of the current license plate (license plate 1) is greater than the confidence of the previously matched license plate (license plate 0). If it is greater, we update the current license plate to match the vehicle; otherwise, we leave it as the original match (i.e., if the confidence of license plate 1 is greater than that of license plate 0, then we match license plate 1 with vehicle 0).

[0074] If the number of vehicles in the candidate matching vehicle table corresponding to a license plate is equal to 0, then there is no matching vehicle for the detected license plate.

[0075] See also Figure 2 As shown, the area ratio A and the area ratio threshold are used to determine the specific intersection area of ​​the license plate and the vehicle / the area of ​​the license plate, as follows:

[0076] The intersection area of ​​license plate 0 and vehicle 0 / the area of ​​license plate 0 = 1;

[0077] The intersection area of ​​license plate 0 and vehicle 1 / the area of ​​license plate 0 = 0;

[0078] The intersection area of ​​license plate 1 and vehicle 0 / the area of ​​license plate 1 = 0;

[0079] The intersection area of ​​license plate 1 and vehicle 1 / the area of ​​license plate 1 = 1;

[0080] Then license plate 0 matches vehicle 0, and license plate 1 matches vehicle 1.

[0081] See also Figure 1 As shown, the intersection area of ​​the license plate and the vehicle / the area of ​​the license plate are as follows:

[0082] The intersection area of ​​license plate 0 and vehicle 0 / the area of ​​license plate 0 = 1;

[0083] The intersection area of ​​license plate 0 and vehicle 1 / the area of ​​license plate 0 = 1;

[0084] The intersection area of ​​license plate 1 and vehicle 0 / the area of ​​license plate 1 = 0;

[0085] The intersection area of ​​license plate 1 and vehicle 1 / the area of ​​license plate 1 = 1;

[0086] At this point, we can first find the license plate that matches only one vehicle and get priority matching rights (priority matching principle - the license plate that matches only one vehicle has priority matching rights). That is, if license plate 1 matches only vehicle 1, then license plate 1 matches vehicle 1.

[0087] License plate 0 and vehicle 0 and vehicle 1 (i.e., vehicle 0 and vehicle 1 exist in the candidate matching vehicle table corresponding to license plate 0) all meet the area ratio threshold (if the area ratio threshold is 0.5) judgment requirement. Since vehicle 1 has been matched with license plate 1 first, license plate 0 can only be matched with vehicle 0.

[0088] It should be noted that the appearance of license plate 0 in the field of vision of vehicle 1 is a changing process. The value of the intersection area of ​​license plate 0 and vehicle 1 / the area of ​​license plate 0 gradually increases from 0 to 1.

[0089] Therefore, for the i-th license plate platei, initialize its candidate matching vehicle list condidate = []; traverse the vehicles, for the j-th vehicle vehiclej, calculate the intersection area intraij of platei and vehiclej. If intraij / platei > threadhold, then condidate.append(j); if the number of condidates is 0, it means no match was found, that is, when the number of vehicles in the candidate matching vehicle list corresponding to a certain license plate is equal to 0, then there is no matching vehicle for that license plate; if the number of condidates is 1, it means a match was found, that is, this license plate is matched with the only vehicle in the corresponding candidate matching vehicle list; matched_ids[j] = i (that is, vehicle number j matches license plate number i). If the number of condidates is greater than 1, then multiple_matched_plates[i] = condidate. At this time, license plate i is not initially matched, but continues to be matched in the subsequent matching or final matching process.

[0090] Preferably, in another embodiment of this application, step "S400, the step of "performing multiple cyclic matchings on each unmatched license plate and its corresponding candidate matching vehicle table after the initial matching according to the cyclic matching principle, until the number of vehicles in the candidate matching vehicle table corresponding to each unmatched license plate after cyclic matching is greater than or equal to the number threshold" specifically includes the following steps:

[0091] S410: After the initial matching, delete the successfully matched vehicles from the candidate matching vehicle table of the remaining unmatched license plates, and rematch the license plates of the remaining vehicles according to the priority matching principle.

[0092] S420: Delete the successfully matched vehicles from the candidate matching vehicle table of the remaining unmatched license plates after the second matching, and rematch the license plates of the remaining vehicles according to the priority matching principle.

[0093] S430, the matching process is repeated until the number of vehicles in the candidate matching vehicle table corresponding to each unmatched license plate after the cyclic matching is detected to be greater than or equal to the number threshold (the number threshold can be set to a natural number such as 2, 3, or 4, but for more accurate matching, the number threshold is set to 2). The cyclic matching is then stopped.

[0094] Preferably, in another embodiment of this application, the step "S410, deleting successfully matched vehicles from the candidate matching vehicle table of the remaining unmatched license plates after the initial matching, and re-matching the remaining vehicle license plates according to the priority matching principle" specifically includes the following steps:

[0095] Get the remaining unmatched license plates when the number of vehicles in the candidate matching vehicle table is greater than or equal to the number threshold;

[0096] Check if there are any vehicles in the candidate matching vehicle table for the remaining unmatched license plates that were successfully matched in the initial matching;

[0097] If there are vehicles that have been successfully matched in the initial matching, then delete the corresponding matched vehicles from the candidate matching vehicle table for each license plate.

[0098] If the number of vehicles in the candidate matching vehicle table corresponding to a license plate is equal to 1 after deleting the matched vehicles, then the detected license plate is matched with the unique vehicle in the corresponding candidate matching vehicle table.

[0099] If the number of vehicles in the candidate matching vehicle table corresponding to a license plate is equal to 0 after deleting the matched vehicles, then there are no matched vehicles for the detected license plate.

[0100] Specifically, in this embodiment, a priority matching strategy has been implemented during the initial matching process—license plates that match only one vehicle have priority matching rights. Since one license plate can only be matched with one vehicle, vehicles that have been successfully matched in the initial matching need to be deleted from the candidate matching vehicle table of the remaining unmatched license plates, that is, vehicle j is deleted from the matching result of platei.

[0101] After deleting the matched vehicles, the license plates of the remaining vehicles are matched again. The matching process is the same as the initial matching process, and the matching is performed according to the priority matching principle. Specifically, if the number of matching results (condidate) of platei is 0, platei is deleted from multiple_matched_plates. If the number of matching results (condidate) of platei is 1, then the result (denoted as vehicle r) matches the license plate i, that is, matched_ids[r] = i, and platei is deleted from multiple_matched_plates.

[0102] Preferably, in another embodiment of this application, the step "S400, performing final matching on each unmatched license plate and its corresponding candidate matching vehicle table after cyclic matching according to a preset area matching rule" specifically includes the following steps:

[0103] S410: Obtain the remaining unmatched license plates and their corresponding candidate matching vehicle tables after the loop matching, and delete the successfully matched vehicles in the candidate matching vehicle tables of the remaining unmatched license plates.

[0104] S420: Select the vehicle with the smallest vehicle frame area from the candidate matching vehicle table for each of the remaining unmatched license plates, and match each of the remaining unmatched license plates one by one.

[0105] Specifically, in this embodiment, the algorithm iterates through the loop and calculates the result. If multiple_matched_plates is empty, the algorithm ends. If multiple_matched_plates is not empty, meaning there are still unmatched license plates, the vehicle with the smallest vehicle frame area in the candidate matching vehicle table for each license plate is selected as the best match.

[0106] The loop generates multiple matching results, multiple_matched_plates. For the i-th license plate platei, calculate the area of ​​the vehicle frame matched by platei. Let v be the vehicle with the smallest area, then matched_ids[v] = i.

[0107] Therefore, this invention applies three matching rules, as follows:

[0108] 1. Threshold rule: If the intersection of license plate i and vehicle j / the area of ​​license plate i is greater than a threshold, then vehicle j is added to the candidate matching vehicle list of license plate i.

[0109] 2. Priority matching strategy – License plates that match only one vehicle have priority matching rights.

[0110] 3. Area matching rule: Select the vehicle with the smallest vehicle frame area in the candidate matching vehicle table for each license plate as the best match.

[0111] Meanwhile, for low-density scenarios, the strategy focuses on accuracy, and the changing proportions of the intersection between license plates and vehicle frames can be considered to further improve accuracy. For example... Figure 1 , 2 As shown, the appearance of license plate 0 in the field of vision of vehicle 1 is a changing process. The value of the intersection of license plate 0 and vehicle 1 / the area of ​​license plate 0 gradually increases from 0 to 1. When this process is recognized, it can be more accurately determined that license plate 0 and vehicle 1 do not match.

[0112] For both high- and low-intensity scenarios, the strategy focuses on quickly matching all targets, while some accuracy can be sacrificed. This can be achieved through the threshold rules and priority matching strategy described above.

[0113] See also Figure 4 As shown, this embodiment of the invention also provides a vehicle license plate matching system, including:

[0114] The dataset module is used to obtain the vehicle frame set corresponding to multiple vehicles and the license plate frame set corresponding to multiple license plates;

[0115] The confidence matching module is connected in communication with the dataset module. It is used to calculate the confidence of the license plates corresponding to the multiple license plate frames in each vehicle frame, and to perform vehicle license plate confidence matching based on the license plate confidence.

[0116] The initial matching module is communicatively connected to the dataset module and the confidence matching module. It is used to traverse and calculate the candidate matching vehicle table corresponding to each license plate in the license plate frame according to the preset threshold matching rule in the vehicle frame set, and perform initial matching of vehicle license plates according to the candidate matching vehicle table corresponding to each license plate and the vehicle license plate confidence matching result, and according to the priority matching principle.

[0117] A cyclic matching module, communicatively connected to the initial matching module, is used to perform multiple cyclic matching operations on each unmatched license plate remaining after the initial matching and its corresponding candidate matching vehicle table, according to the cyclic matching principle, until the number of vehicles in the candidate matching vehicle table corresponding to each unmatched license plate remaining after cyclic matching, after deleting matched vehicles, is greater than or equal to a number threshold, at which point the cyclic matching stops; and,

[0118] The area matching module is communicatively connected to the cyclic matching module and is used to perform final matching on each unmatched license plate and its corresponding candidate matching vehicle table after cyclic matching according to a preset area matching rule.

[0119] Therefore, the present invention first obtains a candidate matching vehicle table corresponding to each license plate, then performs initial matching of vehicles and license plates according to the priority matching principle, performs multiple cyclic matching of the remaining unmatched license plates and their corresponding candidate matching vehicle tables according to the cyclic matching principle, and finally performs final matching of the remaining unmatched license plates and their corresponding candidate matching vehicle tables according to the preset area matching rule, so as to achieve a one-to-one accurate matching of all vehicles and license plates.

[0120] Specifically, this embodiment corresponds one-to-one with the above method embodiments. The functions of each module have been described in detail in the corresponding method embodiments, so they will not be repeated here.

[0121] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements all or part of the method steps of the above method.

[0122] The present invention can implement all or part of the processes in the above methods, or it can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0123] Based on the same inventive concept, embodiments of this application also provide an electronic device, including a memory and a processor. The memory stores a computer program that runs on the processor. When the processor executes the computer program, it implements all or part of the method steps described above.

[0124] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the computer device, connecting all parts of the computer device through various interfaces and lines.

[0125] Memory can be used to store computer programs and / or modules. The processor performs various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area can store the operating system and at least one application program required for a function (e.g., sound playback, image playback, etc.); the data storage area can store data created based on the use of the mobile phone (e.g., audio data, video data, etc.). Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, SmartMedia Cards (SMC), Secure Digital (SD) cards, Flash Cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0126] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, servers, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0127] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), servers, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0128] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0129] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0130] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for matching vehicle license plates, characterized in that, Specifically, the following steps are included: Obtain the vehicle frame set corresponding to multiple vehicles and the license plate frame set corresponding to multiple license plates; Calculate the license plate confidence score corresponding to each license plate frame within each vehicle frame, and perform vehicle license plate confidence score matching based on the license plate confidence score. In the vehicle frame set, the candidate matching vehicle table corresponding to each license plate in the license plate frame is calculated by traversing the vehicle frame according to the preset threshold matching rule, and the initial matching of vehicle license plates is performed according to the candidate matching vehicle table corresponding to each license plate and the vehicle license plate confidence matching result, based on the priority matching principle. The cyclic matching principle is used to perform multiple cyclic matchings on each unmatched license plate and its corresponding candidate matching vehicle table after the initial matching, until the number of vehicles in the candidate matching vehicle table corresponding to each unmatched license plate after the cyclic matching is greater than or equal to the number threshold is detected. According to the preset area matching rules, the remaining unmatched license plates after the cyclic matching and their corresponding candidate matching vehicle tables are finally matched; The step of "initially matching vehicle license plates according to the candidate matching vehicle table and vehicle license plate confidence matching results, and according to the priority matching principle" specifically includes the following steps: Get the number of vehicles in the candidate matching vehicle table corresponding to each license plate; When the number of vehicles in the candidate matching vehicle table corresponding to a license plate is equal to 1, and the only vehicle in the candidate matching vehicle table fails to match the license plate with the confidence level, then the detected license plate is matched with the only vehicle in the corresponding candidate matching vehicle table. When the number of vehicles in the candidate matching vehicle table corresponding to a license plate is equal to 1, and the only vehicle in the candidate matching vehicle table has been matched with the license plate through confidence, the license plate confidence of the detected license plate is calculated and compared with the license plate confidence of the only vehicle that has been matched with the license plate through confidence. The license plate corresponding to the license plate with the highest license plate confidence is selected to update the matching. If the number of vehicles in the candidate matching vehicle table corresponding to one of the license plates is equal to 0, then there is no matching vehicle for that license plate. The step of "performing multiple cyclic matchings on each unmatched license plate and its corresponding candidate matching vehicle table after the initial matching according to the cyclic matching principle, until the number of vehicles in the candidate matching vehicle table corresponding to each unmatched license plate after cyclic matching is greater than or equal to the number threshold" specifically includes the following steps: After the initial matching, delete the successfully matched vehicles from the candidate matching vehicle table of the remaining unmatched license plates, and then match the remaining vehicle license plates again according to the priority matching principle. After the second matching, delete the successfully matched vehicles from the candidate matching vehicle table of the remaining unmatched license plates, and then match the remaining vehicle license plates again according to the priority matching principle. The matching process is repeated until the number of vehicles in the candidate matching vehicle table corresponding to each unmatched license plate after the cyclic matching is detected is greater than or equal to the number threshold.

2. The vehicle license plate matching method as described in claim 1, characterized in that, The step of "calculating the license plate confidence score corresponding to each license plate frame within each vehicle frame, and performing vehicle license plate confidence score matching based on the license plate confidence score" specifically includes the following steps: When multiple license plate frames are detected within a vehicle frame, the license plate confidence score for each frame is calculated, and the license plate with the highest confidence score is selected to match one of the detected vehicles.

3. The vehicle license plate matching method as described in claim 1, characterized in that, The step of "traversing and calculating the candidate matching vehicle table corresponding to each license plate in the license plate frame according to the preset threshold matching rule in the vehicle frame set" specifically includes the following steps: The area ratio A corresponding to each license plate i in the license plate frame is calculated by traversing the vehicle frame set as follows: A = Area of ​​intersection between license plate i and vehicle j / Area of ​​license plate i; When the area ratio corresponding to vehicle j is detected to be greater than the area ratio threshold, vehicle j is added to the candidate matching vehicle table of license plate i.

4. The vehicle license plate matching method as described in claim 1, characterized in that, The step of "deleting successfully matched vehicles from the candidate matching vehicle table of the remaining unmatched license plates after the initial matching, and re-matching the remaining vehicle license plates according to the priority matching principle" specifically includes the following steps: Get the remaining unmatched license plates when the number of vehicles in the candidate matching vehicle table is greater than or equal to the number threshold; Check if there are any vehicles in the candidate matching vehicle table for the remaining unmatched license plates that were successfully matched in the initial matching; If there are vehicles that have been successfully matched in the initial matching, then delete the corresponding matched vehicles from the candidate matching vehicle table for each license plate. If the number of vehicles in the candidate matching vehicle table corresponding to a license plate is equal to 1 after deleting the matched vehicles, then the detected license plate is matched with the unique vehicle in the corresponding candidate matching vehicle table. If the number of vehicles in the candidate matching vehicle table corresponding to a license plate is equal to 0 after deleting the matched vehicles, then there are no matched vehicles for the detected license plate.

5. The vehicle license plate matching method as described in claim 1, characterized in that, The step of "performing a final match on each unmatched license plate and its corresponding candidate vehicle table after cyclic matching according to a preset area matching rule" specifically includes the following steps: Get the remaining unmatched license plates after the loop matching and their corresponding candidate matching vehicle table, and delete the successfully matched vehicles in the candidate matching vehicle table of the remaining unmatched license plates. Select the vehicle with the smallest vehicle frame area from the candidate matching vehicle table for each of the remaining unmatched license plates, and match each of the remaining unmatched license plates one by one.

6. A vehicle license plate matching system, characterized in that, include: The dataset module is used to obtain the vehicle frame set corresponding to multiple vehicles and the license plate frame set corresponding to multiple license plates; The confidence matching module is connected in communication with the dataset module. It is used to calculate the confidence of the license plates corresponding to the multiple license plate frames in each vehicle frame, and to perform vehicle license plate confidence matching based on the license plate confidence. The initial matching module is communicatively connected to the dataset module and the confidence matching module. It is used to traverse and calculate the candidate matching vehicle table corresponding to each license plate in the license plate frame according to the preset threshold matching rule in the vehicle frame set, and perform initial matching of vehicle license plates according to the candidate matching vehicle table corresponding to each license plate and the vehicle license plate confidence matching result, and according to the priority matching principle. The cyclic matching module is communicatively connected to the initial matching module. It is used to perform multiple cyclic matchings on each unmatched license plate after the initial matching and its corresponding candidate matching vehicle table according to the cyclic matching principle, until the number of vehicles in the candidate matching vehicle table corresponding to each unmatched license plate after the cyclic matching is greater than or equal to the number threshold is detected. as well as, An area matching module, communicatively connected to the cyclic matching module, is used to perform a final match on each unmatched license plate and its corresponding candidate vehicle table after cyclic matching according to a preset area matching rule; The step of "initially matching vehicle license plates according to the candidate matching vehicle table and vehicle license plate confidence matching results, and according to the priority matching principle" specifically includes the following steps: Get the number of vehicles in the candidate matching vehicle table corresponding to each license plate; When the number of vehicles in the candidate matching vehicle table corresponding to a license plate is equal to 1, and the only vehicle in the candidate matching vehicle table fails to match the license plate with the confidence level, then the detected license plate is matched with the only vehicle in the corresponding candidate matching vehicle table. When the number of vehicles in the candidate matching vehicle table corresponding to a license plate is equal to 1, and the only vehicle in the candidate matching vehicle table has been matched with the license plate through confidence, the license plate confidence of the detected license plate is calculated and compared with the license plate confidence of the only vehicle that has been matched with the license plate through confidence. The license plate corresponding to the license plate with the highest license plate confidence is selected to update the matching. If the number of vehicles in the candidate matching vehicle table corresponding to one of the license plates is equal to 0, then there is no matching vehicle for that license plate. The step of "performing multiple cyclic matchings on each unmatched license plate and its corresponding candidate matching vehicle table after the initial matching according to the cyclic matching principle, until the number of vehicles in the candidate matching vehicle table corresponding to each unmatched license plate after cyclic matching is greater than or equal to the number threshold" specifically includes the following steps: After the initial matching, delete the successfully matched vehicles from the candidate matching vehicle table of the remaining unmatched license plates, and then match the remaining vehicle license plates again according to the priority matching principle. After the second matching, delete the successfully matched vehicles from the candidate matching vehicle table of the remaining unmatched license plates, and then match the remaining vehicle license plates again according to the priority matching principle. The matching process is repeated until the number of vehicles in the candidate matching vehicle table corresponding to each unmatched license plate after the cyclic matching is detected is greater than or equal to the number threshold.

7. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the vehicle license plate matching method as described in any one of claims 1 to 5.

8. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that, When the processor runs the computer program, it implements the vehicle license plate matching method as described in any one of claims 1 to 5.

Images