Method and related device for associating overlapping license plates of multiple vehicles

Abstract

The present application discloses a method for associating overlapping license plates of multiple vehicles and a related device, the method comprising: acquiring a target video, detecting a target detection frame corresponding to a target vehicle in the target video, determining the moving direction of the target vehicle, and determining the license plate of the target vehicle in the target detection frame; selecting a reference point from the target detection frame according to the moving direction of the target vehicle and the target detection frame; determining a detection line matching the target detection frame according to the moving direction of the target vehicle and the reference point, and determining the license plate associated area of ​​the target vehicle in the target detection frame according to the reference point and the detection line; associating the license plate located in the license plate associated area with the target vehicle corresponding to the target detection frame where the license plate associated area is located. The method of the present application can improve the accuracy of license plate association.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for associating overlapping license plates of multiple vehicles and a related device. Background Art

[0002] In terms of license plate recognition technology, the development of artificial intelligence has made great progress. High-precision license plate recognition technology has been widely used in various fields of intelligent transportation. At present, license plate recognition technology is quite mature, but when multiple vehicles in adjacent lanes are close, due to the angle limitation of the acquisition device, the image captured by the acquisition device will appear to be included in the target detection frame of multiple vehicles at the same time, resulting in incorrect binding of the license plate and the vehicle, which leads to false information. In view of this, how to improve the accuracy of license plate association has become an urgent problem to be solved. Summary of the invention

[0003] The present application provides a method and related device for associating overlapping license plates of multiple vehicles, which can improve the accuracy of license plate association.

[0004] In a first aspect, the present application provides a method for associating overlapping license plates of multiple vehicles, the method comprising: acquiring a target video, detecting a target detection frame corresponding to a target vehicle in the target video, determining a direction of movement of the target vehicle, and determining the license plate of the target vehicle within the target detection frame; selecting a reference point from the target detection frame based on the direction of movement of the target vehicle and the target detection frame; determining a detection line that matches the target detection frame based on the direction of movement of the target vehicle and the reference point, and determining a license plate association area of ​​the target vehicle in the target detection frame based on the reference point and the detection line; and associating a license plate located in the license plate association area with the target vehicle corresponding to the target detection frame where the license plate association area is located.

[0005] A second aspect of the present application provides an electronic device, which includes: a memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method of the first aspect.

[0006] A third aspect of the present application provides a computer-readable storage medium having program data stored thereon, and the method of the first aspect is implemented when the program data is executed by a processor.

[0007] Different from the prior art, the present application obtains a target video, detects a target detection frame of a target vehicle in the target video, and determines the direction of motion of the target vehicle and the license plate in the target detection frame, then selects a reference point from the target detection frame according to the direction of motion of the target vehicle and the target detection frame, and then determines the detection line that matches the target detection frame according to the direction of motion of the target vehicle and the reference point, determines the license plate association area of ​​the target vehicle in the target detection frame according to the reference point and the detection line, and finally associates the license plate located in the license plate association area with the target vehicle corresponding to the target detection frame where the license plate association area is located. It can be seen from the above content that the present application associates the license plate and the target vehicle through the license plate association area. In addition to being based on the target detection frame, the license plate association area is also based on the direction of motion of the target vehicle, the reference point on the target detection frame, and the detection line. Therefore, the range of the license plate association area is limited by the above method so that the size of the license plate association area is smaller than the size of the target detection frame, thereby avoiding the problem of license plates being misassociated when multiple vehicles overlap due to the large size of the target detection frame, and ultimately improving the accuracy of license plate association. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0009] Figure 1 It is a flowchart of one implementation method of the method for associating overlapping license plates of multiple vehicles of the present application;

[0010] Figure 2 This is a schematic diagram of the association of overlapping license plates of multiple vehicles in an application scenario of the present application;

[0011] Figure 3 yes Figure 1 A schematic flow chart of an implementation method of step S100;

[0012] Figure 4 yes Figure 3 A schematic diagram of an implementation method corresponding to step S120 in FIG.

[0013] Figure 5 yes Figure 4 Schematic diagram of straight line fitting for each marked point in ;

[0014] Figure 6 yes Figure 1 A schematic flow chart of an implementation method of step S200;

[0015] Figure 7 yes Figure 6 A schematic diagram of an application scenario corresponding to step S210;

[0016] Figure 8 yes Figure 6 A schematic diagram of an application scenario corresponding to step S220;

[0017] Fig. 9 yes Figure 1 A schematic flow chart of an implementation method of step S300;

[0018] Fig.10 yes Figure 1 A schematic diagram of a flow chart of an implementation method before step S400;

[0019] Fig.11 This is a schematic diagram of the association of overlapping license plates of multiple vehicles in another application scenario of the present application;

[0020] Fig.12 It is a structural schematic diagram of an implementation method of the association device of the present application;

[0021] Fig.13 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0022] Fig.14 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments, and different implementation methods can be adaptively combined. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] See also Figure 1 The present application provides a method for associating overlapping license plates of multiple vehicles, the method comprising:

[0025] S100: Acquire a target video, detect a target detection frame corresponding to a target vehicle in the target video, determine a moving direction of the target vehicle, and determine a license plate of the target vehicle in the target detection frame.

[0026] Specifically, the target video includes a video of the movement of a target vehicle. A target detection frame of the target vehicle in the target video is detected by a vehicle detection algorithm. The target detection frame is used to determine the outline of the target vehicle. Generally speaking, the target detection frame can be a rectangular frame or a polygonal frame. The present application does not limit the specific shape of the target detection frame.

[0027] Further, the moving direction of the target vehicle is determined by the moving direction of the target detection frame detected in the target video. It is understandable that there must be a license plate of the target vehicle in the target detection frame, so the license plate can be further directly obtained in the target detection frame.

[0028] In one application scenario, combined with Figure 2 , by detecting the target video, a first target detection frame A of the first target vehicle and a second target detection frame B of the second target vehicle are detected, and then according to the movement of the target vehicles in the target video, a first moving direction Z1 of the first target vehicle and a second moving direction Z2 of the second target vehicle are determined. The first moving direction Z1 and the second moving direction Z2 may be the same or different. The first license plate A1 of the first target vehicle can be detected in the first target detection frame A, and the second license plate B1 of the second target vehicle can be detected in the second target detection frame B.

[0029] It should be noted that in other application scenarios, the number of target vehicles may be other, and the moving directions of the target vehicles may also be other directions.

[0030] S200: Selecting a reference point from the target detection frame according to the moving direction of the target vehicle and the target detection frame.

[0031] Specifically, the direction of movement of the target vehicle is related to the shooting position angle of the camera. For example, when the camera is shooting toward the left front, the direction of movement of the target vehicle includes from the right front to the left rear and from the left rear to the right front, wherein the direction from the right front to the left rear is the target vehicle's entry direction, and the direction from the left rear to the right front is the target vehicle's exit direction. The above is the normal driving direction of the target vehicle. If reversing occurs, the corresponding exit and entry are just the opposite. For another example, when the camera is shooting toward the right front, the direction of movement of the target vehicle includes from the left front to the right rear and from the right rear to the left front, wherein the direction from the left front to the right rear is the target vehicle's entry direction, and the direction from the right rear to the left front is the target vehicle's exit direction. The above is the normal driving direction of the target vehicle. If reversing occurs, the corresponding exit and entry are just the opposite. It can be seen that the position of the camera determines the direction of movement of the target vehicle in the target video.

[0032] Furthermore, when the camera is facing the left front, no matter the target vehicle is moving from the right front to the left rear, or from the left rear to the right front, no matter it is driving normally or reversing, no matter the license plate is located at the front or rear of the vehicle, the license plate of the target vehicle that can be photographed by the camera is in the left area of ​​the target vehicle, that is, in the left area of ​​the target detection frame; similarly, when the camera is facing the right front, no matter the target vehicle is moving from the left front to the right rear, or from the right rear to the left front, no matter it is driving normally or reversing, no matter the license plate is located at the front or rear of the vehicle, the license plate of the target vehicle that can be photographed by the camera is in the right area of ​​the target vehicle, that is, in the right area of ​​the target detection frame. From the above analysis, it can be understood that the target vehicle's moving direction determines the position of the license plate in the target detection frame, but determining the approximate area position of the license plate by the moving direction is relatively rough.

[0033] The present application further selects a reference point related to the direction of movement of the target vehicle from the target detection frame. The reference point is used to establish a license plate association area in the subsequent steps (the specific content of the license plate association area can be found in the content of the subsequent steps), so that the license plate is subsequently associated with the target vehicle through the license plate association area. Compared with the former, the reference point can be used to more accurately determine the position of the license plate association area in the subsequent steps, thereby reducing association errors and improving the accuracy of association. The selection of the reference point will affect the size of the subsequent license plate association area, and the size of the license plate association area will affect the association effect. Preferably, the reference point can be selected on the target detection frame, such as Figure 2 The first reference point P1 in the first target detection frame A and the second reference point P2 in the second target detection frame B. At the same time, in order to ensure the accuracy of the subsequent license plate association area, the height of the reference point from the lower frame of the target detection frame is greater than the height of the license plate from the lower frame of the target detection frame. Of course, it can be understood that the present application does not impose specific restrictions on the selection of the specific position of the reference point, and the position of the reference point can be adjusted according to actual conditions in different embodiments.

[0034] S300: Determine a detection line that matches a target detection frame according to a moving direction and a reference point of the target vehicle, and determine a license plate associated area of ​​the target vehicle in the target detection frame according to the reference point and the detection line.

[0035] Specifically, the license plate association area of ​​the present application is used in the subsequent steps to associate the license plate in the license plate association area with the target vehicle of the target detection frame corresponding to the license plate association area. In order to better determine the specific position of the license plate association area, in addition to the reference point in the above step S200, this step further provides a detection line, which is determined according to the movement direction of the target vehicle and the reference point. More specifically, the extension direction of the detection line is parallel to or close to the movement direction of the target vehicle. For example, Figure 2, the extension direction of the first detection line A3 is the same as the first movement direction Z1 of the first target vehicle, and the extension direction of the second detection line B3 is the same as the second movement direction Z2 of the second target vehicle. And, in order to better determine the position of the detection line in the target detection frame, the detection line passes through the reference point, or the detection line is set on one side of the reference point. Thus, the license plate associated area of ​​the target vehicle is determined in the target detection frame through the reference point and the detection line, so that the license plate associated area of ​​the target vehicle is further defined, wherein the reference point can be used to determine the height of the license plate associated area, and the detection line can be used to determine the width of the license plate associated area. For example, in Figure 2 In the process, a first intersection point A4 with the lower frame of the first target detection frame A is determined according to the first reference point P1 and the first detection line A3, thereby determining the position of the first license plate associated area A2 in the first target detection frame A. Similarly, a second intersection point B4 with the lower frame of the second target detection frame B is determined according to the second reference point P2 and the second detection line B3, thereby determining the position of the second license plate associated area B2 in the second target detection frame B. S400: Associating the license plate located in the license plate associated area with the target vehicle corresponding to the target detection frame where the license plate associated area is located.

[0036] Specifically, the license plate association area of ​​the target vehicle is within the target detection frame of the target vehicle, and the range is further limited by the direction of movement, reference point and detection line. The license plate association area is significantly smaller in size than the target detection frame. In the case of multiple overlapping vehicles, that is, when the target detection frames of multiple detected target vehicles overlap, but the sizes of their corresponding license plate association areas are reduced, the method of associating the license plate located in the license plate association area with the target vehicle corresponding to the target detection frame where the license plate association area is located can avoid the problem of license plates being misassociated when multiple vehicles overlap due to the large size of the target detection frame, thereby improving the accuracy of associating license plates.

[0037] In one embodiment, see Figure 3 , the moving direction of the target vehicle is determined in the above step S100, including:

[0038] S110: Acquire a target image in a target video; wherein the target video includes a plurality of target images related to the movement of a target vehicle.

[0039] Specifically, a target video is acquired through camera acquisition, and at least a plurality of target images in the target video have a target vehicle, and the plurality of target images contain motion information of the target vehicle. The target image can be selected and extracted from the target video according to the size of computing resources.

[0040] S120: Detect a target vehicle in the target image, and obtain a target detection frame of the target vehicle and marking points of the target detection frame of the target vehicle.

[0041] Specifically, a target detection algorithm can be used for the target image, such as a YOLO series deep learning target detection method. A detection model is built to train the target scene training data set. The trained detection model is used to detect the target vehicle in the target video to obtain the target detection frame of the target vehicle and the marking point of the target detection frame of the target vehicle. The marking point can be determined directly from the target vehicle, such as the midpoint of the bottom edge of the target vehicle; or it can be determined from the target detection frame of the target vehicle, such as the midpoint of the bottom frame of the target detection frame. Of course, the position of the marking point can be set according to the situation, as long as it is ensured that the marking point obtained in the target video is a fixed position point on the target vehicle or a fixed position point on the target detection frame.

[0042] In one application scenario, combined with Figure 4 The target video includes multiple target images related to the movement of the first target vehicle. The first target vehicle moves along the first movement direction Z1. Therefore, the marking point C on the first target detection frame A in the multiple target images can be obtained by detection. The marking point C is roughly arranged along the first movement direction Z1.

[0043] S130: Determine the slope of the moving direction of the target vehicle according to the marking points in the plurality of target images.

[0044] Specifically, the target vehicle is moving, and the marking points in the multiple target images are also moving with the target vehicle, so the moving direction of the target vehicle can be determined according to the arrangement direction of the marking points in the multiple target images. At the same time, the slope of the moving direction is determined according to the specific positions of the multiple marking points in the target image. It can be understood that the slope can accurately reflect the specific angle of the moving direction of the target vehicle based on the multiple target images.

[0045] In one embodiment, in combination Figure 4 and Figure 5 , by fitting the marked points in multiple target images with a linear function, the slope of the target vehicle’s moving direction is obtained.

[0046] Generally speaking, the target vehicle's short-term motion trajectory can be regarded as a straight line. Linear function fitting can better reflect the target vehicle's short-term motion direction. Specifically, a straight line can be obtained by fitting multiple marking points with a linear function, and the slope of the straight line can be further obtained.

[0047] Furthermore, the linear function fitting can be calculated in a matrix form, and the linear equation can be set as ,in, is the horizontal coordinate of the line, is the horizontal coordinate of the line, is the slope of the straight line, is the intercept of the straight line, and the slope is determined according to the following equation and the intercept :

[0048] ;

[0049] ;

[0050] ;

[0051] .

[0052] The target video includes n+1 frames of target images related to the target vehicle. The coordinates of the marking points of the first frame of the target image are , the coordinates of the marker point of the target image in the second frame are , and so on, the coordinates of the marker point of the target image in the n+1th frame are , put the horizontal coordinates into the first column of matrix X, and the vertical coordinates into the first column of matrix Y, and use the above matrix formula to obtain matrix D, so as to obtain the slope and the intercept .

[0053] It should be noted that when calculating the slope Then, we can calculate the slope The inverse tangent value of is the angle between the straight line and the horizontal line. Therefore, it can be understood that the slope can make the movement direction more specific.

[0054] In one embodiment, after the above step S130, the method further includes:

[0055] The identification of the target vehicle, the type of the target vehicle, the slope of the moving direction of the target vehicle, the associated license plate information of the target vehicle and the coordinates of the target detection frame of the target vehicle are written into the tracking linked list; wherein the type of the target vehicle is obtained by detecting the target image in the target image, the associated license plate information of the target vehicle is generated based on the detected license plate, and the tracking linked list is used to store the information of the target vehicle and is updated as the number of target images increases.

[0056] Specifically, the identification of the target vehicle is unique, and different target vehicles have different identifications. The types of target vehicles include trucks, buses, and cars, etc. The slope of the moving direction of the target vehicle includes the specific value of the slope. The associated license plate information of the target vehicle includes the license plate number, license plate type, and license plate color, etc. The coordinates of the target detection frame of the target vehicle include the coordinates of the two vertices of the diagonal of the target detection frame. The above content is written into the tracking chain list. It can be understood that the above information of the target vehicle is stored through the tracking chain list, which has a one-to-one correspondence, which is convenient for subsequent viewing of the information of different target vehicles. At the same time, the tracking chain list can also be updated in real time according to the real-time increase of the number of target images. For example, when the first frame of the target image is detected, the above information of the target vehicle can be written into the tracking chain list. When a new frame of the target image is detected, the information of the target vehicle in the tracking chain list is updated.

[0057] For further information, see Figure 6 The above step S200 includes:

[0058] S210: In response to the moving direction of the target vehicle being a first target moving direction, determining that the reference point is located on a first border of the target detection frame.

[0059] Specifically, the first target movement direction may be a single direction or a series of directions. As long as the movement direction of the target vehicle is the first target movement direction, the reference point is selected on the first border of the target detection frame.

[0060] S220: In response to the moving direction of the target vehicle being the second target moving direction, determining that the reference point is located on a second border of the target detection frame.

[0061] Specifically, the second target motion direction may be a single direction or a series of directions. As long as the motion direction of the target vehicle is the second target motion direction, the reference point is selected on the second frame of the target detection frame. The first target motion direction and the second target motion direction are different, and the first frame and the second frame are also different. The reference point may select different positions depending on whether the motion direction of the target vehicle is the first target motion direction or the second target motion direction.

[0062] Furthermore, in one embodiment, the target detection frame of the target vehicle is a rectangular frame. When the slope of the target vehicle's movement direction is a positive value, that is, the first target movement direction is from the lower left to the upper right in the target video, the reference point is determined to be in the right frame of the target detection frame of the target vehicle; when the slope of the target vehicle's movement direction is a negative value, that is, the second target movement direction is from the lower right to the upper left in the target video, the reference point is determined to be in the left frame of the target detection frame of the target vehicle.

[0063] Specifically, see Figure 7 When the slope of the first moving direction Z1 of the first target vehicle is positive, the first reference point P1 can be selected on the right frame of the first target detection frame A of the first target vehicle; Figure 8 When the slope of the first moving direction Z1 of the first target vehicle is a negative value, the first reference point P1 can be selected on the left border of the first target detection frame A of the first target vehicle.

[0064] For further information, see Fig. 9 The above step S300 further includes:

[0065] S310: Determine a detection line that matches the target detection frame according to the slope of the target vehicle's moving direction and the reference point, wherein the detection line passes through the reference point and the slope of the detection line is equal to the slope of the target vehicle's moving direction.

[0066] Specifically, the slope of the detection line is equal to the slope of the moving direction of the target vehicle, and the detection line passes through the reference point, so the unique position of the detection line can be determined in the target detection frame.

[0067] In an application scenario, such as Figure 2 The slope of the first detection line A3 of the first target vehicle is equal to the slope of the first movement direction Z1 of the first target vehicle, and the first detection line A3 passes through the first reference point P1; the slope of the second detection line B3 of the second target vehicle is equal to the slope of the second movement direction Z2 of the second target vehicle, and the second detection line B3 passes through the second reference point P2.

[0068] S320: Determine the license plate associated area of ​​the target vehicle in the target detection frame according to the horizontal coordinate of the intersection of the detection line and the lower frame of the target detection frame of the target vehicle, the vertical coordinate of the reference point, the lower frame of the target detection frame of the target vehicle, and the frame opposite to the frame where the reference point is located.

[0069] Specifically, there is an intersection between the detection line of the reference point and the lower frame of the target detection frame. The distance between the intersection and the relative frame of the frame where the reference point is located is taken as the width of the license plate associated area, that is, it can be calculated by the horizontal coordinate of the intersection and the horizontal coordinate of the relative frame of the frame where the reference point is located. The distance between the reference point and the lower frame is taken as the height of the license plate associated area, that is, it can be calculated by the vertical coordinate of the reference point and the vertical coordinate of the lower frame. The license plate associated area of ​​the target vehicle is further finally determined by the above height and the above width as well as the relative frame of the lower frame and the frame where the reference point is located, and the obtained license plate associated area is a rectangular area.

[0070] In one application scenario, see Figure 2According to the horizontal coordinate of the first intersection A4, the vertical coordinate of the first reference point P1, the lower frame and the right frame of the first target detection frame A of the first target vehicle, a rectangular area can be determined, and the rectangular area is the first license plate associated area A2 of the first target vehicle. Similarly, according to the horizontal coordinate of the second intersection B4, the vertical coordinate of the second reference point P2, the lower frame and the right frame of the second target detection frame B of the second target vehicle, a rectangular area can be determined, and the rectangular area is the second license plate associated area B2 of the second target vehicle.

[0071] In an application scenario, please continue to refer to Figure 2 , the width of the first target detection box A of the first target vehicle is w , Gao Wei h , the first reference point P1 is located at the midpoint of the right frame, and the slope of the first detection line A3 is k , it can be calculated that the height of the first license plate associated area A2 is 0.5 h , width is w -0.5 h / k .

[0072] In one embodiment, see Fig.10 , before the above step S400, it also includes:

[0073] S330: In response to the target video including multiple target vehicles, obtaining the intersection between the license plate associated areas corresponding to every two target vehicles.

[0074] Specifically, the corresponding license plate associated areas of every two target vehicles in the target video are subjected to an intersection operation. If the intersection is an empty set, it means that the corresponding license plate associated areas of the two target vehicles do not overlap. If the intersection is a non-empty set, it means that the corresponding license plate associated areas of the two target vehicles overlap.

[0075] In an application scenario, such as Figure 2 , the first license plate associated area A2 of the first target vehicle and the second license plate associated area B2 of the second target vehicle do not overlap, so the intersection between the two is an empty set.

[0076] In another application scenario, Fig.11 , the first license plate associated area A2 of the first target vehicle and the second license plate associated area B2 of the second target vehicle overlap, so the intersection between the two is a non-empty set.

[0077] S340: Eliminate the license plate associated area corresponding to the target vehicle whose intersection is a non-empty set.

[0078] Specifically, since there is overlap in the license plate association areas corresponding to the target vehicles whose intersection is a non-empty set, association errors may occur when the license plate is subsequently associated with the target vehicle through the license plate association areas. Therefore, eliminating the license plate association areas corresponding to the target vehicles whose intersection is a non-empty set can improve the accuracy of license plate association.

[0079] In one implementation, the above step S400 further includes:

[0080] All reserved license plate association areas are traversed, and when the center point of the bottom edge of the license plate is in one of the license plate association areas of the target vehicle, the license plate located in the license plate association area is associated with the target vehicle corresponding to the target detection frame where the license plate association area is located.

[0081] The license plate associated areas of all target vehicles retained in the target video are detected, and the license plates in the license plate associated areas are associated with the target vehicles to which the license plate associated areas belong. It should be noted that when there are multiple target vehicles in the target video, it is sometimes not guaranteed that the license plates of each target vehicle in the target video can be completely captured, that is, the number of license plates in the target video is usually less than or equal to the number of target vehicles. The license plate associated areas within the target detection frame can be determined, but the license plates corresponding to each target vehicle in the target video are not necessarily visible. Therefore, traversing all license plate associated areas through the license plate can ensure normal association without errors. At the same time, when judging by the license plate, the bottom edge center point of the license plate can be selected. Generally speaking, the two ends of the license plate are more easily deformed, while the center point of the bottom edge of the license plate is not easily deformed. By selecting the bottom edge center point as the position of the license plate, the calculation process can be simplified.

[0082] Of course, it is understandable that the point used as the license plate position is not limited to the above embodiment. In some other implementations, multiple feature points on the license plate can be selected to calculate and obtain the final position of the license plate.

[0083] See also Fig.12 , Fig.12 It is a structural diagram of an embodiment of a device for associating overlapping license plates of multiple vehicles of the present application. The associating device 300 includes a first determination module 310, a selection module 320, a second determination module 330 and an associating module 340.

[0084] The first determination module 310 is used to acquire a target video, detect a target detection frame corresponding to a target vehicle in the target video, determine a moving direction of the target vehicle, and determine a license plate of the target vehicle in the target detection frame.

[0085] The selection module 320 is used to select a reference point from the target detection frame according to the moving direction of the target vehicle and the target detection frame.

[0086] The second determination module 330 is used to determine the detection line matched by the target detection frame according to the moving direction and reference point of the target vehicle, and to determine the license plate associated area of ​​the target vehicle in the target detection frame according to the reference point and the detection line.

[0087] The association module 340 is used to associate the license plate located in the license plate association area with the target vehicle corresponding to the target detection frame where the license plate association area is located.

[0088] Among them, the association device 300 executes the method steps in any of the above-mentioned implementation modes when working. The detailed method steps can be found in the above-mentioned related contents and will not be repeated here.

[0089] The associating device 300 may be a device having data processing and storage functions.

[0090] See also Fig.13 , Fig.13 It is a structural diagram of an embodiment of an electronic device of the present application, wherein the electronic device 40 includes a memory 401 and a processor 402 coupled to each other, wherein the memory 401 stores program data (not shown), and the processor 402 calls the program data to implement the method in any of the above embodiments. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0091] See also Fig.14 , Fig.14 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 50 stores program data 500. When the program data 500 is executed by a processor, the method in any of the above embodiments is implemented. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0092] Among them, the computer-readable storage medium 50 can specifically be a device that can store the program data 500, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or it can also be a server that stores the program data 500. The server can send the stored program data 500 to other devices for execution, or it can also execute the stored program data 500 by itself.

[0093] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for associating overlapping license plates of multiple vehicles, characterized in that: The method comprises: Acquire a target video, detect a target detection frame corresponding to a target vehicle in the target video, determine a moving direction of the target vehicle, and determine a license plate of the target vehicle within the target detection frame; wherein the target detection frame of the target vehicle is a rectangular frame; According to the moving direction of the target vehicle and the target detection frame, a reference point is selected from the target detection frame; wherein the height of the reference point from the lower frame of the target detection frame is greater than the height of the license plate from the lower frame of the target detection frame; when the slope of the moving direction of the target vehicle is a positive value, the right frame of the reference point in the target detection frame of the target vehicle is determined; when the slope of the moving direction of the target vehicle is a negative value, the left frame of the reference point in the target detection frame of the target vehicle is determined; Determine the detection line that matches the target detection frame according to the moving direction of the target vehicle and the reference point, and determine the license plate associated area of ​​the target vehicle in the target detection frame according to the reference point and the detection line; specifically include: determine the detection line that matches the target detection frame according to the slope of the moving direction of the target vehicle and the reference point, wherein the detection line passes through the reference point and the slope of the detection line is equal to the slope of the moving direction of the target vehicle; determine the license plate associated area of ​​the target vehicle in the target detection frame according to the horizontal coordinate of the intersection of the detection line and the lower frame of the target detection frame of the target vehicle, the vertical coordinate of the reference point, the lower frame of the target detection frame of the target vehicle, and the frame opposite to the frame where the reference point is located; The license plate located in the license plate association area is associated with the target vehicle corresponding to the target detection frame where the license plate association area is located.

2. The method for associating overlapping license plates of multiple vehicles according to claim 1, characterized in that: The steps of acquiring a target video, detecting a target detection frame corresponding to a target vehicle in the target video, and determining a moving direction of the target vehicle include: Acquire a target image in the target video; wherein the target video includes a plurality of target images related to the movement of the target vehicle; Detecting the target vehicle in the target image, and obtaining a target detection frame of the target vehicle and marking points of the target detection frame of the target vehicle; The slope of the moving direction of the target vehicle is determined according to the marking points in the target images.

3. The method for associating overlapping license plates of multiple vehicles according to claim 2, characterized in that: The step of determining the slope of the moving direction of the target vehicle according to the marking points in the target images comprises: The slope of the moving direction of the target vehicle is obtained by fitting a plurality of marking points in the target image through a linear function.

4. The method for associating overlapping license plates of multiple vehicles according to claim 2, characterized in that: After the step of determining the slope of the moving direction of the target vehicle according to the marking points in the target images, the method further includes: The identification of the target vehicle, the type of the target vehicle, the slope of the moving direction of the target vehicle, the associated license plate information of the target vehicle and the coordinates of the target detection frame of the target vehicle are written into a tracking linked list; wherein the type of the target vehicle is obtained by detecting the target image in the target image, the associated license plate information of the target vehicle is generated based on the detected license plate, and the tracking linked list is used to store the information of the target vehicle and is updated as the number of the target images increases.

5. The method for associating overlapping license plates of multiple vehicles according to claim 1, characterized in that: Before the step of associating the license plate located in the license plate association area with the target vehicle corresponding to the target detection frame where the license plate association area is located, the method further includes: In response to the target video including a plurality of the target vehicles, the intersection between the license plate associated regions corresponding to every two of the target vehicles is obtained; and the license plate associated regions corresponding to the target vehicles whose intersection is a non-empty set are eliminated.

6. The method for associating overlapping license plates of multiple vehicles according to claim 5, characterized in that: The step of associating the license plate located in the license plate association area with the target vehicle corresponding to the target detection frame in which the license plate association area is located comprises: All the reserved license plate associated areas are traversed, and when the center point of the bottom edge of the license plate is in one of the license plate associated areas of the target vehicle, the license plate located in the license plate associated area is associated with the target vehicle corresponding to the target detection frame where the license plate associated area is located.

7. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the steps in the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program can be executed by a processor to implement the steps in the method according to any one of claims 1 to 6.

Images