A method for associating license plate with vehicle and related device

Abstract

The present application discloses a method for associating a license plate with a vehicle and a related device, the method comprising: detecting an image to be detected to obtain at least one vehicle region and at least one license plate region; determining the license plate associated region of each vehicle region based on preset key points in each vehicle region; and determining the license plate region associated with each vehicle region using the positional relationship between the license plate associated region and the license plate region. The solution provided by the present application can improve the accuracy of associating a license plate with a vehicle, thereby better ensuring the stability of the output result of associating a license plate with a vehicle.

Description

Technical Field

[0001] The present application relates to the field of intelligent image processing technology, and in particular to a method for associating a license plate with a vehicle and a related device. Background Art

[0002] With the development of artificial intelligence technology, the accuracy of vehicle and license plate detection and recognition in traffic scenes has been continuously improved. Algorithm-based vehicle and license plate detection and recognition has been widely used in various urban roads, parking lots, toll stations and other scenes. Due to the limitations of scenes and camera installation, there are difficulties such as large angles, low illumination, and overlapping occlusion of multiple vehicles, which have a great impact on the accuracy of the results of the association between license plates and vehicles, thereby reducing the stability and scene adaptability of the output results of vehicle capture and its associated license plates. Therefore, a technical solution that can solve the above technical problems is urgently needed. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a method and related device for associating license plates with vehicles, which can improve the accuracy of associating license plates with vehicles, and thus better ensure the stability of the output results of associating license plates with vehicles.

[0004] In order to solve the above technical problems, a technical solution adopted by the present application is: to provide a method for associating a license plate with a vehicle, the method comprising:

[0005] Detect the image to be detected to obtain at least one vehicle area and at least one license plate area;

[0006] Determining the license plate associated area of ​​each of the vehicle areas based on the preset key points in each of the vehicle areas;

[0007] The license plate area associated with each of the vehicle areas is determined by utilizing the positional relationship between the license plate associated area and the license plate area.

[0008] In order to solve the above technical problems, another technical solution adopted by the present application is: to provide an electronic device, the electronic device comprising a processor and a memory coupled to the processor; wherein,

[0009] The memory is used to store computer programs;

[0010] The processor is configured to run the computer program to perform any one of the methods described above.

[0011] To solve the above technical problems, another technical solution adopted in the present application is: providing a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program that can be executed by a processor, and the computer program is used to implement any of the methods described above.

[0012] The beneficial effects of the present application are as follows: Different from the prior art, the technical solution provided by the present application detects the image to be detected to obtain at least one vehicle area and at least one license plate area, and then determines the license plate associated area in each vehicle area based on the preset key points in each vehicle area, and determines the license plate area associated with each vehicle area by using the positional relationship between the license plate associated area and the license plate area. The technical solution provided by the present application improves the accuracy of the association between the license plate area and the vehicle area by determining a license plate associated area smaller than the vehicle area based on the preset key points in the detected license plate area, and then determines the license plate area associated with each vehicle area by using the positional relationship between the license plate associated area in the vehicle area and the license plate area, thereby better ensuring the stability of the output result of the license plate and vehicle association, and achieving a good technical effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a flow chart of an embodiment of a method for associating a license plate with a vehicle in the present application;

[0014] Figure 2 This is a schematic diagram of a process in an embodiment of a method for associating a license plate with a vehicle in the present application;

[0015] Figure 3 A schematic diagram of a process in another embodiment of a method for associating a license plate with a vehicle in the present application;

[0016] Figure 4 This is a schematic diagram of a process in another embodiment of a method for associating a license plate with a vehicle in the present application;

[0017] Figure 5 A schematic diagram of a flow chart of another embodiment of a method for associating a license plate with a vehicle in the present application;

[0018] Figure 6 This is a flow chart of another embodiment of a method for associating a license plate with a vehicle in the present application;

[0019] Figure 7 This is a flow chart of another embodiment of a method for associating a license plate with a vehicle in the present application;

[0020] Figure 8 This is a structural diagram of an electronic device in an embodiment of the present application;

[0021] Fig. 9 This is a schematic diagram of the structure of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0023] In the description of the present application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically limited. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0024] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0025] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of a method for associating a license plate with a vehicle in the present application. In the current embodiment, the method provided by the present application includes S110 to S130.

[0026] S110: Detect the image to be detected to obtain at least one vehicle area and at least one license plate area.

[0027] When associating a license plate with a vehicle, it is first necessary to detect the image to be detected, and then obtain at least one vehicle area and at least one license plate area from the image to be detected. Among them, the image to be detected is an image waiting for the license plate and vehicle association process, and the image to be detected can be obtained in real time by the electronic device that performs the license plate and vehicle association, or it can be obtained by an external image acquisition device and transmitted to the electronic device currently used to perform license plate and vehicle management. The vehicle area is the detection area corresponding to the vehicle, which can be specifically reflected as the area where the vehicle detection frame is located, and the license plate area is the detection area corresponding to the license plate, which can be specifically reflected as the area where the license plate detection frame is located.

[0028] After the image to be detected is acquired, the image to be detected will be detected, and all vehicle regions detected in the image to be detected will be output, and all license plate regions detected in the image to be detected will also be output, that is, multiple vehicle regions can be detected in the same image to be detected, and multiple license plate regions can also be detected in the same image to be detected. It should be noted that the number of detected vehicle regions and the number of license plate regions may not be equal, such as multiple vehicle regions and one license plate region can be detected in the same image to be detected, or one vehicle region and multiple license plate regions can be detected in the same image to be detected.

[0029] In one embodiment, it should be noted that, since the solution provided by the present application is to associate the license plate area with the vehicle area, if after the image to be detected is detected, only one vehicle area is detected but no license plate area is detected, or only one license plate area is detected but no vehicle area is detected, then the current cycle can be directly terminated, that is, the following steps S120 and S130 will not be executed. It is understandable that in another embodiment, it can also be set that after the image to be detected is detected, only one vehicle area is obtained but no license plate area is detected, or only one license plate area is detected but no vehicle area is detected, after the current cycle is directly terminated, step S110 will be further re-circulated to repeatedly detect the current image to be detected until at least one vehicle area and at least one license plate area are detected, or the cycle is terminated after the preset number of times, or the next image to be detected is directly detected, which is specifically set according to actual needs. Among them, the preset number of times is pre-set according to the empirical value, and the preset number of times is not limited here.

[0030] Further, step S110 includes: using a deep learning detection model to detect the image to be detected to obtain at least one vehicle area and at least one license plate area. The deep learning detection model is a pre-set algorithm model based on a deep learning algorithm and used to detect vehicle areas and license plate areas.

[0031] S120: Determine the license plate associated area of ​​each vehicle area based on the preset key points in each vehicle area.

[0032] After detecting and obtaining at least one vehicle region and at least one license plate region, the license plate associated regions in each vehicle region are further determined based on preset key points in each vehicle region. The preset key points are points corresponding to the vehicle structure that are pre-set based on the vehicle appearance structure for obtaining the license plate associated region. The preset key points can also be understood as points corresponding to the vehicle structure that are used to further narrow down the vehicle region to obtain the approximate range of the license plate. The preset key points can be determined based on the layout of the license plate relative to the overall appearance of the vehicle.

[0033] The license plate associated area is the area in the vehicle area that is associated with the license plate. The area of ​​the license plate associated area is smaller than the vehicle area, that is, the range covered by the license plate associated area is smaller than the vehicle area. There is no limitation on the size relationship between the license plate associated area and the license plate area.

[0034] Furthermore, step S120 determines the license plate associated area of ​​each vehicle area based on the preset key points in each vehicle area, including: for each vehicle area, using the preset key points, determining the size information and preset position of the license plate associated area corresponding to each vehicle area; and determining the license plate associated area of ​​each vehicle area based on the size information and preset position of the license plate associated area.

[0035] The size information of the license plate associated area corresponding to the vehicle area includes the width and height of the license plate associated area, one of the height and the width of the license plate associated area is the size of the license plate associated area in the direction of the first coordinate axis, and the other is the size of the license plate associated area in the direction of the second coordinate axis. The preset position is the vertex of the license plate associated area determined based on the border of the vehicle area, the vertex of the vehicle area and the preset key point, and may specifically include the first vertex and the fourth vertex described below.

[0036] Furthermore, the license plate associated area is a quadrilateral of a preset shape, such as the license plate associated area may be a rectangle.

[0037] Furthermore, for the preset key points, according to the number of structural points and the layout classification on the vehicle, the preset key points include the first vehicle key points and the second vehicle key points. Among them, the first vehicle key points are points corresponding to the structures arranged in pairs on the vehicle and arranged symmetrically with the reference plane where the vehicle centerline is located, and the second vehicle key points are points corresponding to the structures arranged on the reference plane where the vehicle centerline is located or at a distance less than the preset distance from the reference plane where the vehicle centerline is located. The number of the same type of second-class key points in the same vehicle area is one. The vehicle centerline is the line between the midpoint of the front boundary of the vehicle chassis and the midpoint of the rear boundary of the vehicle chassis, and the reference plane where the vehicle centerline is located is the plane passing through the vehicle centerline and perpendicular to the horizontal plane.

[0038] Furthermore, the first type of key points may include, but are not limited to, any one of: headlights, taillights, fog lights, reflectors, chassis front boundary endpoints, chassis rear boundary endpoints, radiator grille endpoints, windows, and tires. The second type of key points includes: vehicle logos.

[0039] For further information, see Figures 2 to 4 , the following will explain the process of determining each vertex of the license plate associated area according to the number and type of preset key points determined in the vehicle area.

[0040] Please combine Figure 2 , Figure 2 This is a schematic diagram of the process of an embodiment of a method for associating a license plate with a vehicle in the present application.

[0041] In one embodiment, if the preset key points are two vehicle key points of the same type, the two first vertices of the license plate associated area are on the first edge of the vehicle area, and the remaining two second vertices are determined based on the two vehicle key points. Among them, the same type of vehicle key points refer to key points corresponding to the same vehicle structure, such as headlights or taillights. The first edge of the vehicle area is the edge of the vehicle area located on the side of the preset key point close to the license plate and parallel to the first coordinate axis of the preset coordinate system. The first vertex of the license plate associated area is determined based on the two second vertices and the vertex of the vehicle area.

[0042] like Figure 2 As shown in FIG. 1 , two vehicle key points A and B of the same type are detected in a certain vehicle area. At this time, the vehicle key points A and B can be directly determined as the two second vertices in the license plate associated area in the current vehicle area. EF is the first side of the vehicle area. The first vertices C and D in the license plate associated area are determined by combining the vehicle key points A and B and the vertices of the vehicle area on the first side EF. Figure 2 In the diagram shown, area 201 is the vehicle area, and ABCD is the license plate associated area.

[0043] Furthermore, combined with Figure 2 The process of determining the two first vertices based on the two vehicle key points of the same type and the vertices of the vehicle area on the first side is as follows: determine the coordinates of the vertex in the vehicle area close to the license plate side, that is, determine the coordinates of the vertex in the vehicle area on the first side, the coordinates of the determined vertex include the first coordinate and the second coordinate, and combine the coordinates of the two vehicle key points output as the second vertices to determine the coordinates of the two first vertices.

[0044] Specifically Figure 2Taking the example, the second coordinate y2 of the vertex on the first side in the vehicle area, the coordinates of the second vertices A and B (the coordinate of A is marked as (x1, y1), and the coordinate of B is marked as (x2, y1)) are determined respectively, and then the second coordinate y2 of the vertex on the first side in the vehicle area is output as the second coordinate of the first vertices C and D. At the same time, the first coordinates x1 and x2 of the second vertices A and B are combined respectively and output as the first coordinates of the two first vertices C and D respectively. The coordinates of the first vertices C and D are (x1, y2) and (x2, y2) respectively.

[0045] Furthermore, if the preset key points are two first vehicle key points of the same type, in order to obtain a more accurate license plate association area, the second coordinates of the two second vertices on the second coordinate axis are determined to be between the second coordinates of the two first vehicle key points. Specifically, the first coordinates of the two second vertices on the first coordinate axis are respectively the first coordinates of the two first vehicle key points, and the second coordinates of the two second vertices on the second coordinate axis are both located between the second coordinates of the two first vehicle key points. For example, the mean of the second coordinates of the two first vehicle key points can be output as the second coordinate of the second vertex.

[0046] Please combine Figure 3 , Figure 3 This is a schematic diagram of the process in another embodiment of a method for associating a license plate with a vehicle in the present application. In another embodiment, if the preset key point is a vehicle key point, a third vertex in the license plate association area is determined based on the vehicle key point, and the remaining three fourth vertices are on the first side and the second side of the vehicle area. Among them, the first side is the side of the vehicle area located on the side of the preset key point close to the license plate and parallel to the first coordinate axis of the preset coordinate system, and the second side is the side of the vehicle area located on the side of the preset key point close to the license plate and parallel to the second coordinate axis of the preset coordinate system.

[0047] If the preset key point is a first vehicle key point, the third vertex is the first vehicle key point, that is, the detected first vehicle key point is directly determined as the third vertex. Figure 3 For example, if only one vehicle key point G is detected, the vehicle key point G will be used as the third vertex in the license plate associated area, and then the remaining three fourth vertices are determined by combining the vehicle key point G, the first side JK of the vehicle area and the second side JL of the vehicle area.

[0048] Specifically, the coordinates (x3, y3) of the key point G of the vehicle are determined, and the coordinates (x4, y4) of the vertex J at the intersection of the first side JK and the second side JL of the vehicle area are determined, and the vertex J is determined as a fourth vertex, and then the coordinates of the remaining fourth vertices H and I are further determined based on the coordinates (x3, y3) of G and the coordinates (x4, y4) of J, the coordinates of H are (x3, y4), and the coordinates of I are (x4, y3), Figure 3 The area GHJI in is the determined license plate associated area. It should be noted that Figure 3 The area 301 in FIG. 1 is the vehicle area.

[0049] Please combine Figure 4 , Figure 4 This is a process diagram of another embodiment of a method for associating a license plate with a vehicle in the present application. In another embodiment, if the preset key point is a second vehicle key point, the second coordinate of the first vertex is the second coordinate of the second vehicle key point, and the distance from the second edge is twice the distance between the second vehicle key point and the second edge. Figure 4 For example, if a second vehicle key point M (x5, y5) is detected, the remaining three vertices in the license plate associated area are determined by combining the coordinates of the second vehicle key point M and the vertex N (x6, y6) at the intersection of the first and second sides of the vehicle area. The remaining three vertices P in the license plate associated area have coordinates of (2x5-x6, y5), Q coordinates of (2x5-x6, y6), and R coordinates of (x6, y5). Figure 4 The illustrated area PQNR is the determined license plate associated area, area 401 is the vehicle area, SN is the second side of the vehicle area, and NT is the first side of the vehicle area.

[0050] In another embodiment, the coordinates and size of the center point of the license plate associated area can be directly determined based on the preset key point, and then the positional relationship between the license plate associated area and the license plate area can be determined based on the center point and size of the license plate associated area, and then step S130 can be performed using the positional relationship between the license plate associated area and the license plate area. First of all, it should be noted that the first side mentioned below is the side of the vehicle area located at the preset key point close to the license plate and parallel to the first coordinate axis of the preset coordinate system, and the second side is the side of the vehicle area located at the preset key point close to the license plate and parallel to the second coordinate axis of the preset coordinate system. Furthermore, the first coordinate axis includes the x-axis, and the second coordinate axis includes the y-axis.

[0051] When the preset key points are two first vehicle key points of the same type, the process of obtaining the size information of the license plate associated area is as follows: the absolute value of the difference between the first coordinates of the two first vehicle key points is output as the width of the license plate associated area. For example, if the first coordinates of the two first vehicle key points are x7 and x8, respectively, then the width of the license plate associated area b = |x7-x8|. The second coordinate of the vertex located on the side of the first vehicle key point close to the license plate in the vehicle area is subtracted from half of the sum of the second coordinates of the two first vehicle key points, and the absolute value of the difference obtained by the subtraction is output as the height of the license plate associated area. For example, if the second coordinate of the vertex located at the intersection of the first side and the second side in the vehicle area is y7, and the second coordinates of the two first vehicle key points are y8 and y9, respectively, then the height of the license plate associated area h = |y7-1 / 2*(y8+y9)|. The process of determining the coordinates of the center point of the license plate associated area based on the first vehicle key point is as follows: half of the sum of the first coordinates of the two first vehicle key points is output as the first coordinate of the center point of the license plate associated area. For example: Continuing with the above example, the first coordinate of the center point of the license plate associated area is 1 / 2*(x7+x8). The second coordinate of the vertex at the intersection of the first edge and the second edge in the vehicle area is added to half of the sum of the second coordinates of the two first vehicle key points, and half of the result of the addition is output as the second coordinate of the center point of the license plate associated area. For example: Continuing with the above example, the second coordinate of the center point of the license plate associated area is 1 / 2*(y7+1 / 2*(y8+y9)).

[0052] When the preset key point is a first vehicle key point, the process of obtaining the size information of the license plate associated area is as follows: subtract the first coordinate of the currently detected first vehicle key point from the first coordinate of the vertex on the second side in the vehicle area, and output the absolute value of the difference obtained by the subtraction as the width of the license plate associated area. For example, if the first coordinate of the first vehicle key point is x10, and the first coordinate of the vertex on the second side in the vehicle area is x11, then the width of the license plate associated area is b = |x10-x11|. Subtract the second coordinate of the first vehicle key point from the second coordinate of the vertex on the first side in the vehicle area, and output the absolute value of the difference obtained by the subtraction as the height of the license plate associated area. For example, if the second coordinate of the first vehicle key point is y10, and the second coordinate of the vertex at the intersection of the first side and the second side in the vehicle area is y11, then the height of the license plate associated area is h = |y10-y11|. The process of determining the coordinates of the center point of the license plate associated area based on a first vehicle key point is as follows: add the first coordinate of the first vehicle key point and the first coordinate of the vertex at the intersection of the first side and the second side in the vehicle area, and output half of the sum as the first coordinate of the center point of the license plate associated area. For example: Continuing with the above example, the first coordinate of the center point of the license plate associated area is 1 / 2*(x10+x11). Add the second coordinate of the first vehicle key point and the second coordinate of the vertex at the intersection of the first side and the second side in the vehicle area, and output half of the sum as the second coordinate of the center point of the license plate associated area. For example: Continuing with the above example, the second coordinate of the center point of the license plate associated area is 1 / 2*(y10+y11).

[0053] When the preset key point is a second vehicle key point, the process of obtaining the size information of the license plate associated area is as follows: subtract the first coordinate of the second vehicle key point from the first coordinate of the vertex at the intersection of the first side and the second side of the vehicle area, and output twice the absolute value of the difference obtained by subtraction as the width of the license plate associated area. For example, the coordinates of the second vehicle key point are (x12, y12) and the coordinates of the vertex at the intersection of the first side and the second side of the vehicle area are (x13, y13), then the width of the license plate associated area b = 2*|x12-x13|. Subtract the second coordinate of the vertex at the intersection of the first side and the second side of the vehicle area from the second coordinate of the second vehicle key point, and output the difference obtained by subtraction as the height of the license plate associated area. For example: the height of the license plate associated area h = |y12-y13|. The process of determining the coordinates of the center point of the license plate associated area based on a second vehicle key point is as follows: the first coordinate of the second vehicle key point is output as the horizontal coordinate of the center point of the license plate associated area, the second coordinate of the vertex at the intersection of the first side and the second side in the vehicle area is added to the second coordinate of the second vehicle key point, and half of the sum is output as the second coordinate of the center point of the license plate associated area. For example, the coordinates of the center point of the license plate associated area are (x12, 1 / 2* (y12+y13)).

[0054] Further, in another embodiment, before determining the license plate associated area of ​​each vehicle area based on the preset key points in each vehicle area in step S120, the method provided by the present application further includes: performing key point detection on each vehicle area to obtain the preset key points of each vehicle area. Among them, the key point detection algorithm can be used to perform key point detection on each vehicle area, and the key point detection algorithm selected is not limited here, and can be set according to actual needs.

[0055] Among them, for the above Figures 2 to 4 In the corresponding embodiments, the first vertex, the second vertex, the third vertex and the fourth vertex are vertices in the quadrilateral of the preset shape corresponding to the license plate associated area, that is, in one embodiment, the vertices in the quadrilateral of the preset shape corresponding to the license plate associated area may be the first vertex and the second vertex; in another embodiment, the vertices in the quadrilateral of the preset shape corresponding to the license plate associated area may be the third vertex and the fourth vertex. In other words, Figure 2 In the corresponding embodiment, the quadrilateral of the preset shape corresponding to the license plate associated area is a rectangle, and the rectangular frame determined based on the first vertex and the second vertex is the detection frame corresponding to the license plate associated area. Figure 3 In the corresponding embodiment, the quadrilateral of the preset shape corresponding to the license plate associated area is a rectangle, and the rectangular frame determined based on the third vertex and the fourth vertex is the detection frame corresponding to the license plate associated area.

[0056] S130: Determine the license plate area associated with each vehicle area by using the positional relationship between the license plate associated area and the license plate area.

[0057] After the license plate associated area in the vehicle area is determined, the positional relationship between each license plate associated area and each license plate area is further determined for each license plate associated area in each vehicle area, and then the license plate area associated with each vehicle area is determined using the positional relationship between the license plate associated area and each license plate area. The positional relationship between the license plate associated area and the license plate area at least includes the distance between the license plate associated area and the license plate area, and the difference in coordinates of a preset position point between the license plate associated area and the license plate area.

[0058] The distance between the license plate association area and the license plate area may be the distance between a preset position point of the license plate association area and a preset position point of the license plate area. When the coordinates include a first coordinate and a second coordinate, the coordinate difference between the preset position point between the corresponding license plate association area and the license plate area includes the difference between the first coordinate and the difference between the second coordinate.

[0059] Further, the preset position point includes any one of a center point and a region vertex, wherein the region vertex includes any one of an upper left vertex, an upper right vertex, a lower left vertex, and a lower right vertex.

[0060] It is understandable that in other embodiments, the preset position point may also be set to include other types of points, such as being set to the midpoint of a border line in the region. For example, the preset position point may be set to the midpoint of the right border line in the license plate region and the midpoint of the right border line in the license plate associated region, and so on. The preset position point may also be set to any one of the midpoint of the left border line in the license plate region and the midpoint of the left border line in the license plate associated region, the midpoint of the upper border line in the license plate region and the midpoint of the upper border line in the license plate associated region, and the midpoint of the lower border line in the license plate region and the midpoint of the lower border line in the license plate associated region.

[0061] Furthermore, in one embodiment, when the positional relationship between the license plate association area and the license plate area is the distance between the license plate association area and the license plate area, after determining the license plate association area in each vehicle area, the distance between each license plate association area and each license plate area is determined respectively, and then based on the distance between each license plate association area and each license plate area, the vehicle area associated with each license plate area is determined respectively. Furthermore, the basis for determining whether the license plate association area and the license plate area are associated and matched may be that the distance between the license plate association area and the license plate area is less than or equal to a first preset value. The first preset value is a pre-set empirical value, which can be set and adjusted according to actual needs, and is not specifically limited to uniqueness.

[0062] Furthermore, in one embodiment, when the coordinate difference between the license plate associated area and the preset position point of the license plate area is used to measure the positional relationship between the two, the basis for determining whether the license plate associated area and the license plate area are associated and matched is that the first coordinate difference of the preset position point on the first coordinate axis is less than or equal to the second preset value, and the second coordinate difference of the preset position point on the second coordinate axis is less than or equal to the third preset value. For the specific process of association matching, please refer to the following Figures 5 to 7 The corresponding embodiment.

[0063] Figure 1 The technical solution provided in the corresponding embodiment detects the image to be detected to obtain at least one vehicle area and at least one license plate area, and then determines the license plate associated area in each vehicle area based on the preset key points in each vehicle area, and determines the license plate area associated with each vehicle area by using the positional relationship between the license plate associated area and the license plate area. The technical solution provided by the present application utilizes the characteristics that the relative positional relationship between the license plate and the vehicle structure corresponding to the preset key points is fixed and not affected by the vehicle posture. By determining a license plate associated area smaller than the vehicle area based on the preset key points in the detected license plate area, it avoids the same license plate appearing in two different license plate associated areas, ensuring the uniqueness of the license plate associated vehicle, and then using the positional relationship between the license plate associated area in the vehicle area and the license plate area, respectively determine the license plate area associated with each vehicle area, thereby improving the accuracy of the license plate area and vehicle area association, thereby better ensuring the stability of the license plate and vehicle association output results, and also improving the scene adaptability of the license plate and vehicle association, which can effectively solve the difficult problem of license plate association.

[0064] See also Figure 5 , Figure 5 This is a flowchart of another embodiment of a method for associating a license plate with a vehicle in the present application. In the current embodiment, the above step S130 uses the positional relationship between the license plate association area and the license plate area to determine the license plate area associated with each vehicle area, and further includes steps S501 to S503.

[0065] S501: For each license plate area, respectively determine the distance relationship between the license plate area and each license plate associated area about a preset position point.

[0066] After the license plate associated area of ​​each vehicle area is determined based on the preset key points in each vehicle area, the distance relationship between the license plate area and each license plate associated area about the preset position point will be determined for each license plate area. The preset position point is a pre-set point for determining the positional relationship between the license plate area and the license plate associated area, and the preset position point includes: any one of the center point of the area and the vertex of the area. The area vertex is classified according to the direction in the area and includes the upper left vertex, the lower left vertex, the upper right vertex and the lower right vertex. When determining the distance relationship between the license plate area and each license plate associated area about the preset position point, the positional relationship between the two can be determined based on only one preset position point, such as the distance relationship between the two can be determined based on the center point of the license plate area and the center point of the license plate associated area, or the distance relationship between the two can be determined based on the upper left vertex of the license plate area and the upper left vertex of the license plate associated area, and the specific setting is subject to the actual setting. It should be noted that when executing step S501, the preset position points selected in the license plate area and the license plate associated area are points of the same type. For example, when the preset position point selected in the license plate area is the center point of the license plate area, the preset position point of the license plate associated area corresponds to the center point of the license plate associated area. When other types of points are selected as the preset position, the same applies.

[0067] If two vehicle areas I and II and three license plate areas ① to ③ are obtained in the image to be detected, the corresponding two license plate associated areas I' and II' will be determined based on the preset key points in each vehicle area. Then in step S501, for the three license plate areas ① to ③, the distance relationship between each license plate area and the two license plate associated areas I' and II' with respect to the preset position points is determined respectively, that is, the distance relationship between the preset position point of license plate area ① and the preset position point of the two license plate associated areas I' and II' is determined respectively, and the distance relationship between the preset position point of license plate area ② and the preset position point of the two license plate associated areas I' and II' is determined at the same time, and the distance relationship between the preset position point of license plate area ③ and the preset position point of the two license plate associated areas I' and II' is determined.

[0068] Among them, when the selected distance relationship is the distance between the preset position points, step S501 is to determine the distance between the license plate area and each license plate associated area about the preset position points for each license plate area, that is, to calculate the distance between the preset position points of each license plate area and the preset position points of each license plate associated area.

[0069] If the selected distance relationship is the coordinate difference of the preset points, step S501 is to calculate the coordinate difference between the preset position point of the license plate area and the preset position point of each license plate associated area for each license plate area, specifically, calculating the first coordinate difference and the second coordinate difference between the coordinates of the preset position point of the license plate area and the preset position point of each license plate associated area.

[0070] It is understandable that, in other embodiments, the distance relationship between the license plate area and each license plate associated area about the preset position point may also include other forms, which are not listed here in detail.

[0071] S502: Determine a license plate associated region that matches the license plate region based on a distance relationship between the license plate region and each license plate associated region.

[0072] After respectively determining the distance relationship between the license plate area and each license plate associated area about the preset position point, further based on the distance relationship between the license plate area and each license plate associated area, determine the license plate associated area that matches the license plate area.

[0073] Further, in step S502, the license plate associated region that matches the license plate region may be determined based on whether the distance relationship between the license plate region and the preset position point of each license plate associated region meets the first preset condition. In the current embodiment, when the distance relationship between the license plate region and the license plate associated region meets the first preset condition, it is determined that the current license plate associated region matches the license plate region, and the current license plate associated region is output as the license plate associated region that matches the license plate region.

[0074] Furthermore, when the distance relationship selected in step S501 is the distance of the preset position point, the first preset condition is that the distance between the license plate area and the license plate associated area about the preset position point is less than or equal to the first preset value, or the distance between the license plate area and the license plate associated area about the preset position point is the minimum. If the distance relationship selected in step S501 is the coordinate difference of the preset position point, the first preset condition is that the first coordinate difference between the preset position point of the license plate area and the preset position point of the license plate associated area on the first coordinate axis is less than or equal to the second preset value, and the second coordinate difference on the second coordinate axis is less than or equal to the third preset value. Among them, the first preset value, the second preset value and the third preset value are pre-set empirical values ​​for judging the distance relationship, which can be set according to actual needs and are not limited here.

[0075] S503: Associating the license plate area with the vehicle area corresponding to the matching license plate associated area.

[0076] After determining the license plate associated region that matches the license plate region based on the distance relationship between the license plate region and each license plate associated region, the license plate region is further associated with the vehicle region corresponding to the license plate associated region that matches it.

[0077] Furthermore, after the association between the license plate area and the vehicle area in the current image to be detected is completed, the association matrix between each license plate area and each vehicle area is output. The association matrix includes the association representation value between each license plate area and each vehicle area, and the association representation value is a first symbol indicating association or a second symbol indicating non-association.

[0078] Furthermore, one of the first symbol and the second symbol can be set to 1, and the other can be set to 0. It can be understood that in other embodiments, the first symbol and the second symbol can also be set to other forms, which are not uniquely limited and are not listed one by one here.

[0079] For example, a license plate area set PD and a vehicle area set VD are detected in a certain image to be detected. The license plate area set PD includes at least one license plate area, and the vehicle area set VD includes at least one vehicle area. When associating the license plate area and the vehicle area, the formula Perform association processing and traverse all license plates in the PD set according to the above formula. Wherein, aij represents the association result between the i-th vehicle area and the j-th license plate area. After traversing all license plates, the association matrix between each license plate area and the vehicle area is output respectively. Wherein, the association matrix includes the association representation value between the license plate area and the vehicle area. The association representation value can be set to 1 to indicate that the license plate area is associated with the vehicle area, and the association representation value can be set to 0 to indicate that the license plate area is not associated with the vehicle area. The xc in the formula j and xc i Refers to the first coordinate of the preset position point in the license plate associated area, yc j and yc i Refers to the second coordinate of the preset position point in the license plate association area, AP wi is the width of the license plate association area, AP Hi is the height of the license plate associated area. In the current embodiment, the second preset value is half of the width of the license plate associated area, and the third preset value is half of the height of the license plate associated area. That is, in the current embodiment, whether the license plate association area and the license plate area match is determined by determining whether the first coordinate difference between the preset position point of the license plate association area and the preset position point of the license plate area is less than half of the width of the license plate associated area, and determining whether the second coordinate difference between the preset position point of the license plate association area and the preset position point of the license plate area is less than half of the height of the license plate associated area.

[0080] For further information, see Figure 6 , Figure 6This is a flowchart of another embodiment of a method for associating a license plate with a vehicle in the present application. In the current embodiment, the above step S130 uses the positional relationship between the license plate association area and the license plate area to determine the license plate area associated with each vehicle area, including: S601 to step S604.

[0081] S601: For each license plate area, respectively determine the distance relationship between the license plate area and each license plate associated area about a preset position point.

[0082] Step S601 in the current embodiment is the same as the above step S501, and please refer to the description of the corresponding part above for details.

[0083] In the current embodiment, step S502 further includes step S602.

[0084] S602: Determine a license plate associated region whose distance relationship with the license plate region meets a first preset condition as a license plate associated region that matches the license plate region.

[0085] After respectively determining the distance relationship between the license plate area and each license plate associated area about the preset position point, the license plate associated area whose distance relationship with the license plate area meets the first preset condition is further determined as the license plate associated area matching the license plate area.

[0086] Specifically, it is determined whether the distance relationship between the license plate area and each license plate associated area meets the first preset condition, and the license plate associated area whose distance relationship with the license plate area meets the first preset condition is determined as the license plate associated area that matches the license plate area. The first preset condition includes at least one of the following: the distance between the license plate area and the license plate associated area about the preset position point is less than the first preset value, the distance between the license plate area and the license plate associated area about the preset position point is the smallest, or the first coordinate difference between the license plate area and the license plate associated area about the preset position point is less than the second preset value and the second coordinate difference is less than the third preset value. The first coordinate difference is the coordinate difference between the preset position point of the license plate area and the preset position point of the license plate associated area on the first coordinate axis, and the second coordinate difference is the coordinate difference between the preset position point of the license plate area and the preset position point of the license plate associated area on the second coordinate axis.

[0087] In one embodiment, when the distance relationship is the distance between the preset position points, the first preset condition is that the distance between the license plate area and the license plate associated area about the preset position point is less than a first preset value. The first preset value is determined based on experience and can be set and adjusted according to actual needs, and is not specifically limited to uniqueness.

[0088] In another embodiment, when the distance relationship is the distance of a preset position point, the first preset condition is that the distance between the license plate area and the license plate associated area about the preset position point is the smallest.

[0089] In another embodiment, when the distance relationship is the coordinate difference of the preset position point, the first preset condition is that the first coordinate difference between the license plate area and the license plate associated area about the preset position point is less than the second preset value and the second coordinate difference is less than the third preset value. Specifically, in the current embodiment, when the distance relationship meets the first preset condition, the first coordinate difference between the preset position point of the license plate area and the preset position point of the license plate associated area on the first coordinate axis is less than the second preset value, and the second coordinate difference on the second coordinate axis is less than the third preset value.

[0090] Furthermore, the second preset value is a first ratio of the width of the corresponding license plate association area, and the third preset value is a second ratio of the height of the corresponding license plate association area. For example, the second preset value can be set to half of the width of the license plate association area, and the third preset value can be set to one-third of the height of the corresponding license plate association area. The width of the license plate association area and the height of the license plate association area can be obtained based on the vertex coordinates of the license plate association area when determining the license plate association area, or can be obtained based on the center point of the license plate association area and the vertex of the license plate association area.

[0091] S603: Associating the license plate area with the vehicle area corresponding to the matching license plate associated area.

[0092] Step S603 in the current embodiment is the same as step S503 described above, and can be specifically described in conjunction with the corresponding parts of the above text.

[0093] In yet another embodiment, see Figure 7 , Figure 7 This is a flow chart of another embodiment of a method for associating a license plate with a vehicle in this application. Figure 6 In the embodiment, it is determined that there are multiple license plate associated areas that meet the first preset condition. To further ensure the accuracy of the matching between the license plate area and the vehicle area, the above step S602 determines the license plate associated area whose distance relationship with the license plate area meets the first preset condition as the license plate associated area that matches the license plate area, and also includes steps S701 to S702.

[0094] S701: Calculate the intersection-over-union ratio of each license plate associated region and the license plate region that meets the first preset condition.

[0095] For multiple license plate associated areas that meet the first preset condition, the intersection-and-union ratio of each license plate associated area that meets the first preset condition and the current license plate area is calculated respectively, so as to further determine the license plate associated area associated and matched with the current license plate area from the dimension of the intersection-and-union ratio.

[0096] S702: Determine the license plate associated region whose intersection-over-union ratio meets the second preset condition as the license plate associated region that matches the license plate region.

[0097] After respectively calculating the intersection-and-union ratio of each license plate associated area that meets the first preset condition and the current license plate area, it is further determined whether the intersection-and-union ratio of each license plate associated area that meets the first preset condition and the license plate area meets the second preset condition, and the license plate associated area whose intersection-and-union ratio meets the second preset condition is determined as the license plate associated area that matches the license plate area.

[0098] The second preset condition includes: the intersection-and-joint ratio is greater than or equal to a fourth preset value and / or the intersection-and-joint ratio is the highest.

[0099] In one embodiment, the second preset condition includes: the intersection-and-union ratio is greater than or equal to a fourth preset threshold value. Therefore, after calculating the intersection-and-union ratios of each license plate associated region that meets the first preset condition and the license plate region respectively in step S701, it is further determined whether the intersection-and-union ratios of the license plate associated region that meets the first preset condition and the license plate region are greater than or equal to a fourth preset value, and if so, it is determined that the license plate associated region is associated with the license plate region and matches.

[0100] Further, in another embodiment, the second preset condition includes: the intersection-and-union ratio is the highest. Correspondingly, after step S701, the license plate associated region with the highest intersection-and-union ratio with the current license plate region is further determined among all the license plate associated regions that meet the first preset condition, and the license plate associated region with the highest intersection-and-union ratio is associated and matched with the current license plate region.

[0101] Furthermore, in another embodiment, the second preset condition includes: the intersection-and-union ratio is greater than or equal to a fourth preset value, and the intersection-and-union ratio is the highest. Correspondingly, after step S701, firstly, a license plate association area whose intersection-and-union ratio is greater than or equal to the fourth preset value is selected from all license plate association areas that meet the first preset condition, and then a license plate association area with the highest intersection-and-union ratio is determined from all license plate association areas whose intersection-and-union ratio is greater than or equal to the fourth preset threshold, and the license plate association area whose intersection-and-union ratio is greater than or equal to the fourth preset threshold and the license plate association area with the highest intersection-and-union ratio is associated and matched with the current license plate area.

[0102] In the current embodiment, when it is determined that there are multiple license plate association areas that meet the first preset condition, the intersection-and-union ratio is further used to further determine a license plate association area whose intersection-and-union ratio meets the second preset condition from the multiple license plate association areas that meet the first preset condition, thereby combining the distance relationship between the license plate association area and the license plate area and the intersection-and-union ratio between the license plate association area and the license plate area, so as to more accurately associate and match the license plate area with the license plate association area, thereby improving the accuracy and stability of the association between the license plate area and the vehicle area.

[0103] Combination Figures 1 to 7Compared with the existing methods, when associating a license plate with a vehicle, the method provided by the present application first narrows the comparison target from a larger vehicle area including the entire vehicle to a smaller license plate association area. Compared with the original method of using only the vehicle area to determine the association between the license plate and the vehicle, it can effectively solve the problem of associating multiple vehicle areas with the same license plate when the vehicles overlap, thereby ensuring the stability and accuracy of the license plate and vehicle association output.

[0104] The technical solution provided by the present application can also solve the problem of the association between the front and rear vehicles. For example, taking the front and rear vehicles of a large truck at a highway toll station as an example, if the vehicle inspection frame corresponding to the vehicle area is used as the basis for the association and matching of the license plate and the vehicle, since there are vehicles in front and behind the large truck, that is, the vehicle area overlaps at this time, and a license plate area is simultaneously included by the vehicle inspection frames of multiple vehicle areas in front and behind, it is difficult to determine the association between the license plate and the vehicle based on the vehicle area. With this solution, the license plate association area determined by the preset key points in the vehicle area cannot be detected in the front and rear overlapping vehicle areas, so the license plate association area cannot be detected in the front and rear vehicle areas, and the vehicle area associated with the current license plate can be determined more accurately. The method provided by the present application is also applicable to the license plate association problem of multiple vehicles overlapping in complex scenes. Taking the preset key point as the headlight as an example, the license plate is set in the license plate association area based on the headlight calculation, and the license plate association areas of different vehicle areas have no intersection, so there will be no result of one license plate area being associated with two vehicle areas.

[0105] See also Figure 8 , Figure 8 800 is a schematic diagram of a structure of an electronic device in an embodiment of the present application. In the current embodiment, the electronic device 800 provided by the present application includes a processor 801 and a memory 802 coupled to the processor 801. The electronic device 800 can execute Figures 1 to 7 And the method described in any corresponding embodiment.

[0106] The memory 802 includes a local storage (not shown) and is used to store a computer program. When the computer program is executed, Figures 1 to 7 And the method described in any corresponding embodiment.

[0107] The processor 801 is coupled to the memory 802, and the processor 801 is used to run the computer program to perform the following steps: Figures 1 to 7 Further, in some embodiments, the electronic device may include any one of a mobile terminal, a vehicle-mounted terminal, a camera, a computer terminal, a computer, an image acquisition device with computing and storage capabilities, a server, etc., and may also include any other device with computing and processing capabilities.

[0108] Furthermore, in another embodiment, the electronic device 800 provided in the present application may also include an image acquisition unit (not shown), which is connected to the processor 801 and is used to acquire images to be detected under the control of the processor 801, and feed the acquired images to be detected back to the processor 801, so as to associate the license plate with the vehicle in the acquired images to be detected.

[0109] See also Fig. 9 , Fig. 9 Schematic diagram of a computer-readable storage medium according to an embodiment of the present invention. The computer-readable storage medium 900 stores a computer program 901 that can be executed by a processor. The computer program 901 is used to implement the above Figures 1 to 7 Specifically, the computer-readable storage medium 900 may be a memory, a personal computer, a server, a network device, or a USB flash drive, and no limitation is made here.

[0110] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for associating a license plate with a vehicle, characterized in that: The method comprises: Detect the image to be detected to obtain at least one vehicle area and at least one license plate area; Based on the preset key points in each of the vehicle areas, the license plate associated area of ​​each of the vehicle areas is determined, wherein the preset key points are points corresponding to the vehicle structure that are pre-set based on the vehicle appearance structure for obtaining the license plate associated area, and are used to further narrow down the vehicle area to obtain the approximate range of the license plate; The license plate area associated with each of the vehicle areas is determined by utilizing the positional relationship between the license plate associated area and the license plate area.

2. The method according to claim 1, characterized in that The determining of the license plate associated area of ​​each vehicle area based on the preset key points in each vehicle area includes: For each of the vehicle regions, using the preset key points, determining the size information and preset position of the license plate associated region corresponding to each of the vehicle regions; and Based on the size information and the preset position of each license plate associated area, the license plate associated area of ​​each vehicle area is determined.

3. The method according to claim 1, characterized in that: The license plate associated area includes a quadrilateral of a preset shape, wherein: If the preset key points are two vehicle key points of the same type, the two first vertices of the license plate associated area are on the first side of the vehicle area, and the remaining two second vertices are determined based on the two vehicle key points respectively; If the preset key point is a vehicle key point, a third vertex in the license plate associated area is determined based on the vehicle key point, and the remaining three fourth vertices are on the first side and the second side of the vehicle area; Among them, the first edge is an edge in the vehicle area located on the side of the preset key point close to the license plate and parallel to the first coordinate axis of the preset coordinate system, the second edge is an edge in the vehicle area located on the side of the preset key point close to the license plate and parallel to the second coordinate axis of the preset coordinate system, and the first vertex, the second vertex, the third vertex and the fourth vertex are vertices in the quadrilateral of the preset shape.

4. The method according to claim 3, characterized in that If the preset key points are two first vehicle key points of the same type, the first coordinates of the two second vertices on the first coordinate axis are respectively the first coordinates of the two first vehicle key points, and the second coordinates of the two second vertices on the second coordinate axis are both located between the second coordinates of the two first vehicle key points; If the preset key point is a key point of the first vehicle, the third vertex is the key point of the first vehicle; If the preset key point is a second vehicle key point, the second coordinate of the first vertex is the second coordinate of the second vehicle key point, and the distance between the first vertex and the second edge is twice the distance between the second vehicle key point and the second edge.

5. The method according to claim 4, characterized in that The first vehicle key points include any one of the headlights, taillights, fog lights, reflectors, chassis front boundary endpoints, chassis rear boundary endpoints, radiator grille endpoints, windows and tires, and the second vehicle key points include: vehicle logos.

6. The method according to claim 2, characterized in that The determining the license plate area associated with each vehicle area by using the positional relationship between the license plate associated area and the license plate area includes: For each of the license plate areas, respectively determining the distance relationship between the license plate area and each of the license plate associated areas about a preset position point; Determining the license plate associated area that matches the license plate area based on the distance relationship between the license plate area and each of the license plate associated areas; The license plate area is associated with the vehicle area corresponding to its matching license plate associated area.

7. The method according to claim 6, characterized in that The determining the license plate associated area matching the license plate area based on the distance relationship between the license plate area and each of the license plate associated areas comprises: The license plate associated region whose distance relationship with the license plate region meets the first preset condition is determined as the license plate associated region matching the license plate region, wherein: The first preset condition includes at least one of the following: the distance between the license plate area and the license plate associated area about the preset position point is less than a first preset value, the distance between the license plate area and the license plate associated area about the preset position point is minimum, or the first coordinate difference between the license plate area and the license plate associated area about the preset position point is less than a second preset value and the second coordinate difference is less than a third preset value, wherein the first coordinate difference is the coordinate difference between the preset position point of the license plate area and the preset position point of the license plate associated area on the first coordinate axis, and the second coordinate difference is the coordinate difference between the preset position point of the license plate area and the preset position point of the license plate associated area on the second coordinate axis.

8. The method according to claim 7, characterized in that The second preset value is a first ratio of the width of the corresponding license plate associated area, and the third preset value is a second ratio of the height of the corresponding license plate associated area.

9. The method according to claim 7, characterized in that: If the license plate associated area that meets the first preset condition includes multiple license plate associated areas, determining the license plate associated area whose distance relationship with the license plate area meets the first preset condition as the license plate associated area that matches the license plate area includes: Calculating the intersection-and-union ratio of each license plate associated area meeting the first preset condition and the license plate area respectively; The license plate associated area whose intersection-over-union ratio meets a second preset condition is determined as the license plate associated area matching the license plate area, wherein the second preset condition includes: the intersection-over-union ratio is greater than or equal to a fourth preset value and / or the intersection-over-union ratio is the highest.

10. The method according to claim 1, characterized in that The detecting the image to be detected to obtain at least one vehicle area and at least one license plate area includes: Detecting the image to be detected using a deep learning detection model to obtain at least one vehicle area and at least one license plate area; And / or, before determining the license plate associated area of ​​each vehicle area based on the preset key points in each vehicle area, the method further includes: Key point detection is performed on each of the vehicle areas to obtain preset key points of each of the vehicle areas.

11. The method according to claim 1, characterized in that After determining the license plate area associated with each of the vehicle areas by using the positional relationship between the license plate associated area and the license plate area, the method further includes: Output an association matrix between each license plate region and each vehicle region, wherein the association matrix includes association representation values ​​between each license plate region and each vehicle region, respectively, and the association representation value is a first symbol indicating association or a second symbol indicating non-association.

12. An electronic device, characterized in that: The electronic device comprises a processor and a memory coupled to the processor; wherein, The memory is used to store computer programs; The processor is configured to run the computer program to perform the method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that can be executed by a processor, and the computer program is used to implement the method according to any one of claims 1 to 11.

Images