License plate positioning and recognition method and device, computer device, and storage medium

Abstract

The application relates to the technical field of computer vision, and provides a license plate positioning and identifying method and device, computer equipment and a storage medium, which comprises the following steps: obtaining a license plate pixel point area image from a license plate image, and extracting all area contours in the license plate pixel point area image; obtaining a convex hull pixel point set by convex hull fitting of each area contour, and calculating the perimeter of the corresponding area contour according to the convex hull pixel point set; performing polygon fitting on the convex hull pixel point set according to the perimeter and the fitting accuracy, and retaining the fitted quadrilateral; when the area of the quadrilateral meets a preset threshold condition, performing polygon fitting on the convex hull pixel point set by using one-half of the fitting accuracy corresponding to the quadrilateral, and obtaining a fitted polygon; determining a license plate edge line according to the vertices of the polygon, extending the license plate edge line to obtain a license plate vertex, determining a license plate area image based on the license plate vertex, and performing character recognition on each license plate area image to obtain license plate characters. The method can improve the license plate character recognition accuracy.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and in particular relates to a license plate location and recognition method, device, computer equipment, and storage medium. Background Technology

[0002] License plates are crucial vehicle identifiers, similar to identity cards, and are the primary basis for vehicle identification. Therefore, license plate recognition is widely used in daily life, such as in automated parking lot payment systems, automatic toll collection at highway entrances, traffic violation detection, and vehicle insurance reimbursement verification. Currently, license plate recognition technology is very mature and widely applied in scenarios with fixed shooting angles, such as automated parking lot payment systems. However, in unrestricted scenarios, license plate recognition still faces certain challenges, such as vehicle insurance reimbursement verification, where personnel are required to verify the recognition results.

[0003] Typically, a complete license plate recognition process includes two main steps: license plate localization and license plate character recognition. However, unlike license plate recognition in fixed scenarios, the license plate images acquired in unrestricted environments are arbitrary and uncontrolled. This results in a series of problems such as blurring, distortion, occlusion, and deformation of the license plate images, leading to inaccurate license plate localization and low accuracy in license plate character recognition. Summary of the Invention

[0004] Therefore, it is necessary to provide a license plate positioning and recognition method, device, computer equipment, and storage medium that can improve the accuracy of license plate positioning and the precision of license plate character recognition, in order to address the above-mentioned technical problems.

[0005] This invention provides a license plate location and recognition method, comprising:

[0006] Obtain the license plate pixel area image from the license plate image, and extract all the region contours in the license plate pixel area image;

[0007] Each region contour is fitted with a convex hull to obtain a set of convex hull pixels, and the perimeter of the corresponding region contour is calculated based on the set of convex hull pixels.

[0008] The fitting accuracy is determined based on the perimeter, and the fitting accuracy is used to perform polygon fitting on the convex hull pixel set, retaining the fitted quadrilateral.

[0009] When the area of ​​the quadrilateral meets the preset threshold condition, the convex hull pixel set is fitted with a polygon using half of the fitting accuracy of the quadrilateral to obtain the fitted polygon.

[0010] The license plate edge line is determined based on the vertices of the polygon, the license plate edge line is extended to obtain the license plate vertices, and the license plate region image is determined based on the license plate vertices.

[0011] Character recognition is performed on each of the license plate area images to obtain the license plate characters.

[0012] In one embodiment, obtaining a license plate pixel region image from a license plate image and extracting all region contours in the license plate pixel region image includes:

[0013] The trained license plate localization model is used to obtain the license plate pixel region image from the license plate image;

[0014] Connectivity region extraction is performed on the license plate pixel area image to obtain a set of contour pixels;

[0015] The region contour is obtained based on the set of contour pixels.

[0016] In one embodiment, the step of fitting the convex hull of each region contour to obtain a set of convex hull pixels, and calculating the perimeter of the corresponding region contour based on the set of convex hull pixels, includes:

[0017] The contours of each region are fitted using the convex hull fitting operator to obtain the set of convex hull pixels;

[0018] Calculate the distance between adjacent pixels in the convex hull pixel set sequentially;

[0019] The summation of each distance is used as the perimeter of the region's outline.

[0020] In one embodiment, the step of determining the fitting accuracy based on the perimeter, using the fitting accuracy to perform polygon fitting on the convex hull pixel set, and retaining the fitted quadrilateral includes:

[0021] The product of each preset hyperparameter coefficient and the perimeter is used as the fitting accuracy.

[0022] The pixels in the convex hull pixel set are fitted using the various fitting accuracies and polygon fitting algorithms respectively to obtain the fitted polygons corresponding to the various fitting accuracies;

[0023] Retain quadrilaterals with 4 vertices from the fitted polygon.

[0024] In one embodiment, determining the license plate edge line based on the vertices of the polygon, extending the license plate edge line to obtain the license plate vertices, and determining the license plate region image based on the license plate vertices includes:

[0025] Draw a circle with the four vertices of the quadrilateral as its center and the fitting accuracy of the quadrilateral as its radius;

[0026] Retain the vertices of the polygon that are inside any one of the circles as candidate vertices;

[0027] Calculate the distance between adjacent candidate vertices. If the distance is greater than twice the fitting accuracy of the quadrilateral, determine the line segment of the adjacent candidate vertices as the license plate edge line.

[0028] Extend the edge line of the license plate to obtain the license plate vertex, and determine the license plate region image based on the license plate vertex.

[0029] In one embodiment, determining the license plate region image based on the license plate vertices includes:

[0030] Calculate the transformation matrix based on the license plate vertices and the preset standard license plate vertices;

[0031] The license plate image is transformed using the transformation matrix to obtain a new license plate image;

[0032] Extract the license plate area image from the new license plate image.

[0033] In one embodiment, after extending the license plate edge line to obtain the license plate vertex, and before determining the license plate region image based on the license plate vertex, the method further includes:

[0034] The lower left corner of the license plate is taken as the first vertex, and the remaining license plate vertices are arranged in clockwise order as the second, third, and fourth vertices.

[0035] Calculate the distances between the second vertex and the first vertex, the third vertex, and the fourth vertex, and take the largest distance as the license plate width;

[0036] Calculate the distances between the first vertex and the second, third, and fourth vertices respectively, and take the largest distance as the license plate height;

[0037] When the license plate height is greater than the license plate width, the license plate height and the license plate width are swapped and the vertex order is adjusted.

[0038] A license plate location and recognition device, comprising:

[0039] The contour extraction module is used to obtain a license plate pixel area image from the license plate image and extract all the region contours in the license plate pixel area image.

[0040] The convex hull fitting module is used to fit the contour of each region with a convex hull to obtain a set of convex hull pixels, and to calculate the perimeter of the corresponding region contour based on the set of convex hull pixels.

[0041] The polygon fitting module is used to determine the fitting accuracy based on the perimeter, and to perform polygon fitting on the convex hull pixel set using the fitting accuracy, while retaining the fitted quadrilateral.

[0042] The polygon fitting module is also used to perform polygon fitting on the convex hull pixel set using half of the fitting accuracy of the quadrilateral when the area of ​​the quadrilateral meets the preset threshold condition, so as to obtain the fitted polygon.

[0043] The license plate extraction module is used to determine the license plate edge line based on the vertices of the polygon, extend the license plate edge line to obtain the license plate vertices, and determine the license plate region image based on the license plate vertices;

[0044] The character recognition module is used to perform character recognition on the images of each license plate area to obtain the license plate characters.

[0045] The present invention also provides a computer device, the computer device including a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the steps of the license plate location and recognition method described above.

[0046] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the license plate location and recognition method described in any of the preceding claims.

[0047] The aforementioned license plate location and recognition method, device, computer equipment, and storage medium, based on the geometric features of the license plate itself, output the license plate region contour by locating the license plate image, and then perform fitting optimization such as convex hull fitting and polygon fitting based on the license plate region contour, thereby extracting a more accurate license plate region and improving the accuracy of subsequent license plate character recognition. Attached Figure Description

[0048] Figure 1 This is an application environment diagram of a license plate location and recognition method in one embodiment.

[0049] Figure 2 This is a flowchart illustrating a license plate location and recognition method in one embodiment.

[0050] Figure 3 This is a schematic diagram of a license plate image in one embodiment.

[0051] Figure 4 This is a schematic diagram of a license plate pixel area image in one embodiment.

[0052] Figure 5 This is a schematic diagram of the region outline in one embodiment.

[0053] Figure 6This is a schematic diagram of the fitted quadrilateral in one embodiment.

[0054] Figure 7 This is a schematic diagram of the license plate vertex determination process in one embodiment.

[0055] Figure 8 This is a schematic diagram of a new license plate image in one embodiment.

[0056] Figure 9 This is a schematic diagram of a license plate area image in one embodiment.

[0057] Figure 10 This is a structural block diagram of a license plate location and recognition device in one embodiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0059] The license plate location and recognition method provided in this application can be applied to, for example... Figure 1 In the application environment shown, the application environment involves a terminal 102 and a server 104. The terminal 102 communicates with the server 104 via a network. The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices, and the server 104 can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0060] When terminal 102 receives a license plate location and recognition instruction, it can implement the license plate location and recognition method independently. Alternatively, terminal 102 can send the license plate location and recognition instruction to the communicating server 104, and server 104 can implement the license plate location and recognition method. Taking server 104 as an example, specifically, server 104 obtains the license plate pixel area image from the license plate image and extracts all the region contours in the license plate pixel area image; server 104 performs convex hull fitting on each region contour to obtain a convex hull pixel set, and calculates the perimeter of the corresponding region contour based on the convex hull pixel set; server 104 determines the fitting accuracy based on the perimeter, and uses the fitting accuracy to perform polygon fitting on the convex hull pixel set, retaining the fitted quadrilaterals; when the area of ​​the quadrilateral meets the preset threshold condition, server 104 uses half of the fitting accuracy corresponding to the quadrilateral to perform polygon fitting on the convex hull pixel set, obtaining the fitted polygon; server 104 determines the license plate edge line based on the vertices of the polygon, extends the license plate edge line to obtain the license plate vertices, and determines the license plate region image based on the license plate vertices; server 104 performs character recognition on each license plate region image to obtain the license plate characters.

[0061] In one embodiment, such as Figure 2 As shown, a license plate location and recognition method is provided. Taking the application of this method to a server as an example, the method includes the following steps:

[0062] Step S201: Obtain the license plate pixel area image from the license plate image, and extract all the region contours in the license plate pixel area image.

[0063] Among them, license plate image refers to the original image to be processed, including the license plate, such as... Figure 3 The diagram illustrates a license plate image. The license plate pixel area image is an image of the license plate located in pixels, such as... Figure 4 The image shown is a schematic diagram of a license plate pixel area image. Figure 4 The white pixel area represents the pixels corresponding to the license plate area.

[0064] Specifically, after the server obtains the license plate image I to be processed, it first locates the license plate pixel region image Ib in the license plate image. Then, it extracts all the region contours in the license plate pixel region image Ib.

[0065] In one embodiment, step S201 includes: obtaining a license plate pixel region image from a license plate image using a trained license plate localization model; extracting connected regions from the license plate pixel region image to obtain a set of contour pixels; and obtaining a region contour based on the set of contour pixels.

[0066] Specifically, the server first uses a license plate localization model to obtain the license plate pixel region image Ib from the license plate image I. In this embodiment, the license plate localization model is trained using the Mask R-CNN semantic segmentation network; however, it should be understood that other existing segmentation networks can also be used depending on actual needs. Then, the server extracts all region contours in the Ib image, mainly using connected component extraction operators in image processing to obtain the set of contour pixels in image Ib. Based on this set of contour pixels, the corresponding region contours can then be obtained, such as... Figure 5 As indicated by the black lines, the set of outline pixels can be recorded as C={c1,c2,…,cn}.

[0067] Step S202: Fit the contour of each region with the convex hull to obtain the set of convex hull pixels, and calculate the perimeter of the corresponding region contour based on the set of convex hull pixels.

[0068] Specifically, for any given region contour, the server performs convex hull fitting to obtain a set of convex hull pixels. Then, the perimeter of the region contour is determined based on the distances between pixels in the convex hull pixel set.

[0069] In one embodiment, step S202 includes: fitting the contour of each region using the convex hull fitting operator to obtain a set of convex hull pixels; calculating the distance between adjacent pixels in the set of convex hull pixels in turn; and summing the distances as the perimeter of the region contour.

[0070] Specifically, the server first uses the convex hull fitting operator in image processing to fit the region contour C, obtaining the convex hull pixel set hull. Then, it calculates the distances between adjacent pixels in the convex hull in turn, and sums all the distances to obtain the perimeter arc of the region contour C.

[0071] Step S203: Determine the fitting accuracy based on the perimeter, and use the fitting accuracy to perform polygon fitting on the convex hull pixel set, retaining the fitted quadrilateral.

[0072] Specifically, after the server obtains the perimeter (arc), it determines the polygon fitting precision (step) based on this perimeter, and then fits a polygon to the convex hull pixel set (hull) based on the fitting precision. Then, once the fitted polygon is a quadrilateral, the quadrilateral and its corresponding fitting precision (step) are retained and recorded.

[0073] In one embodiment, step S203 includes: using the product of each preset hyperparameter coefficient and the perimeter as the fitting accuracy; using each fitting accuracy and a polygon fitting algorithm to fit the pixels in the convex hull pixel set to obtain the fitting polygon corresponding to each fitting accuracy; and retaining quadrilaterals with 4 vertices from the fitting polygons.

[0074] Specifically, different fitting accuracies, step = a × arc, are used sequentially from smallest to largest, where a is a hyperparameter coefficient. In this embodiment, the preset values ​​are 0.05, 0.10, 0.15, 0.20, and 0.25. Then, the server uses a polygon fitting algorithm in image processing to fit the pixels in the convex hull. If the fitted polygon has 4 vertices, it indicates a quadrilateral, and the current quadrilateral is retained as approx and its corresponding fitting accuracies step. The fitted quadrilateral is as follows: Figure 6 As shown.

[0075] Step S204: When the area of ​​the quadrilateral meets the preset threshold condition, use half of the fitting accuracy of the quadrilateral to perform polygon fitting on the set of convex hull pixels to obtain the fitted polygon.

[0076] Specifically, after fitting the quadrilateral, the approximation area of ​​the quadrilateral is calculated. If the area meets the set threshold condition, the quadrilateral's fitting accuracy step / 2 is used to refit the pixels in the convex hull using a polygon fitting algorithm in image processing, resulting in the approximation2 of the fitted polygon. The area threshold condition is set based on the actual size and area of ​​the license plate.

[0077] Step S205: Determine the license plate edge line based on the vertices of the polygon, extend the license plate edge line to obtain the license plate vertices, and determine the license plate region image based on the license plate vertices.

[0078] Specifically, after the server obtains the second fitted polygon, it uses the vertices of the polygon to filter out the edge lines of the license plate. Generally, there are four edge lines for a license plate. Then, these four edge lines are extended to obtain four intersection points, which are recorded as license plate vertices. Based on these vertices, the license plate area image can be accurately extracted.

[0079] In one embodiment, step S205 includes: drawing circles with the four vertices of the quadrilateral as centers and the fitting accuracy corresponding to the quadrilateral as the radius; retaining the vertices of the polygon that are inside any one of the circles as candidate vertices; calculating the distance between adjacent candidate vertices, and if the distance is greater than twice the fitting accuracy corresponding to the quadrilateral, determining the line segment containing the adjacent candidate vertices as the license plate edge line; extending the license plate edge line to obtain the license plate vertices, and determining the license plate region image based on the license plate vertices.

[0080] For details, please refer to Figure 7 First, such as Figure 7 As shown in (c), circles are drawn with the four vertices of the quadrilateral as centers and the corresponding fitting accuracy step as the radius. Next, for all vertices in polygon approx2, they are judged sequentially according to their original outline order. If any vertex in the polygon lies within the interval of any circle, that vertex is added to the set pps2 as a candidate vertex. Then, for points in set pps2, the distance between two adjacent candidate vertices is calculated. If the distance is greater than 2 × step, the line segment containing those two adjacent candidate vertices is the license plate edge line. This calculation ultimately yields four line segments, i.e., four license plate edge lines, as shown below. Figure 7 As shown in (d). Finally, extend the four license plate edge segments to obtain four intersection points, which are the vertices of the license plate, as shown in (d). Figure 7 As shown in (e), taking the lower left corner intersection as the first vertex p1, and sorting clockwise, we get vertices p2, p3, and p4 in sequence, as follows. Figure 7 As shown in (f), the license plate region image is then extracted based on the license plate vertices [p1,p2,p3,p4].

[0081] In one embodiment, determining the license plate region image based on license plate vertices includes: calculating a transformation matrix based on the license plate vertices and preset standard license plate vertices; performing a perspective transformation on the license plate image using the transformation matrix to obtain a new license plate image; and extracting the license plate region image from the new license plate image.

[0082] Specifically, the transformation matrix M is calculated between the license plate vertices [p1,p2,p3,p4] and the standard license plate points [(0+β,height+γ),(0+β, 0+γ),(width+β,0+γ),(width+β,height+γ)], where β and γ are adjustment coefficients. In this embodiment, β = 0.3 × width and γ = 0.1 × height. Because license plate character recognition often involves capturing more image information from the surrounding area, standard transformation helps improve recognition accuracy. Then, the original license plate image I is transformed using the transformation matrix M to obtain a new license plate image Is, as shown below. Figure 8 As shown, an image region with the top left corner at (0,0) and the bottom right corner at (width+2×β, height+2×γ) is extracted from the new image Is and considered as the license plate region image, as shown. Figure 9 As shown.

[0083] In one embodiment, to avoid the vertical capture of the license plate affecting recognition accuracy, after extending the license plate edge line to obtain the license plate vertices and before determining the license plate region image based on the license plate vertices, the method further includes: taking the lower left corner license plate vertex as the first vertex, and sorting the remaining license plate vertices clockwise as the second, third, and fourth vertices; calculating the distances between the second vertex and the first, third, and fourth vertices respectively, and taking the largest distance as the license plate width; calculating the distances between the first vertex and the second, third, and fourth vertices respectively, and taking the largest distance as the license plate height; when the license plate height is greater than the license plate width, swapping the license plate height and width and adjusting the vertex order.

[0084] Specifically, first, the vertex at the lower left corner of the license plate is regarded as the first vertex p1 in the original way, and the remaining license plate vertices are sorted clockwise as the second vertex p2, the third vertex p3, and the fourth vertex p4. Then, calculate the distances between the second vertex p2 and the first vertex p1, the third vertex p3, and the fourth vertex p4 respectively, and take the maximum distance as the license plate width width. And calculate the distances between the first vertex p1 and the second vertex p2, the third vertex p3, and the fourth vertex p4 respectively, and take the maximum distance as the license plate height heigth. Finally, compare the sizes of the height and the width. When the license plate height height is greater than the license plate width width, it means that it does not conform to the conventional license plate size and may be a vertically photographed license plate. Therefore, it is necessary to swap the license plate height and the license plate width, change the license plate height to the license plate width, and change the license plate width to the license plate height. At the same time, adjust the vertex order, regard the original vertex p4 as the first vertex p1, and then sort clockwise to obtain the vertices p2, vertex p3, and vertex p4 in sequence again, so as to avoid the influence of vertical shooting.

[0085] Step S206: Perform character recognition on each license plate area image to obtain license plate characters.

[0086] Specifically, when all the regional contours C obtain the corresponding license plate area images based on the above method, the server then uses the license plate character recognition model to recognize all the license plate area images to obtain license plate characters. Figure 8 and Figure 9 The characters corresponding to the license plate shown are "Jin KVAXXX". Among them, the license plate character recognition model in this embodiment is preferably trained by a character recognition network constructed by ResNet18 + BiLSTM + CTC.

[0087] The above license plate positioning and recognition method, based on the geometric characteristics of the license plate itself, outputs the license plate area contour by positioning the license plate image, and then performs fitting optimizations such as convex hull fitting and polygon fitting based on the license plate area contour, so as to extract a more accurate license plate area, and further improve the accuracy of subsequent license plate character recognition.

[0088] It should be understood that although Figure 2 the steps in the flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, Figure 2At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0089] In one embodiment, such as Figure 10 As shown, a license plate location and recognition device is provided, comprising:

[0090] The contour extraction module 1001 is used to obtain the license plate pixel area image from the license plate image and extract all the region contours in the license plate pixel area image.

[0091] The convex hull fitting module 1002 is used to fit the contour of each region with a convex hull to obtain a set of convex hull pixels, and to calculate the perimeter of the corresponding region contour based on the set of convex hull pixels.

[0092] The polygon fitting module 1003 is used to determine the fitting accuracy based on the perimeter, and to perform polygon fitting on the set of convex hull pixels using the fitting accuracy, thus preserving the fitted quadrilateral.

[0093] The polygon fitting module 1003 is also used to perform polygon fitting on the set of convex hull pixels using half of the fitting accuracy of the quadrilateral when the area of ​​the quadrilateral meets the preset threshold condition, so as to obtain the fitted polygon.

[0094] The license plate extraction module 1004 is used to determine the license plate edge line based on the vertices of the polygon, extend the license plate edge line to obtain the license plate vertices, and determine the license plate region image based on the license plate vertices.

[0095] The character recognition module 1005 is used to perform character recognition on the images of each license plate area to obtain the license plate characters.

[0096] In one embodiment, the contour extraction module 1001 is further configured to obtain a license plate pixel region image from a license plate image using a trained license plate localization model; extract connected regions from the license plate pixel region image to obtain a set of contour pixels; and obtain the region contour based on the set of contour pixels.

[0097] In one embodiment, the convex hull fitting module 1002 is further configured to fit the contour of each region using the convex hull fitting operator to obtain a set of convex hull pixels; calculate the distance between adjacent pixels in the set of convex hull pixels in turn; and sum the distances as the perimeter of the region contour.

[0098] In one embodiment, the polygon fitting module 1003 is further configured to use the product of each preset hyperparameter coefficient and the perimeter as the fitting accuracy; to use each fitting accuracy and the polygon fitting algorithm to fit the pixels in the convex hull pixel set to obtain the fitting polygon corresponding to each fitting accuracy; and to retain quadrilaterals with 4 vertices from the fitting polygons.

[0099] In one embodiment, the license plate extraction module 1004 is further configured to draw circles with the four vertices of the quadrilateral as centers and the fitting accuracy of the quadrilateral as the radius; retain the vertices of the polygon that are inside any one of the circles as candidate vertices; calculate the distance between adjacent candidate vertices, and if the distance is greater than twice the fitting accuracy of the quadrilateral, determine the line segment of the adjacent candidate vertices as the license plate edge line; extend the license plate edge line to obtain the license plate vertices, and determine the license plate region image based on the license plate vertices.

[0100] In one embodiment, the license plate extraction module 1004 is further configured to calculate a transformation matrix based on the license plate vertices and a preset standard license plate vertices; perform perspective transformation on the license plate image using the transformation matrix to obtain a new license plate image; and extract the license plate region image from the new license plate image.

[0101] In one embodiment, the license plate extraction module 1004 is further configured to take the lower left corner of the license plate as the first vertex, and sort the remaining license plate vertices in a clockwise direction as the second, third, and fourth vertices; calculate the distances between the second vertex and the first, third, and fourth vertices respectively, and take the largest distance as the license plate width; calculate the distances between the first vertex and the second, third, and fourth vertices respectively, and take the largest distance as the license plate height; when the license plate height is greater than the license plate width, swap the license plate height and the license plate width and adjust the vertex order.

[0102] Specific limitations regarding the license plate location and recognition device can be found in the limitations of the license plate location and recognition method described above, and will not be repeated here. Each module in the aforementioned license plate location and recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various license plate location and recognition method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms.

[0103] In one embodiment, a computer device is provided, which may be a server, including a processor, memory, and a network interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a license plate location and recognition method. Exemplarily, the computer program may be divided into one or more modules, one or more of which are stored in memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the computer device.

[0104] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the computer device, connecting various parts of the computer device via various interfaces and lines.

[0105] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0106] Those skilled in the art will understand that the computer device structure shown in this embodiment is only a partial structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components, or combine certain components, or have different component arrangements.

[0107] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the license plate location and recognition method described in any of the above embodiments.

[0108] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the license plate location and recognition method described in any of the above embodiments.

[0109] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0110] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A license plate positioning recognition method, characterized in that, The method comprises the following steps: obtaining a license plate pixel point area image from a license plate image, extracting all region contours in the license plate pixel point area image; fitting each region contour to obtain a convex hull pixel point set, and calculating the perimeter of the region contour according to the convex hull pixel point set; determining the fitting accuracy according to the perimeter, fitting the convex hull pixel point set to a polygon using the fitting accuracy, and retaining the fitted quadrilateral; the product of each preset hyperparameter coefficient and the perimeter is used as the fitting accuracy; when the area of the quadrilateral meets a preset threshold condition, fitting the convex hull pixel point set to a polygon using one-half of the fitting accuracy corresponding to the quadrilateral, and obtaining a fitted polygon; determining a license plate edge line according to the vertices of the polygon, extending the license plate edge line to obtain a license plate vertex, and determining a license plate region image based on the license plate vertex; specifically, taking the four vertices of the quadrilateral as the centers and the fitting accuracy corresponding to the quadrilateral as the radius to draw a circle; retaining the vertices in the polygon that are within any one circle as candidate vertices; calculating the distance between adjacent candidate vertices, and determining that the line segment between adjacent candidate vertices is a license plate edge line if the distance is greater than twice the fitting accuracy corresponding to the quadrilateral; extending the license plate edge line to obtain a license plate vertex, and determining a license plate region image based on the license plate vertex; performing character recognition on each license plate region image to obtain license plate characters.

2. The method of claim 1, wherein, The method of obtaining a license plate pixel point area image from a license plate image and extracting all region contours in the license plate pixel point area image comprises the following steps: obtaining a license plate pixel point area image from a license plate image using a trained license plate positioning model; performing connected region extraction on the license plate pixel point area image to obtain a contour pixel point set; obtaining region contours according to the contour pixel point set.

3. The method of claim 1, wherein, The method of fitting each region contour to obtain a convex hull pixel point set and calculating the perimeter of the region contour according to the convex hull pixel point set comprises the following steps: fitting each region contour using a convex hull fitting operator to obtain a convex hull pixel point set; calculating the distance between adjacent pixel points in the convex hull pixel point set in sequence; summing each distance as the perimeter of the region contour.

4. The method of claim 1, wherein, The method of fitting the convex hull pixel point set to a polygon using the fitting accuracy and retaining the fitted quadrilateral comprises the following steps: fitting the pixel points in the convex hull pixel point set using each fitting accuracy and a polygon fitting algorithm to obtain a fitted polygon corresponding to each fitting accuracy; retaining a quadrilateral with four vertices from the fitted polygon.

5. The method of claim 1, wherein, The method of determining a license plate region image based on the license plate vertex comprises the following steps: calculating a transformation matrix according to the license plate vertex and a preset standard license plate vertex; performing perspective transformation on the license plate image using the transformation matrix to obtain a new license plate image; cutting a license plate region image from the new license plate image.

6. The method of claim 1, wherein, After extending the license plate edge line to obtain a license plate vertex, and before determining a license plate region image based on the license plate vertex, the method further comprises the following steps: Taking the top corner of the left lower corner as the first vertex, sequentially arranging the remaining top corners of the license plate as the second vertex, the third vertex and the fourth vertex in a clockwise direction; Calculating the distances between the second vertex and the first vertex, the third vertex and the fourth vertex respectively, and taking the maximum distance as the license plate width; Calculating the distances between the first vertex and the second vertex, the third vertex and the fourth vertex respectively, and taking the maximum distance as the license plate height; When the license plate height is greater than the license plate width, swapping the license plate height and the license plate width and adjusting the vertex order.

7. A license plate location recognition apparatus characterized by comprising: Comprise: A contour extraction module for obtaining a license plate pixel region image from a license plate image, and extracting all region contours in the license plate pixel region image; A convex hull fitting module for fitting a convex hull to each of the region contours to obtain a convex hull pixel set, and calculating the perimeter of the corresponding region contour according to the convex hull pixel set; A polygon fitting module for determining a fitting accuracy according to the perimeter, performing polygon fitting on the convex hull pixel set using the fitting accuracy, and retaining a fitted quadrilateral; and multiplying each of preset hyperparameter coefficients by the perimeter to obtain the fitting accuracy; The polygon fitting module is further configured to, when the area of the quadrilateral meets a preset threshold condition, perform polygon fitting on the convex hull pixel set using one-half of the fitting accuracy corresponding to the quadrilateral to obtain a fitted polygon; A license plate extraction module for determining a license plate edge line according to the vertices of the polygon, extending the license plate edge line to obtain a license plate vertex, and determining a license plate region image based on the license plate vertex; specifically, taking the four vertices of the quadrilateral as the centers and the fitting accuracy corresponding to the quadrilateral as the radius to draw a circle; retaining the vertices in the polygon that are within any one circle as candidate vertices; Calculating the distance between adjacent candidate vertices, and if the distance is greater than twice the fitting accuracy corresponding to the quadrilateral, determining the line segment between the adjacent candidate vertices as a license plate edge line; extending the license plate edge line to obtain a license plate vertex, and determining a license plate region image based on the license plate vertex; A character recognition module for performing character recognition on each of the license plate region images to obtain license plate characters.

8. A computer device comprising a processor and a memory, the memory storing a computer program, characterized in that, The processor is configured to execute the computer program to implement the license plate positioning and recognition method of any one of claims 1-6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is configured to be executed by the processor to implement the license plate positioning and recognition method of any one of claims 1-6.

Images