Method for extracting characters from a vehicle license plate and license plate character extraction device for performing the method

Abstract

A method by which a license plate character extraction device extracts characters from a vehicle license plate is provided. The method can include the steps of converting a pre-input image obtained by photographing a license plate of a vehicle into a gray scale image; comparing a color value of a pixel included in the gray scale image with a first average value of color values of pixels in the vicinity of the pixel in order to generate a conversion image; comparing the conversion image with a binary image obtained by binarizing the conversion image in order to generate a thinning image; and extracting characters included in the thinning image.

Description

Technical Field

[0001] This disclosure relates to a method for extracting characters from a vehicle's license plate and a license plate character extraction apparatus for performing the method. Background Technology

[0002] A license plate recognition system is a system that detects vehicle license plates and identifies the characters on them, and can be used to manage vehicle entry and exit from parking lots.

[0003] In a license plate recognition system, in order to identify a vehicle's license plate, it is necessary to photograph the license plate and then separate the license plate characters from the photographed plate.

[0004] In existing license plate recognition systems, methods such as edge detection and binarization are used to separate license plate characters from the license plate. Edge detection and binarization methods use thresholds to separate license plate characters, but this approach has the problem that the recognition rate of license plate characters cannot be guaranteed when the brightness distribution across the entire license plate changes due to variations in lighting position, brightness, etc. Summary of the Invention

[0005] This disclosure provides a method for extracting license plate characters from an image by comparing the value of a pixel with the average value of the pixels surrounding that pixel.

[0006] However, the problems to be solved by this disclosure are not limited to those mentioned above, and those skilled in the art will clearly understand from the following description other problems not mentioned.

[0007] According to one aspect of this disclosure, a method for extracting characters from a vehicle's license plate, performed by a license plate character extraction device, is provided. The method includes: converting an input image obtained by photographing the vehicle's license plate into a grayscale image; generating a converted image based on a comparison of the value of at least one pixel included in the grayscale image with a first average value indicating the average value of pixels adjacent to at least one pixel; generating a thinned image based on a comparison of the converted image with a binarized image obtained by binarizing the converted image; and extracting characters included in the thinned image.

[0008] Generating a converted image may include: determining a converted value of at least one pixel that reflects the result of comparing the value of at least one pixel to be included in a grayscale image with a first average value; and generating a converted image including at least one pixel having the converted value.

[0009] The method further includes generating at least one marker by marking a grayscale image, the at least one marker comprising pixels having predetermined transformation values ​​and being in contact with each other.

[0010] Determining the conversion value may include: increasing the value of all pixels in the grayscale image that are less than a first average value, and decreasing the value of all pixels in the grayscale image that are equal to or greater than the first average value, in the absence of a marker in the grayscale image with a length equal to or greater than a predetermined length.

[0011] The method further includes generating at least one marker by marking a grayscale image, the at least one marker comprising pixels having predetermined values ​​and being in contact with each other.

[0012] Determining the conversion value may include: if there are markers in the grayscale image with a length equal to or greater than a predetermined length, converting the values ​​of pixels in the grayscale image that are equal to or less than a first average value.

[0013] Extracting characters included in a thinned image includes: determining the character bounding box of each character included in the thinned image; calculating the degree of overlap between the character bounding box and the template bounding boxes included in various predetermined license plate templates; and based on the calculated degree of overlap, using the license plate template with the highest degree of overlap with the character bounding box in the predetermined license plate templates to extract the character.

[0014] The method further includes: determining a second average value of the values ​​of pixels that have a preset value or a larger value among the pixels included in a predetermined region in the converted image; and generating a binarized image from the converted image based on the result of comparing the values ​​of the pixels included in the converted image with the second average value.

[0015] According to another aspect of this disclosure, a license plate character extraction apparatus is provided, the apparatus comprising: a camera configured to photograph a vehicle's license plate; a memory configured to store a license plate character extraction model for extracting characters from an image obtained by photographing the vehicle's license plate; and a processor.

[0016] The processor is configured to: execute a license plate character extraction model and convert an image obtained by photographing a vehicle's license plate into a grayscale image; generate a converted image based on a comparison of the values ​​of pixels included in the grayscale image with a first average of the values ​​of pixels surrounding that pixel; generate a thinned image based on a comparison of the converted image with a binarized image obtained by binarizing the converted image; and extract characters included in the thinned image.

[0017] According to another aspect of this disclosure, a non-transitory computer-readable storage medium including computer-executable instructions is provided, wherein the instructions, when executed by a processor, cause the processor to perform a method for extracting characters from a vehicle's license plate, the method comprising: converting an input image obtained by photographing the vehicle's license plate into a grayscale image; generating a converted image based on a comparison of the value of at least one pixel included in the grayscale image with a first average of the values ​​of pixels adjacent to that pixel; generating a thinned image based on a comparison of the converted image with a binarized image obtained by binarizing the converted image; and extracting characters included in the thinned image.

[0018] According to embodiments of this disclosure, the value of a pixel is compared with the average value of the values ​​of pixels surrounding such a pixel, and license plate characters are extracted from the image, ensuring a certain high recognition rate regardless of changes in the position or brightness of the lighting. Attached Figure Description

[0019] Figure 1 This is a block diagram illustrating a license plate character extraction device according to an embodiment of the present disclosure.

[0020] Figure 2 This is a block diagram conceptually illustrating the functionality of a license plate character extraction model according to an embodiment of this disclosure.

[0021] Figure 3 (a) and (b) show examples of grayscale images and altered images, in which the values ​​have been changed from those in the grayscale image.

[0022] Figure 4 (a) through (c) show another example illustrating a grayscale image and a modified image, in which the values ​​have been changed from those in the grayscale image.

[0023] Figure 5 (a) and (b) are examples illustrating the process of detecting a license plate border in a license plate border detection unit according to an embodiment of the present disclosure.

[0024] Figure 6 An example of a binarized image generated by a license plate binarization unit according to an embodiment of this disclosure is shown.

[0025] Figure 7 An example of a thinned image generated by an image thinning unit according to an embodiment of this disclosure is shown.

[0026] Figure 8 Images (a) through (d) show examples of license plate templates according to embodiments of this disclosure.

[0027] Figure 9 It shows Figure 8 The form of setting the area of ​​interest in the license plate template of (a) is used.

[0028] Figure 10 (a) and (b) show examples of the process of obtaining a matching template in a template matching unit according to an embodiment of the present disclosure.

[0029] Figure 11 An example of a license plate character extracted by a character extraction unit according to an embodiment of this disclosure is shown.

[0030] Figure 12 This is a flowchart illustrating a method for extracting license plate characters of a vehicle according to an embodiment of the present disclosure. Detailed Implementation

[0031] The advantages and features of the embodiments, as well as the methods of implementing the embodiments, will become clear from the following description taken in conjunction with the accompanying drawings. However, the embodiments are not limited to those described, as they can be implemented in various forms. It should be noted that these embodiments are provided for the purpose of full disclosure and to enable those skilled in the art to understand the full scope of the embodiments. Therefore, the embodiments are defined only by the scope of the appended claims.

[0032] The terminology used in this specification will be briefly described, and the contents of this disclosure will be described in detail.

[0033] The terminology used in this disclosure employs commonly used terms that are as widely applicable as possible, while taking into account the functions of this disclosure. However, these terms may be changed based on the intent or precedent of those skilled in the art, the emergence of new technologies, etc. Furthermore, in some cases, terms may be arbitrarily chosen by the applicant, and in such cases, the meaning of the terms will be described in detail in the description of the corresponding invention. Therefore, the terms used in this disclosure should be defined based on their meaning and the overall content of this disclosure, and not merely by their names.

[0034] When a section of the instruction manual describes a part as "including" a component, this means, rather than excluding, that other components may also be included, unless otherwise specified.

[0035] Additionally, the terms "unit" or "part" used in this specification refer to software components or hardware components (e.g., FPGAs or ASICs), and "unit" and "part" perform a certain function. However, "unit" or "part" is not limited to software or hardware. A "part" or "unit" may be configured in addressable storage media or may be configured to reproduce one or more processors. Thus, for example, a "unit" or "part" includes components (e.g., software components, object-oriented software components, class components, and task components), processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided in components and "units" may be combined into a smaller number of components and "units," or may be further divided into other components and "units."

[0036] In the following description, embodiments of the present disclosure will be illustrated in detail with reference to the accompanying drawings, enabling those skilled in the art to readily implement the present disclosure. In the drawings, portions irrelevant to the description have been omitted for clarity.

[0037] Figure 1 This is a block diagram illustrating a license plate character extraction device according to an embodiment of the present disclosure.

[0038] Reference Figure 1 and Figure 2 The vehicle license plate character extraction device 100 may include a processor 110, a memory 120, and a transceiver 130.

[0039] The processor 110 can control all functions of the vehicle license plate character extraction device 100.

[0040] The memory 120 can store the license plate character extraction model 200 and the information required to execute the license plate character extraction model 200.

[0041] In this specification, "license plate character extraction model 200" refers to software (computer program code) for performing the function of extracting license plate characters from an image or set of images obtained by photographing a vehicle's license plate, and can be implemented through a series of instructions.

[0042] The processor 110 can load the license plate character extraction model 200 stored in the memory 130 and the information required to execute the license plate character extraction model 200, so as to execute the license plate character extraction model 200.

[0043] The processor 110 can execute the license plate character extraction model 200 to convert an image of a vehicle license plate received from a camera device (not shown) into a grayscale image, extract a converted image obtained by converting the pixel values ​​of the license plate characters from the grayscale image, detect the license plate border from the converted image, binarize the converted image to generate a binarized image, refine the binarized image using the converted image from which the license plate border has been detected, and match the refined image with a license plate template to extract the license plate characters. In this specification, "license plate characters" refers to the characters that constitute the vehicle number on the license plate and may include both numbers and characters.

[0044] The processor 110 can control at least one of the memory 120 and the transceiver 130 to execute the license plate character extraction model 200.

[0045] Reference Figure 2 A more detailed description of the functions of license plate character extraction model 200.

[0046] The transceiver 130 can receive images of vehicle license plates from a camera device (not shown) that captures vehicle license plates.

[0047] This specification has described the use of transceiver 130 to receive images of vehicle license plates from a camera device (not shown), but the disclosure is not limited thereto. That is, according to an embodiment, the license plate character extraction device 100 may include input and output devices (not shown), and the license plate character extraction device 100 may use the input and output devices to receive images of vehicle license plates.

[0048] Figure 2 This is a block diagram conceptually illustrating the functionality of a license plate character extraction model according to an embodiment of this disclosure.

[0049] Reference Figure 2 The license plate character extraction model 200 may include an image receiving unit 210, an image preprocessing unit 220, a pixel value conversion unit 230, a license plate border detection unit 240, a license plate binarization unit 250, an image thinning unit 260, a template matching unit 270, and a character extraction unit 280.

[0050] exist Figure 2In the license plate character extraction model 200 shown, the functions of the image receiving unit 210, image preprocessing unit 220, pixel value conversion unit 230, license plate border detection unit 240, license plate binarization unit 250, image thinning unit 260, template matching unit 270, and character extraction unit 280 are conceptually divided to facilitate the explanation of the functions of the license plate character extraction model 200, and this disclosure is not limited thereto. That is, according to the embodiment, in the license plate character extraction model 200, the functions of the image receiving unit 210, image preprocessing unit 220, pixel value conversion unit 230, license plate border detection unit 240, license plate binarization unit 250, image thinning unit 260, template matching unit 270, and character extraction unit 280 can be combined or separated, and can be implemented by a series of instructions included in one or more programs.

[0051] The image receiving unit 210 can receive images obtained by taking pictures of vehicle license plates using the camera device 120.

[0052] According to the implementation method, when the received image of the vehicle license plate does not meet the preset standard, the image preprocessing unit 220 can correct the received image of the vehicle license plate.

[0053] Preset standards include whether the license plate is tilted at a preset angle (e.g., 5 degrees) or greater in the image of the vehicle license plate, whether the vehicle license plate includes the center of the image, and whether the horizontal and vertical lengths of the license plate are equal to or greater than half of the horizontal and vertical lengths of the image.

[0054] For example, when the license plate is tilted at a preset angle (e.g., 5 degrees) or greater in an image of a vehicle license plate, the image preprocessing unit 220 can reduce the tilt of the license plate by performing a distortion.

[0055] Furthermore, when the vehicle's license plate does not include the center of the image, the image preprocessing unit 220 can move the vehicle's license plate and receive the image of the vehicle's license plate again, or take another picture of the vehicle's license plate from the entire image to correct the image so that the vehicle's license plate includes the center of the image.

[0056] Finally, when the horizontal length of the license plate is less than half the horizontal length of the image or the vertical length of the license plate is less than half the vertical length of the image, the image preprocessing unit 220 can receive the vehicle license plate image again, or take another image of the vehicle license plate from the entire image, and correct the image so that the horizontal and vertical lengths of the license plate are equal to or greater than half the horizontal and vertical lengths of the image.

[0057] The image of the vehicle license plate received by the image receiving unit 210 can be a color image or a grayscale image.

[0058] When the image receiving unit 210 receives a color image, the image preprocessing unit 220 can convert the received color image into a grayscale image. This is intended to reduce the complexity of extracting license plate characters, since the background or color of the license plate characters is not fixed.

[0059] According to the embodiment, the image preprocessing unit 220 can perform smoothing on the image of the vehicle license plate. This is intended to avoid the problem that reduces the accuracy of license plate character extraction due to character segmentation and merging caused by noise in the image or background. Since smoothing is a known technique, its description will be omitted.

[0060] The pixel value conversion unit 230 can amplify the pixel values ​​of the license plate characters in the grayscale image of the vehicle license plate in order to separate the license plate characters from the vehicle's license plate. In the prior art, a threshold is used to perform binarization on the entire image; however, in this case, there is a problem that the accuracy of character extraction changes due to variations in the lighting distribution.

[0061] To address this issue, the pixel value conversion unit 230 can determine whether to use the color values ​​of pixels surrounding the pixel instead of a threshold to convert the pixel's color value in order to separate characters from the grayscale image.

[0062] More specifically, the pixel value conversion unit 230 can compare the value of a pixel with the average value of the pixels surrounding that pixel, and determine whether to convert the pixel value based on the comparison result. Here, pixels surrounding the pixel can refer to pixels that are adjacent to the pixel and located within a predetermined distance or range from it.

[0063] According to an embodiment, the pixel value conversion unit 230 can use the following formula 1 to convert the values ​​of pixels included in the grayscale image of a vehicle license plate to generate a converted image.

[0064] [Formula 1]

[0065]

[0066] Here, D(p) can refer to the value of pixel p in the transformed image D, G(p) can refer to the value of pixel p in the grayscale image G, and M(p) can refer to the average value of the pixels surrounding pixel p.

[0067] That is, if the value of pixel p in the grayscale image G(p) representing the vehicle license plate is less than (or equal to or less than) the average value M(p) of the surrounding pixels, the pixel value conversion unit 230 can determine that the color of pixel p is black (or a color close to black), and determine the value of pixel p in the converted image D by subtracting the value of pixel p G(p) from 255.

[0068] On the other hand, if the value of pixel p in the grayscale image G(p) representing the vehicle license plate is greater than (or exceeds) the average value M(p) of the surrounding pixels, the pixel value conversion unit 230 can determine that the color of pixel p is white (or a color close to white) and set the value of pixel p in the converted image D to 0.

[0069] For example, further reference Figure 3 of (a), Figure 3 (a) shows the grayscale image G before it is converted to the transformed image D, and Figure 3 (b) shows the transformed image G derived from the grayscale image G using Equation 1.

[0070] like Figure 3 As shown in (a) and (b), dark areas, such as license plate characters or license plate borders (i.e., areas where G(p) < M(p) in Equation 1), can be converted into bright areas in the converted image (i.e., pixel value D(p) can be amplified to the value obtained by subtracting the value G(p) of pixel p from 255), and bright areas, such as the background inside or outside the license plate (i.e., areas corresponding to "otherwise" in Equation 1), can be converted into dark areas in the converted image (i.e., pixel value D(p) can be converted to 0).

[0071] However, further reference Figure 4 (a) through (c) depend on the type of vehicle license plate, and there may be license plates with white license plate characters.

[0072] Therefore, when Figure 4 When the grayscale image in (a) is substituted into Equation 1, as shown... Figure 4 As shown in (b), bright areas in the grayscale image, such as license plate characters or license plate borders (i.e., areas corresponding to "otherwise" in Equation 1), are converted to dark areas in the converted image (i.e., pixel value D(p) is converted to 0), and dark areas, such as the background inside or outside the license plate (i.e., areas where G(p) < M(p) in Equation 1), are converted to bright areas in the converted image (i.e., pixel value D(p) can be amplified to the value obtained by subtracting the value G(p) of pixel p from 255).

[0073] That is, as from Figure 3 and Figure 4As can be seen, the portion of the pixel value that is magnified in the grayscale image can vary depending on the type of license plate. Therefore, in order to obtain the same converted image regardless of the license plate type, the pixel value conversion unit 230 needs to determine whether the region in the image obtained by substituting the grayscale image into Equation 1 is the character region of the license plate or the background region of the license plate.

[0074] Therefore, the pixel value conversion unit 230 can perform marking on the converted image obtained using Equation 1.

[0075] Here, "marking" can refer to the process of grouping adjacent pixels with the same pixel value into a group by setting the same mark to adjacent (i.e., connected) pixels in an image that have the same pixel value (or pixel values ​​within a predetermined range or equal to or greater than a predetermined pixel value) (here, pixel value greater than 0). For example, due to the constituting Figure 3 In (b), the pixels of the digit 5 ​​are adjacent to each other and have the same pixel value equal to or greater than a predetermined value, so the pixels constituting the digit 5 ​​can form a marker LB1. On the other hand, in Figure 4 In (b), since the pixels located outside the boundary of digit 8646 are adjacent to each other and have the same pixel value equal to or greater than a predetermined value, the pixels located outside the boundary of digit 8646 and surrounding digit 8646 can form a marker (LB2).

[0076] That is, assuming the horizontal and vertical lengths of the converted image are w and h respectively, and each license plate character in the converted image forms a mark, since there should be at least four license plate characters in the converted image, the horizontal length of the mark for each license plate character should be less than w / 4. On the other hand, assuming the background forms the mark in the converted image, a mark with a length greater than w / 4 can appear in the center of the license plate.

[0077] Therefore, if the horizontal length of the mark that contacts the line segment connecting (w / 4, h / 2) to (3w / 4, h / 2) in the converted image is less than w / 4 (i.e., if there is no mark with a horizontal length greater than w / 4), the pixel value conversion unit 230 can determine that the license plate character has been enlarged. For example, further refer to... Figure 3 (b) is marked with {9} as it contacts the line segment connecting (w / 4, h / 2) to (3w / 4, h / 2) in the transformed image. {6}, {3}, {2} and some small markers in the background, and due to {9}, The characters {6}, {3}, {2}, and some small marks in the background do not have a horizontal length greater than w / 4, so the pixel value conversion unit 230 can determine that the license plate characters have been enlarged.

[0078] On the other hand, if the horizontal length of the mark that contacts the line segment connecting (w / 4, h / 2) to (3w / 4, h / 2) in the converted image is greater than w / 4, the pixel value conversion unit 230 can determine that the background of the license plate has been enlarged. For example, further refer to Figure 4 In (b), the mark that contacts the line segment connecting (w / 4, h / 2) to (3w / 4, h / 2) in the converted image is the mark LB2 located outside the boundary of the number 8646 and surrounding the number 8646. Since the horizontal length of the mark LB2 is greater than w / 4 (i.e., the length from the leftmost to the rightmost of the mark is greater than w / 4), the pixel value conversion unit 230 can determine that the background of the license plate has been enlarged.

[0079] Therefore, when it is determined that the license plate background is magnified, the pixel value conversion unit 230 can use Equation 2 instead of Equation 1 to convert the pixel values ​​of the grayscale image to generate... Figure 4 The transformed image in (c).

[0080]

Formula 2

[0081]

[0082] In this specification, the horizontal length of a marker can refer to the difference between the x-coordinate of the leftmost pixel and the x-coordinate of the rightmost pixel included in the marker. Furthermore, as will be described below, the vertical length of a marker can refer to the difference between the y-coordinate of the topmost pixel and the y-coordinate of the bottommost pixel included in the marker.

[0083] Furthermore, for ease of description, the transformed image can be represented in this specification as a rectangle with vertices (0, 0), (0, h), (w, 0), and (w, h).

[0084] The license plate border detection unit 240 can detect the license plate border from the converted image generated by the pixel value conversion unit 230.

[0085] More specifically, the license plate border detection unit 240 can perform upward, downward, leftward, and rightward scanning from the center of the converted image, and when a mark is encountered, the point where the mark with a horizontal length equal to or greater than a predetermined length (e.g., w / 4) is encountered is determined as the point where the license plate border is located on the top, bottom, left, and right sides.

[0086] For example, refer to Figure 5 In (a) to (b), when the license plate border detection unit 240 performs a scan from the center (w / 2, h / 2) of the converted image to the left, it will encounter the following in sequence during the scan: The markings for '9' and '5'.

[0087] However, due to Since the horizontal lengths of the '9' mark, the '9' mark, and the '5' mark do not exceed w / 4, the license plate border detection unit 240 can continue scanning.

[0088] Then, since the horizontal length of the mark scanned at the first point LP1 is equal to or greater than w / 4, the license plate frame detection unit 240 can determine that the left side of the license plate frame is located at the first point LP1.

[0089] When the same processing is performed from the center (w / 2, h / 2) to the right, up and down, the license plate frame detection unit 240 can determine that the right side of the license plate frame is located at the second point LP2, the upper side of the license plate frame is located at the third point LP3, and the lower side of the license plate frame is located at the fourth point LP4.

[0090] like Figure 5 As shown in (b), when a rectangle connecting the first point LP1, the second point LP2, the third point LP3 and the fourth point LP4 is formed, the license plate border detection unit 240 can detect the license plate border LP.

[0091] The license plate binarization unit 250 can binarize the converted image generated by the pixel value conversion unit 230. Here, unlike a grayscale image representing light intensity, binarization can refer to representing the image with black and white.

[0092] like Figure 3 As shown in (a) and (b), in the pixel value conversion unit 230, when generating the image of the character conversion, not only can the license plate characters be enlarged, but also a portion of the background inside or outside the license plate can be enlarged (i.e., the values ​​of pixels corresponding to a portion of the background inside or outside the license plate can also be enlarged). Therefore, the license plate binarization unit 250 can binarize the converted image in order to remove the background inside or outside the license plate that does not need to be enlarged from the converted image.

[0093] Assuming a marker exists at the midpoint of the transformed image, the license plate binarization unit 250 can calculate a threshold on the line connecting the midpoint (i.e., h / 2) for binarization.

[0094] More specifically, the license plate binarization unit 250 can calculate the average value of the pixels whose pixel values ​​have been amplified (i.e., pixels whose pixel values ​​are not 0) among the pixels located on the line segment while scanning the line segment connecting (0, h / 2) to (w-1, h / 2) from left to right, and determine the calculated average value as the threshold.

[0095] The license plate binarization unit 250 can compare the threshold with the values ​​of pixels included in the transformed image and generate a binarized image based on the comparison result.

[0096] That is, if the value of a pixel included in the converted image is less than (or equal to or less than) a threshold, the license plate binarization unit 250 can change the value of the pixel to '0', and if the value of a pixel included in the converted image is equal to or greater than (or exceeds) a threshold, it can either not change the value of the pixel or change the value of the pixel to a predetermined value equal to or greater than '0' to generate a binarized image.

[0097] Further reference Figure 6 , Figure 6 The image shown can be a binary image generated by the license plate binarization unit 250. When... Figure 6 The binarized image shown is Figure 3 When compared with the transformed image shown in (b), it can be confirmed that binarization has greatly removed the magnified portions of the background inside and outside the license plate.

[0098] The image thinning unit 260 can compare the binarized image generated by the license plate binarization unit 250 with the transformed image from which the license plate border detection unit 240 has detected the license plate border to generate a thinned image.

[0099] More specifically, the image thinning unit 260 can generate a thinned image through the following three steps.

[0100] In the first step, the image thinning unit 260 can perform marking on the binarized image and remove markings from the markings included in the transformed image that do not overlap with the markings included in the binarized image. This is intended to remove markings located in the background of the transformed image.

[0101] In the second step, the image thinning unit 260 can remove markings located outside the license plate border from the converted image, having already removed markings that do not overlap with the markings included in the binarized image in the first step. Here, markings located outside the license plate border can include not only those completely outside the border but also those partially outside. This is because markings located outside the border do not correspond to the license plate characters.

[0102] In the third step, the image thinning unit 260 can remove markers with a length greater than a preset value from the markers included in the binarized image. Here, the length greater than the preset value can include a horizontal length greater than w / 4 and a vertical length greater than 0.9h. This is because it is recognized that since a license plate includes at least four license plate characters, the horizontal length is no greater than w / 4, and the license plate characters are no greater than 0.9 times the vertical length of the grayscale image. However, depending on the implementation, the horizontal and vertical lengths used as the criteria for marker removal can be different.

[0103] According to the implementation method, if the third step is not required, the image thinning unit 260 may only perform the first and second steps.

[0104] The image thinning unit 260 can generate a thinned image through the above steps.

[0105] Further reference Figure 7 , Figure 7 The image shown can be obtained using Figure 5 The converted image of the license plate border has been detected in (b) and Figure 6 The thinned image generated from the binarized image is shown. From Figure 7 It can be confirmed that all markings except for the license plate characters have been removed through the purification process from the first to the third step.

[0106] The template matching unit 270 can compare the pre-input license plate template with the refined image to determine the matching template in the license plate template that matches the refined image.

[0107] More specifically, the template matching unit 270 can compare the region of interest included in each license plate template with the region of interest included in the refined image, and determine the template with the most matching region of interest as the matching template.

[0108] Here, the area of ​​interest included in the license plate template can be a region representing the area where a character in the license plate template is located. An equal number of areas of interest as the number of characters included in the license plate template can be pre-defined in the license plate template, and for example, the areas of interest included in the license plate template can be bounding boxes.

[0109] Furthermore, the region of interest included in the thinned image is the region in which the license plate characters included in the thinned image are marked, and for example, the region of interest can be a bounding box.

[0110] When the template matching unit 270 receives the thinned image, the template matching unit 270 can detect the region of interest included in the thinned image.

[0111] More specifically, the template matching unit 270 can detect the markers of each license plate character in the thinned image in order to detect the region of interest from the thinned image. For example, the template matching unit 270 can identify markers with horizontal and vertical lengths within a preset range among the markers included in the thinned image as markers of license plate characters.

[0112] The template matching unit 270 can calculate the horizontal and vertical lengths of the license plate character mark and set the area of ​​interest to have the same horizontal and vertical lengths as the license plate character mark, or set it to a length greater than the horizontal and vertical lengths of the license plate character mark with a predetermined margin.

[0113] According to the implementation, the template matching unit 270 can apply a refined image with a focus area set for license plate characters and a license plate template with a preset focus area to Equation 3 below to determine a matching template.

[0114]

Formula 3

[0115] in

[0116]

[0117] Here, A n B represents the nth area of ​​interest in license plate template A. m This represents the m-th region of interest in thinning image B, and includes(A) n B m The region of interest (A) represents the degree of overlap between the nth region of interest of license plate template A and the mth region of interest of the refined image B. area(A) represents the width of the region of interest of license plate template A, and area(A∩B) represents the width of the overlapping area between the regions of interest of license plate template A and the regions of interest of refined image B.

[0118] Therefore, the template matching unit 270 can target the area of ​​interest (A) included in the license plate template A. n Each region of interest in B is determined to include the region of interest in the thinned image B. m The overlap values ​​of the most overlapping regions of interest in the image are determined, and the determined overlap values ​​are multiplied to determine the matching template in the license plate template that best matches the thinned image.

[0119] That is, the template matching unit 270 can determine the license plate template with the maximum inclusion degree of Formula 3 as the matching template.

[0120] Further reference Figures 8 to 10 , Figure 8 Examples of license plate templates are shown in (a) to (d). Figure 9 The license plate template T_LP is shown, in which the areas of interest T_BBX1 to T_BBX7 are preset, and Figure 10 The template matching unit 270 is shown to perform a process in which the area of ​​interest of the license plate template is matched with the refined image to obtain a matching template.

[0121] from Figure 10 (a) can be confirmed when making Figure 8 of (a) to (d) Figure 8 (a) and Figure 9 The license plate template T_LP shown is Figure 7 When the image is refined, the region of interest of the refined image overlaps significantly with the regions of interest of the license plate template T_LP, from T_BBX1 to T_BBX7.

[0122] On the other hand, from Figure 10 (b) confirms that when... Figure 8 of (a) to (d) Figure 8 The license plate template shown in (c) is similar to Figure 7 When the image is thinned, the region of interest in the thinned image is compared to... Figure 10 The overlapping areas shown are less overlapping with the area of ​​interest in the license plate template.

[0123] This allows the template matching unit 270 to... Figure 8 The license plate template T_LP of (a) is determined as the matching template.

[0124] The character extraction unit 280 can use the determined matching template to extract only the license plate characters from the thinned image.

[0125] More specifically, the character extraction unit 280 can make the region of interest included in the matching template correspond to the thinned image, and remove the markers not included in the region of interest of the license plate template from the markers included in the thinned image to finally extract the license plate characters.

[0126] Here, being included in the area of ​​interest of the license plate template can mean that the entire mark of the refined image is included in the area of ​​interest of the license plate template, but it can also mean that a preset range or a larger range of the mark of the refined image is included in the area of ​​interest of the license plate template.

[0127] Further reference Figure 11 , Figure 11 The image shows license plate characters extracted by character extraction unit 280. Figure 11 It can be confirmed that license plate characters can be extracted from the license plate through the operations of template matching unit 270 and character extraction unit 280.

[0128] Figure 12 This is a flowchart illustrating a method for extracting license plate characters of a vehicle according to an embodiment of the present disclosure.

[0129] Reference Figure 2 and Figure 12When the image receiving unit 210 receives the image of the vehicle license plate captured by the camera device 120 (S1200), the pixel value conversion unit 230 can convert the pixel values ​​of the license plate characters in the grayscale image of the vehicle license plate (i.e., the pixel values ​​corresponding to the license plate characters) (S1210).

[0130] Subsequently, the license plate border detection unit 240 can detect the license plate border from the converted image obtained by converting the pixel values ​​of the characters (S1220), and the license plate binarization unit 250 can binarize the converted image obtained by converting the pixel values ​​of the license plate characters (S1230).

[0131] In this specification, for ease of description, the case where the license plate border detection unit 240 detects the license plate border in the converted image, and then the license plate binarization unit 250 binarizes the converted image, has been described; however, this disclosure is not limited thereto. That is, according to embodiments, license plate border detection and binarization of the converted image can be performed simultaneously, or the converted image can be binarized first, and then license plate border detection can be performed.

[0132] The image thinning unit 260 can compare the binarized image obtained by binarizing the converted image with the converted image from which the license plate border has been detected to generate a thinned image obtained by thinning the markers of the converted image by using the binarized image (S1240).

[0133] The template matching unit 270 can compare the pre-input license plate template with the refined image to determine the matching template in the license plate template that matches the refined image (S1250).

[0134] The character extraction unit 280 can extract license plate characters from the vehicle's license plate using the matching template determined by the template matching unit 270 (S1260).

[0135] The combination of steps in each flowchart appended to this disclosure can be executed by computer program instructions. Since computer program instructions can be mounted on a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, the instructions, which are executed by the processor of the computer or other programmable data processing apparatus, create means for performing the function described in each step of the flowchart. The computer program instructions can also be stored on a computer-usable or computer-readable storage medium that can direct the computer or other programmable data processing apparatus to implement the function in a particular manner. Therefore, the instructions stored on a computer-usable or computer-readable recording medium can also produce an article of art containing instruction means that perform the function described in each step of the flowchart. The computer program instructions can also be mounted on a computer or other programmable data processing apparatus. Therefore, a series of operational steps are performed on the computer or other programmable data processing apparatus to create a computer-executable process, and the instructions can also execute the computer or other programmable data processing apparatus to provide steps for performing the function described in each step of the flowchart.

[0136] Furthermore, each step may represent a portion of a module, segment, or code containing one or more executable instructions for performing a specified logical function. It should also be noted that in some alternative implementations, the functions mentioned in the steps may occur out of order. For example, two steps shown consecutively may actually be executed substantially simultaneously, or these steps may sometimes be executed in reverse order according to their corresponding functions.

[0137] The above description is merely an exemplary description of the technical scope of this disclosure, and those skilled in the art will understand that various changes and modifications can be made without departing from the original characteristics of this disclosure. Therefore, the embodiments disclosed in this disclosure are intended to illustrate, not limit, the technical scope of this disclosure, and the technical scope of this disclosure is not limited by the embodiments. The scope of protection of this disclosure should be interpreted based on the appended claims, and it should be understood that all technical scopes included within the scope equivalent to the appended claims are included within the scope of protection of this disclosure.

Claims

1. A method for extracting characters from a license plate, the method comprising: Receive input images including vehicle license plates captured by a camera device; Convert the input image to grayscale; The value of at least one pixel in the grayscale image is compared with a first average of the values ​​of the pixels adjacent to the at least one pixel to generate a converted image; A thinned image is generated by comparing the transformed image with a binarized image obtained by binarizing the transformed image; as well as The characters included in the thinned image are extracted using the following steps: Determine the character bounding box of the character in the thinned image; Determine the degree of overlap between the determined character bounding box and the bounding boxes in a plurality of templates, each of the plurality of templates defining a different layout of the bounding box corresponding to the expected character position; as well as The character is extracted from the thinned image using the template that has the highest degree of overlap with the character's bounding box in the thinned image. The method further includes: Determine a second average value of the pixels with a preset value or a larger value included in a predetermined region of the converted image; as well as The binarized image is generated from the transformed image based on the result of comparing the values ​​of the pixels included in the transformed image with the second average value.

2. The method according to claim 1, wherein, The transformed image is generated through the following operations: Determine a transformed value for the at least one pixel that reflects the result of comparing the value of the at least one pixel included in the grayscale image with the first average value; as well as Update the at least one pixel to have the transformed value.

3. The method according to claim 2, further comprising: At least one marker is generated in the grayscale image, the at least one marker comprising pixels having predetermined transformation values ​​and being in contact with each other. Determining the conversion value includes: when there are no markers in the grayscale image with a length equal to or greater than a predetermined length, increasing the value of a first pixel that is less than the first average value, and decreasing the value of a second pixel that is equal to or greater than the first average value.

4. The method according to claim 2, further comprising: At least one marker is generated in the grayscale image, the at least one marker comprising pixels having predetermined pixel values ​​and being in contact with each other. Determining the conversion value includes: when there are markers in the grayscale image with a length equal to or greater than a predetermined length, converting the values ​​of pixels that are equal to or less than the first average value.

5. A license plate character extraction device, comprising: A camera device configured to capture an input image including the vehicle's license plate; One or more processors; as well as A memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: Receive the input image from the camera device; Convert the input image to grayscale. A transformed image is generated based on the result of comparing the values ​​of pixels included in the grayscale image with the first average of the values ​​of pixels surrounding those pixels. A thinned image is generated by comparing the transformed image with a binarized image obtained by binarizing the transformed image. Determine the character bounding boxes of the characters in the thinned image; Determine the degree of overlap between the determined character bounding box and the bounding boxes in a plurality of templates, each of the plurality of templates defining a different layout of the bounding box corresponding to the expected character position; as well as The character is extracted from the thinned image using the template that has the highest degree of overlap with the character's bounding box in the thinned image. The instructions cause the one or more processors to: Determine a second average value of the pixels with a preset value or a larger value included in a predetermined region of the converted image; and The binarized image is generated from the transformed image based on the result of comparing the values ​​of the pixels included in the transformed image with the second average value.

6. The license plate character extraction device according to claim 5, wherein, The instructions cause the one or more processors to: Determine the transformed value of the at least one pixel, reflecting the result of comparing the value of at least one pixel included in the grayscale image with the first average value, and Update the at least one pixel to have the transformed value.

7. The license plate character extraction device according to claim 6, wherein, The instructions cause the one or more processors to: At least one marker is generated in the grayscale image, the at least one marker comprising pixels having predetermined transformation values ​​and being in contact with each other, and In the case where there are no markers in the grayscale image with a length equal to or greater than a predetermined length: Increase the value of the first pixel in the grayscale image that is less than the first average value, and Reduce the value of the second pixel in the grayscale image that is equal to or greater than the first average value.

8. The license plate character extraction device according to claim 6, wherein, The instructions cause the one or more processors to: At least one marker is generated in the grayscale image, the at least one marker comprising pixels having predetermined values ​​and being in contact with each other, and If there are markers in the grayscale image with a length equal to or greater than a predetermined length, the values ​​of pixels in the grayscale image that are equal to or less than the first average value are converted.

9. A non-transitory computer-readable storage medium comprising computer-executable instructions, wherein, The instructions, when executed by one or more processors, cause the one or more processors to: Receive input images captured by a camera device; Convert the input image to grayscale; A transformed image is generated based on the result of comparing the value of at least one pixel included in the grayscale image with the first average of the values ​​of the neighboring pixels; A thinned image is generated by comparing at least the transformed image with a binarized image obtained by binarizing the transformed image; Determine the character bounding boxes of the characters in the thinned image; Determine the degree of overlap between the determined character bounding box and the bounding boxes in a plurality of templates, each of the plurality of templates defining a different layout of the bounding box corresponding to the expected character position; as well as The character is extracted from the thinned image using the template that has the highest degree of overlap with the character's bounding box in the thinned image. The instructions cause the one or more processors to: Determine a second average value of the pixels with a preset value or a larger value included in a predetermined region of the converted image; and The binarized image is generated from the transformed image based on the result of comparing the values ​​of the pixels included in the transformed image with the second average value.

Images