Method, apparatus and system for controlling vehicle license plate using image style conversion and distortion correction

Abstract

One aspect of the present invention discloses a method for collecting vehicle number plates using image style conversion and distortion correction. The method comprises the steps of: extracting an image of a vehicle included in an original image; detecting coordinates for a license plate area of ​​the vehicle to extract a license plate image of the vehicle; performing a first preprocessing of the license plate image to convert the style of the license plate image; performing a second preprocessing of the license plate image after the first preprocessing is completed to generate a vehicle number plate image of the vehicle; and extracting a vehicle number plate of the vehicle based on the vehicle number plate image.

Description

Technology Field

[0001] The present invention relates to a method, apparatus, and system for collecting vehicle license plate numbers using image style conversion and distortion correction, and more specifically, to a method, apparatus, and system for improving the number recognition rate by unifying the style of vehicle license plate images and correcting distortion. Background Technology

[0002] Identifying license plate numbers is essential for identifying vehicles of interest in various fields, such as reporting traffic violations, criminal investigations and tracking suspect vehicles, and access or parking management. However, several problems arise when collecting license plate numbers in real-world environments, as follows.

[0003] First, it is difficult to apply consistent recognition algorithms to domestic license plates because their formats, such as color, font, and layout, are not standardized. For instance, the coexistence of various background colors (yellow, white, green, etc.), different fonts, and diverse layouts makes it difficult to process all types of license plates using a single recognition method.

[0004] Second, the shape of the license plate is distorted depending on the shooting position or angle. In real-world environments, vehicles are photographed from various angles rather than directly from the front, so license plates often become deformed or tilted due to perspective, hindering accurate recognition.

[0005] Third, motion blur occurs as the vehicle moves. In particular, when photographing vehicles traveling at high speeds, images frequently become blurry or license plate characters appear smudged, significantly reducing recognition rates.

[0006] To address these issues, conventional methods such as installing high-resolution cameras, recording after the vehicle stops, and shooting from a fixed angle have been used; however, these methods require additional cost and time and have limitations in real-time monitoring or application in various environments.

[0007] Furthermore, most existing license plate recognition systems are optimized for specific environments or conditions, making it difficult to respond to various problems arising in diverse situations. In particular, there is a lack of integrated solutions capable of handling the various forms of domestic license plates and environmental changes.

[0008] Therefore, there is a need for technology capable of recognizing license plates with high accuracy even in various types of license plates and shooting environments, and new technology is required to solve these problems. Prior art literature

[0009] Korean Registered Patent No. 10-2480582 The problem to be solved

[0010] The present invention aims to solve the aforementioned problems by providing a method, device, and system that improves the number recognition rate by unifying the style of vehicle license plate images and correcting distortion. means of solving the problem

[0011] According to one aspect of the present invention, a method for collecting vehicle number plates using image style conversion and distortion correction is provided, which is performed by a server. The method for collecting vehicle number plates using image style conversion and distortion correction comprises the steps of: extracting an image of a vehicle included in an original image; detecting coordinates for a license plate area of ​​the vehicle to extract a license plate image of the vehicle; performing a first preprocessing of the license plate image to convert the style of the license plate image; performing a second preprocessing of the license plate image after the first preprocessing is completed to generate a vehicle number plate image of the vehicle; and extracting a vehicle number plate of the vehicle based on the vehicle number plate image.

[0012] In one aspect, the step of converting the style includes separating the foreground and background of the license plate image using at least one of deep learning-based style transfer, image-to-image translation techniques and blob detection, and changing the color of the foreground and removing the background.

[0013] In another aspect, it further includes the step of unifying the color of the foreground, the font of the text included in the foreground, and the layout of the license plate included in the foreground into a single style.

[0014] In another aspect, the first preprocessing includes a step of removing distortion contained in the original image.

[0015] In another aspect, the step of removing the distortion includes removing the distortion using an image debugging technique or optical flow analysis.

[0016] In another aspect, the second preprocessing includes the steps of detecting a text area in the license plate image, filtering a false detection area included in the text area, correcting shape distortion of the text area, and generating the license plate image based on aligning the text area.

[0017] In another aspect, the step of detecting the text area includes the step of dividing a plurality of text areas from the license plate image using a region segmentation technique based on semantic segmentation or instance segmentation.

[0018] In another aspect, the step of filtering the false detection area includes filtering the false detection area based on analyzing at least one of the size of the text area, the ratio of the size of the license plate area to the size of the text area, and the aspect ratio of the text area.

[0019] In another aspect, the step of correcting the shape distortion includes the step of correcting the angle by utilizing inverse trigonometric functions after extracting any two points from the edge of the text area, correcting the angle by estimating an affine matrix after extracting any three points from the edge of the text area, or correcting the distortion by applying a perspective transform after estimating a homography using any four points from the edge of the text area.

[0020] In another aspect, the step of generating the vehicle number image based on aligning the text areas includes, when there are multiple text areas, aligning the coordinates of the multiple text areas based on the top-left corner, unifying the sizes of the multiple text areas, and connecting the multiple text areas in order from left to right to generate the vehicle number image.

[0021] In another aspect, the step of extracting the vehicle number includes the step of extracting the vehicle number using a text extraction technique based on scene text recognition, and the step of verifying the vehicle number based on the number of digits of the extracted vehicle number and the combination information of Korean characters and numbers.

[0022] In another aspect, the verification step includes verifying whether the number of digits of the extracted vehicle number and the combination of Korean characters and numbers comply with domestic vehicle number rules.

[0023] In another aspect, the above-mentioned scene text recognition-based text extraction technique can be performed based on at least one of Parseq, ABINet, and CRNN.

[0024] In another aspect, the step of extracting an image of the vehicle includes detecting an area containing the vehicle within the original image using a vehicle detection algorithm, and cropping the area containing the vehicle to extract the vehicle image.

[0025] In another aspect, prior to extracting the image of the vehicle, the method further includes the step of receiving the original image from an imaging device mounted on a moving vehicle, wherein the imaging device can capture and transmit the original image in real time.

[0026] In another aspect, the method further includes a step of matching and storing an image of the vehicle and the vehicle number of the vehicle, and further includes a step of generating tracking information for the vehicle based on repeating the matching and storing step at predetermined intervals.

[0027] According to another aspect of the present invention, a vehicle number collection device using image style conversion and distortion correction is provided. The vehicle number collection device using image style conversion and distortion correction includes a communication module that receives an original image and a processor that extracts an image of a vehicle included in the original image, detects coordinates for a license plate area of ​​the vehicle to extract a license plate image of the vehicle, performs a first preprocessing of the license plate image to convert the style of the license plate image, performs a second preprocessing of the license plate image after the first preprocessing is completed to generate a vehicle number image of the vehicle, and extracts a vehicle number of the vehicle based on the vehicle number image.

[0028] In one aspect, the processor includes a vehicle image collection unit that extracts an image of the vehicle from the original image, a license plate area detection unit that extracts a license plate image of the vehicle, a license plate image enhancement unit that performs the first preprocessing, a vehicle number image generation unit that performs the second preprocessing, and a vehicle number extraction unit that extracts the vehicle number. Effects of the invention

[0029] Embodiments of the present invention may have effects including the following advantages. However, since the embodiments of the present invention are not required to include all of these, the scope of the present invention should not be understood as being limited by them.

[0030] A method, device, and system for collecting vehicle license plates using image style conversion and distortion correction according to one embodiment of the present invention can maintain uniform detection performance regardless of the type and format of the license plate.

[0031] In addition, by unifying the various license plate styles into a single style, they become standardized, allowing for more universal use regardless of the specific country.

[0032] In addition, vehicle license plates can be accurately recognized even from video captured in real-time while driving, which can reduce the cost and time associated with patrol vehicles stopping.

[0033] In addition, it provides stable license plate recognition performance even in complex environments, allowing for efficient use in various fields such as reporting traffic violations, criminal investigations, and access or parking management. Brief explanation of the drawing

[0034] FIG. 1 is a schematic diagram showing a vehicle number collection system using image style conversion and distortion correction according to one embodiment of the present invention. Figure 2 is a drawing showing an example of a license plate image. Figure 3 is a diagram showing an example of extracting a license plate image from a vehicle image. Figure 4 is a diagram showing an example of distortion correction and style conversion. Figure 5 is a diagram showing an example of vehicle number image generation. Figure 6 is a diagram showing an example of a vehicle number rule. FIG. 7 is a flowchart of a method for collecting license plate numbers using image style conversion and distortion correction according to an embodiment of the present invention. Figure 8 is a detailed flowchart for step S750 of Figure 7. Figure 9 is a detailed flowchart for step S840 of Figure 8. FIG. 10 is a block diagram of a vehicle number collection device using image style conversion and distortion correction according to an embodiment of the present invention. Specific details for implementing the invention

[0035] The present invention is capable of various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail.

[0036] However, this is not intended to limit the invention to specific embodiments, and it should be understood that it includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention.

[0037] Terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0038] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0039] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to specify the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0040] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0041] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. In order to facilitate an overall understanding of the present invention, the same reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0042] FIG. 1 is a schematic diagram showing a vehicle number collection system using image style conversion and distortion correction according to one embodiment of the present invention.

[0043] Referring to FIG. 1, a vehicle number collection system (100) using image style conversion and distortion correction according to one embodiment of the present invention is a system that identifies a vehicle's license plate based on an original image transmitted in real time from a video device mounted on a moving vehicle, and can generate information for monitoring the location and movement of the vehicle based on the identified license plate and the image of the vehicle. In particular, it can generate tracking information for efficiently tracking specific targets, such as vehicles violating traffic laws, vehicles reported by citizens, or vehicles associated with crime.

[0044] A vehicle number collection system (100) utilizing such image style conversion and distortion correction includes an image device (110), a vehicle number collection server (120), and a database (130).

[0045] The video device (110) is a device mounted on a vehicle to capture images of the surroundings. For example, the video device (110) may be mounted on a police car or a patrol car and may capture images of vehicles parked or moving on the road around the vehicle. Images captured by the video device (110) may be referred to as original images. The video device (110) may transmit the captured original images to a vehicle number collection server (120). The video device (110) may perform the shooting and transmission of original images in real time. The video device (110) may preferably be implemented as a black box mounted on the vehicle, a smart terminal, PC, laptop, etc., of a user riding in the vehicle.

[0046] The vehicle number collection server (120) is a device that recognizes a vehicle's license plate based on an original image received from an image device (110). The vehicle number collection server (120) can extract an image of a vehicle from the original image and detect a license plate area in the vehicle image to extract a license plate image. The vehicle number collection server (120) can generate a vehicle number image in which distortion is corrected and the license plate style is unified by performing a first preprocessing and a second preprocessing of the license plate image. Then, it can extract the number of the corresponding vehicle based on the vehicle number image. The vehicle number collection server (120) can transmit the extracted vehicle number and the vehicle image to a database (130) for storage. The vehicle number collection server (120) can match and store the vehicle image and the vehicle number.

[0047] The vehicle number collection server (120) can be implemented as a large-capacity PC, workstation, server PC, etc.

[0048] The database (130) is a device that stores information transmitted from the vehicle number collection server (120). The database (130) can receive information from the vehicle number collection server (120) that matches an image of a vehicle with the vehicle number of the vehicle. This information can be used as tracking information to track the vehicle. The database (130) may be implemented as a separate network storage or as a storage device of the vehicle number collection server (120).

[0049] Figure 2 is a drawing showing an example of a license plate image.

[0050] Figure 2(a) shows motion blur that occurs when capturing an image while the vehicle is moving, Figure 2(b) shows shape distortion that occurs depending on the shooting position or angle, and Figure 2(c) shows different types of license plates with different layouts and colors depending on the type or purpose of the vehicle.

[0051] Referring to (a) to (c) of FIG. 2, when a general text recognition model or license plate recognition model is applied, it may be difficult to accurately recognize the number. For example, as in (a), text recognition errors may occur when the text is captured blurry, and as in (b), text recognition errors may also occur when the license plate is tilted or distorted by perspective. Furthermore, as in (c), when the license plate format is not standardized, the algorithm must perform additional operations (e.g., 1-line operation -> 2-line operation) to enable number recognition, which may lead to an increase in computational resources.

[0052] Figure 3 is a diagram showing an example of extracting a license plate image from a vehicle image.

[0053] Referring to FIG. 3, the vehicle number collection server (120) can extract an image of a vehicle included in the original image. The extraction operation for the vehicle image can be performed through an existing artificial intelligence-based object detection model (vehicle detection algorithm). For example, the original image can be input into an object detection model that has been trained in advance to detect vehicles to detect an area containing a vehicle within the original image, and the area containing the vehicle can be cropped to extract the vehicle image. At this time, the background or other objects around the vehicle can be dropped as they are information not used for vehicle number recognition.

[0054] Meanwhile, the vehicle number collection server (120) can extract a license plate image from the extracted vehicle image. For example, the vehicle number collection server (120) can detect the coordinates of the vehicle's license plate area within the bounding box of the vehicle, and then extract the vehicle's license plate image based on the coordinates. The operation of extracting the license plate image can also be performed using an object detection-based object of interest detection algorithm. The example in FIG. 3 shows a patrol vehicle equipped with an image device (110) moving along with a target vehicle traveling in front, and the license plate image is extracted by analyzing the original image taken from the rear by the vehicle number collection server (120); however, an image taken of the front of the target vehicle can also be used.

[0055] Figure 4 is a diagram showing an example of distortion correction and style conversion.

[0056] Figures 4 (a) and (b) show a detailed first preprocessing step for the license plate image of Figure 3, where Figure 4 (a) shows the correction of shaking distortion and Figure 4 (b) shows the conversion of the license plate style.

[0057] First, referring to FIG. 4(a), distortion in the image may be caused by vibrations of a moving vehicle or road conditions, and as the image is captured, shaking distortion may occur, resulting in unclear characters on the license plate. Such shaking distortion mainly occurs when capturing a vehicle moving at high speed and is a major factor that hinders accurate number recognition. The vehicle number collection server (120) can remove distortion using various types of image debugging techniques or optical flow analysis. More specifically, to effectively remove distortion, advanced image processing techniques such as Multi-Scale CNN-based debugging, Scale-Recurrent Network, and optical flow-based shaking distortion removal techniques can be applied.

[0058] Multi-Scale CNN-based Deblurring utilizes convolutional neural networks of various scales to restore image details step by step, making it particularly effective for license plates with complex patterns. This method leverages an image pyramid structure to extract shake information at multiple resolutions and progressively restore the image to a sharp state.

[0059] The Scale-Recurrent Network utilizes a recurrent neural network structure to encode shake information at various image scales and integrate them to generate sharper images. This network can provide consistent restoration performance through a memory function that uses the processing results of the previous scale as input for the next scale.

[0060] Optical Flow-based shake distortion removal techniques analyze pixel movement vectors between consecutive frames to identify the direction and intensity of shake, and can restore the original image by correcting it in the opposite direction. This method is particularly effective for motion shake occurring in a specific direction.

[0061] By applying various types of techniques like this, shaking distortion can be effectively removed, allowing the numbers and letters on the license plate to be restored clearly.

[0062] Next, referring to Figure 4(b), since the probability of error may increase when recognizing original license plate images with a mixture of various background colors (yellow, white, green, etc.), border styles, and text fonts as they are, at least one of deep learning-based Style Transfer, Image-to-Image Translation, and Blob Detection can be used to convert license plates of various styles into a unified single style.

[0063] Deep learning-based Style Transfer is a technology that converts only the style while maintaining the content of the original license plate. In particular, by utilizing a GAN (Generative Adversarial Network)-based model, it can learn various styles of license plates and convert them into a common style.

[0064] Image-to-Image Translation is a technology that converts images from one domain to another. By utilizing algorithms such as pix2pix or CycleGAN, it can transform various license plate styles into a target style. This method effectively converts visual characteristics while preserving the structural properties of the original image.

[0065] The Blob Detection-based foreground / background separation technique clearly separates the foreground (license plate) and background (area excluding the license plate) from an image, then removes the background to extract only the text. The color of the foreground, the font of the text included in the foreground, and the layout of the license plate included in the foreground can be unified into a single style, which can be particularly effective for license plates with clear contrast between the text and the background. Through such distortion correction and style conversion processes, license plates captured under various conditions that were difficult to process with conventional methods can be effectively normalized, thereby improving the license plate recognition rate.

[0066] Figure 5 is a diagram showing an example of vehicle number image generation.

[0067] Referring to FIG. 5, a license plate image that has been preprocessed in the first stage through the process of FIG. 4 can be preprocessed in the second stage to generate a license plate image to be used for recognizing the actual license plate number. The second stage preprocessing operation of the license plate number collection server (120) can be divided into the operation of detecting text areas in the license plate image, filtering out false detection areas included in the text areas, correcting shape distortion of the text areas, and aligning the text areas to generate a license plate image.

[0068] First, the operation of detecting text regions can be performed by segmenting multiple text regions from license plate images using region segmentation techniques based on Semantic Segmentation or Instance Segmentation. Semantic Segmentation techniques perform classification at the image pixel level, enabling accurate distinction between text and non-text regions. In particular, advanced segmentation architectures such as Fully Convolutional Networks (FCN), U-Net, and DeepLab can be utilized; these networks effectively leverage image contextual information to accurately identify the boundaries of text regions. Instance Segmentation techniques can detect multiple independent text elements (region names, classification numbers, serial numbers, etc.) within a license plate by classifying them as separate instances. Algorithms such as Mask R-CNN or YOLACT can be used for this purpose, and they can accurately separate even overlapping or adjacent text regions.

[0069] In one embodiment, the vehicle number collection server (120) can accurately identify the text areas of the two parts "XX12" and "Y 0000" as bounding boxes, as shown in the upper left license plate image of FIG. 5. In this process, non-text areas such as license plate borders or decorative elements are ignored, and only the text area containing actual vehicle number information can be selectively extracted.

[0070] Next, the operation of filtering false detection areas can be performed based on analyzing at least one of the size of the text area, the ratio of the license plate area to the text area size, and the aspect ratio of the text area. Based on the analysis of the text area size, areas that fall outside the typical size range of actual license plate characters can be filtered; in this case, areas that are too small or too large may be identified as noise or other elements of the license plate. Based on the analysis of the ratio of the license plate area to the text area size, it can be verified whether the ratio occupied by the text area relative to the total license plate size is within a certain range. Based on the analysis of the text area aspect ratio, it can be verified whether the width-to-height ratio of each character or number block falls within the typical aspect ratio range of license plate characters. This can be particularly effective in removing false detections that are elongated bar-shaped or close to a square.

[0071] The vehicle number collection server (120) can filter out unnecessary elements while leaving only meaningful vehicle number components, as shown in the second image from the top left of FIG. 5, by filtering out the false detection area. That is, it can remove the rest while leaving only "XX12" and "Y 0000" corresponding to the actual vehicle number.

[0072] Next, the action for correcting shape distortion may be an action that corrects the angle, such as an image taken from the front, using multiple arbitrarily selected points (coordinates). For example, correction can be performed by extracting and correcting two arbitrary points from the edges of the text area, or by extracting and correcting three or four points.

[0073] Two-point based angle correction is a method that extracts two random points from the edges of a text area, calculates the tilt angle using an inverse trigonometric function (arctan), and then applies a horizontal rotational transformation. This can be effective for simple rotational distortions. Three-point based angle correction is a method that extracts three random points from the edges of a text area, calculates an affine transformation matrix, and applies it to correct distortions such as scaling, rotation, and skewness in a single step. This can be effective for correcting text areas that are distorted in a specific direction or are slanted. Four-point based angle correction is a method that extracts four random points from the edges of a text area, calculates a Homography matrix, and applies it to correct complex distortions caused by perspective. This can be most effective for correcting text areas that are distorted in different directions or have strong perspective effects.

[0074] The vehicle number collection server (120) can horizontally align slanted or distorted text areas and normalize their shape through distortion correction. That is, it can correctly align both text blocks of "XX12" and "Y 0000" into a rectangular shape.

[0075] Next, the operation of aligning text areas to generate a license plate image is an operation of integrating the aligned text areas to generate a final license plate image. As illustrated in FIG. 5, there may be multiple text areas, and the license plate collection server (120) can align and merge multiple text areas into a single final image. For example, the text may be aligned based on the top-left corner, which is the same as the reading direction in Korea, but the alignment criteria can be changed at any time by the settings of the license plate collection server (120). When generating a license plate image, the license plate collection server (120) can standardize the size of each text area and adjust it to a consistent ratio. That is, text areas of different sizes can be aligned to a uniform scale. Then, the aligned text areas can be concatenated in order from the left to generate a single integrated license plate image. At this time, depending on the settings, an appropriate spacing can be placed between each merged text area. The license plate collection server (120) can generate a final license plate image "XX12 Y 0000" in which all text areas are aligned and integrated according to the process described above.

[0076] Figure 6 is a diagram showing an example of a vehicle number rule.

[0077] Figure 6 shows examples of invalid license plates for various reasons. For instance, if the image is captured with part of the license plate obscured as in the top left, obscured by foreign matter as in the top right, impossible to identify the number due to shading as in the bottom left, or impossible to identify the number because it was captured under too low a light level as in the bottom right, the original image itself is defective, and thus the success rate of number recognition may also be lowered. Therefore, there is a need to verify whether the extracted license plate is actually valid.

[0078] Referring again to FIG. 6, the vehicle number collection server (120) can extract a vehicle number using a scene text recognition-based text extraction technique (Parseq, ABINet, CRNN, etc.) on a vehicle number image generated according to the process of FIG. 5. The vehicle number collection server (120) can further perform a process of verifying whether the extracted vehicle number conforms to domestic vehicle number rules, such as the number of digits and the combination of Korean characters and numbers. For example, verification can be performed by storing information on domestic vehicle number writing rules (which may be a template or a sample) in advance and then comparing whether the extracted vehicle number matches one of the stored writing rules.

[0079] FIG. 7 is a flowchart of a method for collecting license plate numbers using image style conversion and distortion correction according to an embodiment of the present invention.

[0080] Referring to FIG. 7, the method for collecting license plate numbers using image style conversion and distortion correction includes steps S710 to S760. The method for collecting license plate numbers using image style conversion and distortion correction can be performed by the operation of the license plate number collection server (120) of FIG. 1.

[0081] In step S710, the vehicle number collection server (120) receives the original image of the vehicle captured from the video device.

[0082] In step S720, the vehicle number collection server (120) extracts an image of a vehicle included in the original image. For example, a vehicle detection algorithm can be used to detect an area containing a vehicle within the original image, and the vehicle image can be extracted by cropping the area.

[0083] In step S730, the vehicle number collection server (120) detects the coordinates of the vehicle's license plate area and extracts the vehicle's license plate image.

[0084] In step S740, the vehicle number collection server (120) performs a first preprocessing of the license plate image to convert the style of the license plate image. The first preprocessing includes a process of removing distortions contained in the original image and a process of unifying the style of the license plate.

[0085] In step S750, the vehicle number collection server (120) generates a vehicle number image by performing a second preprocessing on a license plate image that has undergone first preprocessing. The second preprocessing includes a process of detecting text areas in the license plate image, a process of filtering out false detection areas, a process of correcting shape distortion, and a process of aligning multiple text areas to generate a vehicle number image.

[0086] In step S760, the vehicle number collection server (120) extracts the vehicle number based on the vehicle number image.

[0087] Figure 8 is a detailed flowchart for step S750 of Figure 7.

[0088] Referring to FIG. 8, step S750 of FIG. 7 can be subdivided into steps S810 to S840. Steps S810 to S840 can be performed by the operation of the vehicle number collection server (120).

[0089] In step S810, the vehicle number collection server (120) detects a text area in the license plate image.

[0090] In step S820, the vehicle number collection server (120) filters out the false detection area included in the text area.

[0091] In step S830, the vehicle number collection server (120) corrects the shape distortion of the text area.

[0092] In step S840, the vehicle number collection server (120) aligns the text area to generate a vehicle number image.

[0093] Figure 9 is a detailed flowchart for step S840 of Figure 8.

[0094] Referring to FIG. 9, step S840 of FIG. 8 can be subdivided into steps S910 to S940. Steps S910 to S940 can be performed by the operation of the vehicle number collection server (120).

[0095] In step S910, the vehicle number collection server (120) determines whether there are multiple text areas. If there are not multiple text areas, steps S910 through S940 can be skipped and step S760 can be performed.

[0096] In step S920, the vehicle number collection server (120) aligns the coordinates of multiple text areas based on the top-left corner.

[0097] In step S930, the vehicle number collection server (120) unifies the sizes of multiple text areas.

[0098] In step S940, the vehicle number collection server (120) generates a vehicle number image by connecting multiple text areas in order from left to right.

[0099] FIG. 10 is a block diagram of a vehicle number collection device using image style conversion and distortion correction according to an embodiment of the present invention.

[0100] Referring to FIG. 10, a vehicle number collection device (1000) using image style conversion and distortion correction includes a communication module (1010), a memory (1020), and a processor (1030). The vehicle number collection device (1000) using image style conversion and distortion correction can be implemented as the vehicle number collection server (120) of FIG. 1.

[0101] The communication module (1010) receives an original image from the image device (110). The communication module (1010) can transmit tracking information stored by matching the image of the vehicle with the vehicle number under the control of the processor (1030) to the database (130). The communication module (1010) can be implemented as a device capable of communicating with the image device (110) or the database (130).

[0102] The memory (1020) stores a set of instructions associated with vehicle number identification.

[0103] The processor (1030) extracts an image of a vehicle included in the original image and detects coordinates for the license plate area of ​​the vehicle to extract an image of the vehicle's license plate. The processor (1030) generates an image of the vehicle's license plate number by performing a first preprocessing and a second preprocessing of the license plate image. The processor (1030) extracts the vehicle's license plate number based on the license plate number image. The processor (1030) may be implemented as a CPU, GPU, or AP.

[0104] Meanwhile, the processor (1030) may be divided into a license plate area extraction unit (1030a), a license plate image improvement unit (1030b), a vehicle number image generation unit (1030c), and a vehicle number extraction unit (1030d) according to its function.

[0105] The license plate area extraction unit (1030a) extracts an image of a vehicle from an original image and detects a license plate area in the vehicle image to extract a license plate image.

[0106] The license plate image enhancement unit (1030b) is responsible for primary preprocessing, removing distortion from the license plate image and converting the style.

[0107] The vehicle number image generation unit (1030c) is responsible for secondary preprocessing and generates a vehicle number image by performing tasks such as text area detection, false detection area filtering, shape distortion correction, and area alignment.

[0108] The vehicle number extraction unit (1030d) extracts a vehicle number from a generated vehicle number image and verifies whether the extracted vehicle number is valid.

[0109] Meanwhile, the vehicle number collection device (1000) using the image style conversion and distortion correction described above can be implemented as computer-readable code on a computer-readable recording medium. Computer-readable recording media include all types of recording media in which data that can be decoded by a computer system is stored. For example, ROM (Read Only Memory), RAM (Random Access Memory), magnetic tape, magnetic disk, flash memory, optical data storage device, etc., may be used. Additionally, computer-readable recording media can be distributed to computer systems connected via a computer network and can be stored and executed as code that can be read in a distributed manner.

[0110] Although the invention has been described with reference to the drawings and embodiments, this does not imply that the scope of protection of the present invention is limited by the drawings or embodiments. Those skilled in the art will understand that various modifications and changes can be made to the present invention without departing from the spirit and scope of the invention as described in the following claims.

Claims

Claim 1 A method for collecting vehicle number plates using image style conversion and distortion correction, performed by a server, comprising: a step of extracting an image of a vehicle included in an original image; a step of detecting coordinates for a license plate area of ​​the vehicle to extract a license plate image of the vehicle; a step of first preprocessing the license plate image to convert the style of the license plate image; a step of second preprocessing the license plate image after the first preprocessing is completed to generate a vehicle number plate image of the vehicle; and a step of extracting a vehicle number plate of the vehicle based on the vehicle number plate image. Claim 2 A method for collecting license plate numbers using image style conversion and distortion correction, wherein the step of converting the style comprises: a step of separating the foreground and background of the license plate image using at least one of deep learning-based style transfer, image-to-image translation technique and blob detection; and a step of changing the color of the foreground and removing the background. Claim 3 A method for collecting vehicle numbers using image style conversion and distortion correction, further comprising the step of unifying the color of the foreground, the font of the text included in the foreground, and the layout of the number plate included in the foreground into a single style in claim 2. Claim 4 A method for collecting license plate numbers using image style conversion and distortion correction, wherein the first preprocessing includes a step of removing distortion included in the original image. Claim 5 A method for collecting license plate numbers using image style conversion and distortion correction, wherein the step of removing the distortion in claim 4 includes the step of removing the distortion using an image debugging technique or optical flow analysis. Claim 6 A method for collecting license plate numbers using image style conversion and distortion correction, wherein the secondary preprocessing comprises: a step of detecting a text area in the license plate image; a step of filtering a false detection area included in the text area; a step of correcting shape distortion of the text area; and a step of generating the license plate number image based on aligning the text area. Claim 7 A method for collecting vehicle numbers using image style conversion and distortion correction, wherein the step of detecting the text area in claim 6 includes the step of dividing a plurality of text areas from the license plate image using a region segmentation technique based on semantic segmentation or instance segmentation. Claim 8 A method for collecting license plate numbers using image style conversion and distortion correction, wherein the step of filtering the false detection area in claim 6 comprises the step of filtering the false detection area based on analyzing at least one of the size of the text area, the ratio of the license plate area to the size of the text area, and the aspect ratio of the text area. Claim 9 In claim 6, the step of correcting the shape distortion comprises the step of correcting the angle by utilizing an inverse trigonometric function after extracting any two points from the edge of the text area, correcting the angle by estimating an affine matrix after extracting any three points from the edge of the text area, or correcting the distortion by applying a perspective transform after estimating a homography using any four points from the edge of the text area. Claim 10 In claim 6, the step of generating the vehicle number image based on aligning the text areas comprises, when there are multiple text areas, aligning the coordinates of the multiple text areas based on the top-left corner; unifying the sizes of the multiple text areas; and connecting the multiple text areas in order from the left to generate the vehicle number image, a method for collecting vehicle numbers using image style conversion and distortion correction. Claim 11 A method for collecting vehicle numbers using image style conversion and distortion correction, wherein the step of extracting the vehicle number comprises: a step of extracting the vehicle number using a text extraction technique based on scene text recognition; and a step of verifying the vehicle number based on the number of digits of the extracted vehicle number and combination information of Korean characters and numbers. Claim 12 A method for collecting vehicle numbers using image style conversion and distortion correction, wherein the verification step includes verifying whether the number of digits of the extracted vehicle number and the combination of Korean characters and numbers conform to domestic vehicle number rules. Claim 13 In claim 11, the text extraction technique based on the Scene text recognition is performed based on at least one of Parseq, ABINet, and CRNN, a method for collecting license plate numbers using image style conversion and distortion correction. Claim 14 A method for collecting vehicle numbers using image style conversion and distortion correction, wherein the step of extracting an image of the vehicle comprises detecting an area containing the vehicle within the original image using a vehicle detection algorithm, and cropping the area containing the vehicle to extract the vehicle image. Claim 15 A method for collecting vehicle numbers using image style conversion and distortion correction, wherein, in claim 1, prior to extracting an image of the vehicle, the method further includes the step of receiving the original image from an imaging device mounted on a moving vehicle, wherein the imaging device captures and transmits the original image in real time. Claim 16 A method for collecting license plate numbers using image style conversion and distortion correction according to claim 1, further comprising the step of matching and storing an image of the vehicle and a license plate number of the vehicle, and further comprising the step of generating tracking information for the vehicle based on repeating the matching and storing step at preset intervals. Claim 17 A vehicle number collection device using image style conversion and distortion correction, comprising: a communication module receiving an original image; and a processor that extracts an image of a vehicle included in the original image, detects coordinates for a license plate area of ​​the vehicle to extract a license plate image of the vehicle, performs a first preprocessing of the license plate image to convert the style of the license plate image, performs a second preprocessing of the license plate image after the first preprocessing is completed to generate a vehicle number image of the vehicle, and extracts a vehicle number of the vehicle based on the vehicle number image. Claim 18 In claim 17, the processor comprises: a vehicle image collection unit that extracts an image of the vehicle from the original image; a license plate area detection unit that extracts a license plate image of the vehicle; a license plate image enhancement unit that performs the first preprocessing; a vehicle number image generation unit that performs the second preprocessing; and a vehicle number extraction unit that extracts the vehicle number, a vehicle number collection device using image style conversion and distortion correction.

Images