Method for generating high-quality license plate image using image restoration model and electronic device performing same

The method generates high-definition license plate images by synthesizing cropped images and using optical flow and number recognition models to address low-resolution challenges, enhancing visibility and recognition accuracy.

WO2026235269A1PCT designated stage Publication Date: 2026-11-12NC& CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/KR2026/002301
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-08
Filing Date
2026-02-06
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Existing license plate recognition models perform poorly with low-resolution images due to issues like degradation and vehicle movement, making it difficult to restore high-quality images for accurate recognition.

Method used

An image generation method involving generating cropped images of a license plate area, synthesizing them into a single high-resolution image, and using optical flow and number recognition models to enhance visibility and recognition performance.

Benefits of technology

The method produces high-definition restored license plate images with improved visibility and recognition accuracy, enabling accurate vehicle number extraction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure KR2026002301_12112026_PF_FP_ABST
    Figure KR2026002301_12112026_PF_FP_ABST
Patent Text Reader

Abstract

This method for generating a license plate image of a vehicle, performed by an electronic device, may comprise the steps of: generating a plurality of cropped images comprising a license plate area of a target vehicle, on the basis of a plurality of basic images of a video in which the target vehicle is captured; generating a first license plate image by synthesizing at least some of the plurality of cropped images into a single image; determining one or more candidate number sets for a numeric string appearing on the first license plate image; and generating one or more restored license plate images respectively corresponding to the one or more candidate number sets, on the basis of the first license plate image and the one or more candidate number sets.
Need to check novelty before this filing date? Find Prior Art

Description

Method for generating high-definition license plate images using an image restoration model and electronic device for performing the same

[0001] The following embodiments relate to an image generation method and an electronic device for performing the same, specifically to a method for restoring an image showing a vehicle license plate and an electronic device for performing the same.

[0002] License plate recognition has established itself as an essential technology in many sectors of society and various industries, including traffic compliance and surveillance, security, and crime prevention. However, the performance of license plate recognition models drops significantly in the case of low-resolution video.

[0003] To resolve this issue, it is necessary to convert low-resolution license plate images into high-resolution ones. However, in this case, it is difficult to properly restore the license plate due to problems such as degradation of the black box and the continuous movement of the vehicle and camera.

[0004] There is a need to provide a method for generating high-quality restored images and an electronic device for performing this, so that visibility and recognition performance are guaranteed.

[0005] One embodiment can provide a method for generating a high-quality restored image based on an image showing a vehicle's license plate.

[0006] One embodiment may provide an electronic device that generates a high-definition restored image based on an image showing a vehicle's license plate.

[0007] However, technical challenges are not limited to the technical challenges described above, and other technical challenges may exist.

[0008] A method for generating a license plate image of a vehicle, performed by an electronic device according to one embodiment, may include: generating a plurality of cropped images including a license plate area of ​​a target vehicle based on a plurality of basic images of a video of a target vehicle; generating a first license plate image by synthesizing at least some of the plurality of cropped images into a single image; determining one or more sets of candidate numbers for a sequence of numbers appearing on the first license plate image; and generating one or more restored license plate images corresponding to each of the one or more sets of candidate numbers based on the first license plate image and the one or more sets of candidate numbers.

[0009] The operation of determining one or more candidate number sets may include the operation of determining a candidate number group for each number in a sequence of numbers appearing on the first number plate image by processing the first number plate image using a number recognition model, and the operation of determining one or more candidate number sets based on the candidate number group determined for each number.

[0010] The operation of determining one or more sets of candidate numbers based on the candidate number group determined for each of the above numbers may include the operation of determining the reliability of each candidate number within the first candidate number group determined for the first number, the operation of excluding the first candidate number from the first candidate number group when the reliability of the first candidate number within the first candidate number group is lower than a preset threshold, and the operation of determining one or more sets of candidate numbers based on the first candidate number group for the first number and the second candidate number group for the second number.

[0011] The operation of determining the candidate number group for each number of the number sequence may include the operation of obtaining license plate type information based on the first license plate image and the operation of determining the candidate number group for each number of the number sequence appearing on the first license plate image based on the license plate type information.

[0012] The operation of generating one or more restored license plate images corresponding to each of the one or more candidate number sets may include the operation of obtaining a first image feature map for the first license plate image, the operation of obtaining a first number feature vector for the first candidate number set among the one or more candidate number sets, and the operation of obtaining a first restored license plate image in which the first license plate image is restored based on the first candidate number set by processing the first image feature map and the first number feature vector using a diffusion model.

[0013] The operation of generating the first license plate image by synthesizing at least some of the plurality of cropped images into a single image may include the operation of determining a reference cropped image among the plurality of cropped images, the operation of calculating a plurality of optical flow values ​​between the reference cropped image and each of the plurality of cropped images, the operation of determining a plurality of alignment images among the plurality of cropped images based on the plurality of optical flow values, and the operation of generating the first license plate image based on the plurality of alignment images.

[0014] The above reference crop image can be determined as any one of the plurality of crop images based on the image generation time associated with each of the plurality of crop images.

[0015] The operation of determining the plurality of alignment images may include the operation of determining an error image among the plurality of crop images based on the plurality of optical flow values, and the operation of determining the plurality of alignment images based on the error image.

[0016] The operation of generating the first number plate image may include the operation of determining a first alignment image requiring correction among the plurality of alignment images, the operation of generating the corrected first alignment image by correcting the first alignment image, and the operation of generating the first number plate image based on the plurality of alignment images including the corrected first alignment image.

[0017] The operation of generating the first license plate image may include the operation of obtaining intensity values ​​of pixels included in the license plate area of ​​each of the plurality of alignment images, the operation of determining pixels corresponding to a preset intensity value among the obtained intensity values, and the operation of generating the first license plate image based on the determined pixels.

[0018] The method for generating a license plate image of the vehicle may further include an operation of determining the vehicle number of the target vehicle based on one or more restored license plate images.

[0019] The operation of determining the vehicle number of the target vehicle based on the one or more restored license plate images may include: determining a first sequence of numbers appearing on the first restored license plate image by processing the first restored license plate image among the one or more restored license plate images using a number recognition model; determining a second sequence of numbers appearing on the second restored license plate image by processing the second restored license plate image among the one or more restored license plate images using the number recognition model; and determining the first sequence of numbers as the vehicle number of the target vehicle when the sequences of numbers determined based on each of the one or more restored license plate images all match.

[0020] According to one embodiment, an electronic device comprises at least one processor including processing circuitry and a memory including one or more storage media for storing instructions, and when the instructions are executed individually or collectively by the at least one processor, the electronic device may be configured to: generate a plurality of cropped images including a license plate area of ​​a target vehicle based on a plurality of basic images of a video of a target vehicle, generate a first license plate image by synthesizing at least some of the plurality of cropped images into a single image, determine one or more sets of candidate numbers for a sequence of numbers appearing on the first license plate image, and generate one or more restored license plate images corresponding to each of the one or more sets of candidate numbers based on the first license plate image and the one or more sets of candidate numbers.

[0021] A method for generating a license plate image of a vehicle, performed by an electronic device according to one embodiment, may include the operation of acquiring a second license plate image in which a license plate of a target vehicle appears; the operation of determining one or more sets of candidate numbers for a sequence of numbers appearing on the second license plate image; the operation of generating one or more restored license plate images corresponding to each of the one or more sets of candidate numbers based on the second license plate image and the one or more sets of candidate numbers; and the operation of determining a vehicle number of the target vehicle based on the one or more restored license plate images.

[0022] The operation of determining one or more candidate number sets may include the operation of determining a candidate number group for each number in a sequence of numbers appearing on the second number plate image by processing the second number plate image using a number recognition model, and the operation of determining one or more candidate number sets based on the candidate number group determined for each number.

[0023] The operation of determining one or more sets of candidate numbers based on the candidate number group determined for each of the above numbers may include the operation of determining the reliability of each candidate number within the first candidate number group determined for the first number, the operation of excluding the first candidate number from the first candidate number group when the reliability of the first candidate number within the first candidate number group is lower than a preset threshold, and the operation of determining one or more sets of candidate numbers based on the first candidate number group for the first number and the second candidate number group for the second number.

[0024] The operation of determining the candidate number group for each number of the number sequence may include the operation of obtaining license plate type information based on the second license plate image and the operation of determining the candidate number group for each number of the number sequence appearing on the second license plate image based on the license plate type information.

[0025] The operation of generating one or more restored license plate images corresponding to each of the one or more candidate number sets may include the operation of obtaining a second image feature map for the second license plate image, the operation of obtaining a first number feature vector for a first candidate number set among the one or more candidate number sets, and the operation of obtaining a first restored license plate image in which the second license plate image is restored based on the first candidate number set by processing the second image feature map and the first number feature vector using a diffusion model.

[0026] The operation of acquiring the second license plate image in which the license plate of the target vehicle appears may include the operation of generating a plurality of cropped images including a license plate area of ​​the target vehicle based on a plurality of basic images of a video in which the target vehicle is filmed, and the operation of generating the second license plate image by synthesizing at least some of the plurality of cropped images into a single image.

[0027] The operation of generating the second license plate image by synthesizing at least some of the plurality of cropped images into a single image may include the operation of determining a reference cropped image among the plurality of cropped images, the operation of calculating a plurality of optical flow values ​​between the reference cropped image and each of the plurality of cropped images, the operation of determining a plurality of alignment images among the plurality of cropped images based on the plurality of optical flow values, and the operation of generating the second license plate image based on the plurality of alignment images.

[0028] The operation of generating the second license plate image may include the operation of determining a first alignment image requiring correction among the plurality of alignment images, the operation of generating the corrected first alignment image by correcting the first alignment image, and the operation of generating the second license plate image based on the plurality of alignment images including the corrected first alignment image.

[0029] The operation of generating the second license plate image may include the operation of obtaining intensity values ​​of pixels included in the license plate area of ​​each of the plurality of alignment images, the operation of determining pixels corresponding to a preset intensity value among the obtained intensity values, and the operation of generating the second license plate image based on the determined pixels.

[0030] FIG. 1 illustrates an electronic device and a target vehicle according to one example.

[0031] FIG. 2 is a drawing showing a system of an electronic device according to one embodiment.

[0032] FIG. 3 is a flowchart of an image generation method according to one embodiment.

[0033] FIG. 4 is a flowchart illustrating the operation of generating cropped images according to one embodiment.

[0034] FIG. 5 is a drawing illustrating cropped images and a reference cropped image according to one example.

[0035] FIG. 6 is a flowchart illustrating the operation of generating a license plate image based on a plurality of cropped images according to one embodiment.

[0036] FIG. 7 is a flowchart illustrating the operation of calculating optical flow values ​​according to one embodiment.

[0037] FIGS. 8A and FIGS. 8B are diagrams illustrating the operation of calculating optical flow values ​​according to one example.

[0038] FIG. 9 is a flowchart illustrating the operation of determining alignment images according to one embodiment.

[0039] FIG. 10 is a flowchart illustrating the operation of determining an error image according to one embodiment.

[0040] FIG. 11 is a flowchart illustrating the operation of generating a license plate image based on a plurality of alignment images according to one embodiment.

[0041] FIG. 12 is a flowchart illustrating the operation of determining a first alignment image according to one embodiment.

[0042] FIG. 13 is a flowchart illustrating the operation of generating a corrected first alignment image according to one embodiment.

[0043] FIG. 14 is a diagram illustrating the operation of correcting a first alignment image according to one example.

[0044] FIG. 15 is a flowchart illustrating the operation of generating a license plate image based on a plurality of alignment images according to one embodiment.

[0045] FIG. 16 is a flowchart illustrating the operation of determining one or more sets of candidate numbers according to one embodiment.

[0046] FIG. 17 is a flowchart illustrating the operation of determining a group of candidate numbers according to one embodiment.

[0047] FIG. 18 is a flowchart illustrating the operation of determining one or more sets of candidate numbers based on groups of candidate numbers according to one embodiment.

[0048] FIG. 19 is a diagram illustrating the operation of determining one or more sets of candidate numbers according to one example.

[0049] FIG. 20 is a flowchart illustrating the operation of acquiring a restored license plate image according to one embodiment.

[0050] FIG. 21 is a diagram illustrating the operation of acquiring a restored license plate image according to one example.

[0051] FIG. 22 is a flowchart illustrating the operation of determining the vehicle number of a target vehicle according to one embodiment.

[0052] FIG. 23 is a flowchart illustrating the operation of determining the vehicle number of a target vehicle based on sequences of numbers appearing on restored license plate images according to one embodiment.

[0053] FIG. 24 is a flowchart of an image generation method according to one embodiment.

[0054] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, actual implementations are not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or substitutions included in the technical concept described by the embodiments.

[0055] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0056] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or coupled with that other component, or that there may be other components in between.

[0057] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to specify the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0058] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0059] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0060] FIG. 1 illustrates an electronic device and a target vehicle according to one example.

[0061] Referring to FIG. 1, a target vehicle (1), a license plate area (12) of the target vehicle, and an electronic device (10) (e.g., a black box) according to one example are illustrated. In one example, the electronic device (10) may include a display module, a sensor, and a camera module. In one example, the sensor may be an inertial sensor for measuring changes in acceleration and a speed sensor for determining the speed of the vehicle. The camera module may capture the surroundings of the vehicle. The electronic device (10) may generate one or more frames of the surroundings of the vehicle (e.g., the target vehicle (1)) based on the camera module. The electronic device (10) may generate a video of the surroundings of the vehicle based on one or more frames.

[0062] In one embodiment, the electronic device (10) can generate a video of a target vehicle (1) being filmed and generate a license plate image of the target vehicle (1) based on the filmed video. The electronic device (10) can generate a plurality of cropped images including a license plate area (12) of the target vehicle (1) based on a plurality of basic images (e.g., frames) of the video of the target vehicle (1). A cropped image may be a part of a basic image including a license plate area (12) of the target vehicle (1). The electronic device (10) can generate a license plate image by synthesizing at least some of the plurality of cropped images into a single image. In one embodiment, the electronic device (10) can determine a reference cropped image among the generated plurality of cropped images. The reference cropped image may be an image that serves as a reference for the cropped images. In one embodiment, the electronic device (10) can calculate a plurality of optical flow values ​​between the determined reference cropped image and each of the plurality of cropped images. In one embodiment, the electronic device (10) can determine a plurality of aligned images among a plurality of cropped images based on a plurality of calculated optical flow values. The aligned image may be an image directly used to generate a license plate image among the cropped images. In one embodiment, the electronic device (10) can generate a license plate image based on a plurality of aligned images. The license plate image may be an image with a higher resolution than the cropped image, including the license plate area (12) of the target vehicle (1).

[0063] According to one embodiment, the electronic device (10) can generate a restored license plate image based on an image including a license plate area (12) of a target vehicle (1). For example, the license plate image may be a cropped image. For example, the license plate image may be an image with higher resolution than the cropped image. In one embodiment, the electronic device (10) can determine one or more sets of numbers for a sequence of numbers appearing on the license plate image. In one embodiment, the electronic device (10) can generate one or more restored license plate images corresponding to each of one or more candidate sets of numbers based on the license plate image and one or more candidate sets of numbers. The electronic device (10) can determine the vehicle number of the target vehicle based on one or more restored license plate images.

[0064] Below, with reference to FIGS. 2 to 24, an image generation method and an electronic device for performing the same are described in detail.

[0065] FIG. 2 is a drawing showing a system of an electronic device according to one embodiment.

[0066] According to one embodiment, an electronic device (10) (e.g., the electronic device (10) of FIG. 1) may include at least one processor (210), a memory (220) for storing instructions, and a communication unit (230).

[0067] The electronic device (10) can generate one or more frames of the surroundings of a vehicle (e.g., target vehicle (1) of FIG. 1). The electronic device (10) can generate a video of the surroundings of the vehicle based on one or more frames.

[0068] The processor (210) may be configured to, when instructions are executed, cause the electronic device (10) to at least: generate a plurality of cropped images including a license plate area of ​​a target vehicle based on a plurality of basic images of a video of a target vehicle, generate a license plate image by synthesizing at least some of the plurality of cropped images into one image, determine one or more sets of candidate numbers for a sequence of numbers appearing on the license plate image, and generate one or more restored license plate images corresponding to each of the one or more sets of candidate numbers based on the license plate image and the one or more sets of candidate numbers.

[0069] The processor (210) can process data received by the communication unit (230) and data stored in memory (220). The "processor" may be a data processing device implemented in hardware having a circuit having a physical structure for executing desired operations. For example, the desired operations may include code or instructions included in a program. For example, the data processing device implemented in hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an Application-Specific Integrated Circuit (ASIC), or a Field Programmable Gate Array (FPGA).

[0070] The processor (210) can execute computer-readable code (e.g., software) stored in memory (e.g., memory (220)) and instructions triggered by the processor (210).

[0071] The memory (220) can store data received by the communication unit (230) and data processed by the processor (210). For example, the memory (220) can store a program (or application, software). The program to be stored may be a set of syntaxes that are coded to control the electronic device (10) and executable by the processor (210).

[0072] In one example, the memory (220) may include one or more volatile memory, non-volatile memory and RAM (Random Access Memory), flash memory, hard disk drive and optical disk drive.

[0073] The memory (220) can store a set of instructions (e.g., software) for operating the electronic device (10). The set of instructions for operating the electronic device (10) can be executed by the processor (210).

[0074] The communication unit (230) can be connected to the processor (210) and memory (220) to transmit and receive data. The communication unit (230) can be connected to other external devices to transmit and receive data. In the following, the expression "transmit and receive A" may indicate transmitting and receiving "information or data representing A".

[0075] The communication unit (230) may be implemented as a circuitry within the electronic device (10). For example, the communication unit (230) may include an internal bus and an external bus. As another example, the communication unit (230) may be an element connecting the electronic device (10) and an external device. The communication unit (230) may be an interface. The communication unit (230) may receive data from an external device and transmit the data to the processor (210) and memory (220).

[0076] FIG. 3 is a flowchart of an image generation method according to one embodiment.

[0077] The following operations 310 to 350 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0078] For example, the electronic device can be a black box. For example, the electronic device can be a computing device that analyzes video or images captured by the black box.

[0079] Before operation 310, the electronic device may acquire one or more frames of the area around the vehicle (e.g., target vehicle (1) of FIG. 1). The electronic device may generate a video of the area around the vehicle based on one or more frames.

[0080] In operation 310, the electronic device may generate a plurality of cropped images including a license plate area of ​​the target vehicle (e.g., license plate area (12) of FIG. 1) based on a plurality of base images of a video of the target vehicle. A cropped image may be a part of a base image including a license plate area of ​​the target vehicle. The size or resolution of the cropped image may vary depending on the distance between the electronic device and the target vehicle. For example, when the distance between the electronic device and the target vehicle is short, the size of the cropped image may be larger or the resolution may be higher compared to when the distance between the electronic device and the target vehicle is long. The operation of generating a plurality of cropped images is described in detail below with reference to FIG. 4.

[0081] In operation 320, the electronic device can generate a license plate image by synthesizing at least some of a plurality of cropped images into a single image. When synthesizing a plurality of cropped images that appear differently as the position of the camera capturing the video and / or the target vehicle changes into a single image, a license plate image with improved visibility and recognition performance compared to the plurality of cropped images can be obtained. An operation to generate a license plate image based on a plurality of cropped images is described in detail below with reference to FIGS. 6 through 14.

[0082] In operation 330, the electronic device may determine one or more sets of candidate numbers for a sequence of numbers appearing on a license plate image. The sets of numbers may include numbers, letters, or other symbols appearing on a vehicle's license plate. The electronic device may determine not only numbers but also letters appearing on the license plate image. The set of candidate numbers may represent the vehicle number of a target vehicle estimated based on the license plate image. An operation for obtaining one or more sets of candidate numbers is described in detail below with reference to FIGS. 15 through 19.

[0083] In operation 340, the electronic device may generate one or more restored license plate images corresponding to each of one or more sets of candidate numbers based on a license plate image and one or more sets of candidate numbers. For example, the electronic device may generate restored license plate images by processing the license plate image and the set of candidate numbers using a text-based image restoration model.

[0084] Since license plates are characterized by displaying numbers in a consistent standard and format, the sequence of numbers in the estimated license plate image, based on these characteristics, can be used to reconstruct the license plate image. By additionally reconstructing the license plate image generated based on video or images, the electronic device can produce a high-definition reconstructed license plate image with further improved visibility and recognition performance.

[0085] According to one embodiment, the electronic device can determine the vehicle number of a target vehicle based on one or more restored license plate images. For example, the electronic device can recognize or extract the vehicle number of the target vehicle as text from a restored license plate image generated based on an optical character recognition (OCR) method. The method by which the electronic device determines the vehicle number of the target vehicle is not limited to the described embodiments. The vehicle number of the target vehicle can be accurately determined based on restored license plate images with improved visibility and recognition performance. An operation for determining the vehicle number of a target vehicle based on one or more restored license plate images is described in detail below with reference to FIGS. 22 and 23.

[0086] FIG. 4 is a flowchart illustrating the operation of generating cropped images according to one embodiment.

[0087] The following operations 410 to 430 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0088] According to one embodiment, the operation 310 described above with reference to FIG. 3 may include operations 410 to 430.

[0089] In operation 410, the electronic device can determine the license plate area of ​​a target vehicle within a first base image. For example, the first base image may be a full-size image generated by the electronic device. For example, the first base image may be a full-size image generated by an external device and transmitted to the electronic device. The first base image may include an image of a target vehicle (e.g., the target vehicle (1) of FIG. 1). For example, the electronic device may determine the license plate area of ​​the target vehicle using a convolutional neural networks (CNN) model. The method by which the electronic device determines the license plate area of ​​the target vehicle is not limited to the described embodiments.

[0090] In operation 420, the electronic device may generate a first basic cropped image including a determined license plate area. For example, the first basic cropped image may be an image obtained by cropping the remaining area of ​​the first basic image excluding the license plate area of ​​the target vehicle. For example, the size or resolution of the first basic cropped image may vary depending on the distance between the electronic device and the target vehicle.

[0091] In operation 430, the electronic device can generate a first cropped image by up-sampling a first basic cropped image. When the distance between the camera that acquired the first basic image and the target vehicle is far, the first basic cropped image may have a smaller image size or lower resolution compared to when the distance between the camera and the target vehicle is close. Based on the up-sampling method, a cropped image of constant size can be generated from basic cropped images of different sizes. In one example, the sizes of a plurality of cropped images may be the same.

[0092] FIG. 5 is a drawing illustrating cropped images and a reference cropped image according to one example.

[0093] Referring to FIG. 5, a plurality of basic images (511, 513, 515, 517, 519), a plurality of basic cropped images (531, 533, 535, 537, 539), and a plurality of cropped images (551, 553, 555, 557, 559) according to one example are illustrated.

[0094] According to one embodiment, an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2) may generate one or more frames including a target vehicle (1) prior to the operation 310 described above with reference to FIG. 3. The electronic device may generate a video of the target vehicle (1) based on one or more frames. The video of the target vehicle (1) may include a plurality of base images (511, 513, 515, 517, 519). In one example, the size of the image of the target vehicle (1) in the plurality of base images (511, 513, 515, 517, 519) may vary based on the distance between the camera capturing the frames and the target vehicle (1). For example, if the distance between the camera and the target vehicle (1) increases, the size of the image of the target vehicle (1) in the plurality of base images (511, 513, 515, 517, 519) may become progressively smaller.

[0095] According to one embodiment, the electronic device can generate a plurality of cropped images (551, 553, 555, 557, 559) including a license plate area (12) of the target vehicle (1), based on a plurality of basic images (511, 513, 515, 517, 519) of a video of the target vehicle (1) captured with reference to FIG. 3 in operation 310 described above.

[0096] Referring to FIG. 4, in the aforementioned operation 410, the electronic device can determine the license plate area (12) of the target vehicle (1) within a plurality of basic images (511, 513, 515, 517, 519).

[0097] The electronic device can generate a plurality of basic crop images (531, 533, 535, 537, 539) including a determined license plate area (12) in operation 420 described above with reference to FIG. 4. In one example, the size of the plurality of basic crop images (531, 533, 535, 537, 539) may vary based on the distance between the electronic device and the target vehicle (1). For example, if the distance between the electronic device and the target vehicle (1) increases, the size of the plurality of basic crop images (531, 533, 535, 537, 539) may become progressively smaller.

[0098] The electronic device can generate a plurality of cropped images (551, 553, 555, 557, 559) by up-sampling a plurality of basic cropped images (531, 533, 535, 537, 539) in operation 430 described above with reference to FIG. 4. The sizes of the plurality of cropped images (551, 553, 555, 557, 559) may be the same. In the process of up-sampling from a basic cropped image to a cropped image, the resolutions of the plurality of cropped images (551, 553, 555, 557, 559) may differ from each other.

[0099] According to one embodiment, the electronic device may determine a reference crop image among a plurality of crop images (551, 553, 555, 557, 559) in operation 320 described above with reference to FIG. 3. The reference crop image may be an image that is referenced among the crop images (551, 553, 555, 557, 559). For example, the reference crop image may be determined as any one of the plurality of crop images (551, 553, 555, 557, 559) based on an image generation time associated with each of the plurality of crop images (551, 553, 555, 557, 559).

[0100] For example, the reference crop image may be determined as the crop image (551) created at the earliest time among a plurality of crop images (551, 553, 555, 557, 559) based on the image creation time. For example, the reference crop image may be determined as the crop image (559) created at the latest time among a plurality of crop images (551, 553, 555, 557, 559) based on the image creation time. For example, the reference crop image may be determined as the crop image (555) created at the middle time among a plurality of crop images (551, 553, 555, 557, 559) based on the image creation time. The criteria and method for determining the reference crop image are not limited to the described embodiments.

[0101] Hereinafter, for the convenience of explanation, the reference crop image will be described as the crop image (555) created at the time of the median value among the multiple crop images (551, 553, 555, 557, 559) based on the time of image creation.

[0102] FIG. 6 is a flowchart illustrating the operation of generating a license plate image based on a plurality of cropped images according to one embodiment.

[0103] The following operations 610 to 640 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0104] According to one embodiment, the operation 320 described above with reference to FIG. 3 may include operations 610 to 640.

[0105] In operation 610, the electronic device may determine a reference crop image among a plurality of crop images (e.g., a plurality of crop images of FIG. 5 (551, 553, 555, 557, 559)). The reference crop image may be an image that serves as a reference for the crop images. For example, the reference crop image may be determined as any one of the plurality of crop images based on an image generation time associated with each of the plurality of crop images.

[0106] In operation 620, the electronic device can calculate multiple optical flow values ​​between a reference crop image and each of the multiple crop images. Below, with reference to FIGS. 7 to 8b, the operation of calculating multiple optical flow values ​​is described in detail.

[0107] In operation 630, the electronic device can determine multiple alignment images among multiple cropped images based on multiple optical flow values. For example, an alignment image may be an image directly used to generate a license plate image among the cropped images. The operation of determining multiple alignment images is described in detail below with reference to FIGS. 9 and 10.

[0108] In operation 640, the electronic device can generate a license plate image based on a plurality of alignment images. The license plate image represents the license plate area of ​​a target vehicle and may be an image with a higher resolution than a cropped image. Below, with reference to FIGS. 11 to 15, the operation of generating a license plate image based on a plurality of alignment images is described in detail.

[0109] FIG. 7 is a flowchart illustrating the operation of calculating optical flow values ​​according to one embodiment.

[0110] The following operations 710 and 720 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0111] According to one embodiment, the operation 620 described above with reference to FIG. 6 may include operations 710 and 720.

[0112] In operation 710, the electronic device can calculate a first position change between a first pixel of a reference crop image and a second pixel of the first crop image corresponding to the first pixel. In a plurality of crop images, the license plate area of ​​a target vehicle may change in position, angle, etc. over time. Based on optical flow, a movement pattern between the plurality of crop images can be identified. For example, based on optical flow, the direction or distance in which a specific pixel has moved between the first crop image and the second crop image can be calculated. In one example, based on optical flow, the position change of a corresponding pixel in each target vehicle license plate area included in the plurality of crop images can be estimated. In one example, a reference crop image (e.g., the reference crop image (555) of FIG. 5) can be used as a reference coordinate for estimating the position change of a corresponding pixel. The electronic device can calculate a first position change between a first pixel of a reference crop image (555) (e.g., the top-left corner of the license plate area of ​​the reference crop image) and a second pixel of a first crop image (e.g., the top-left corner of the license plate area of ​​the first crop image) corresponding to the first pixel (e.g., the crop image (551) of FIG. 5).

[0113] In operation 720, the electronic device can calculate a first optical flow value of a first cropped image based on a first position change. In one example, the first optical flow value of a first cropped image can be calculated based on a first position change between a first pixel of a reference cropped image and a second pixel of a first cropped image.

[0114] FIGS. 8A and FIGS. 8B are diagrams illustrating the operation of calculating optical flow values ​​according to one example.

[0115] Referring to FIG. 8a, a reference cropped image (555) and a plurality of cropped images (551, 553, 555, 557, 559) according to one example are shown.

[0116] According to one embodiment, the electronic device can calculate a first position change (e.g., a position change between (x3, y3) and (x1, y1)) between a first pixel (815) of a reference crop image (555) and a second pixel (811) of a first crop image (551) corresponding to the first pixel (815) in operation 710 described above with reference to FIG. 7. In this way, the electronic device can calculate each position change between a first pixel (815) of a reference crop image (555) and each pixel of a plurality of crop images corresponding to the first pixel (815).

[0117] According to one embodiment, the electronic device can calculate a first optical flow value of a first cropped image (551) based on a first position change in operation 720 described above with reference to FIG. 7. In this way, the electronic device can calculate the optical flow values ​​of each of a plurality of cropped images.

[0118] Referring to FIG. 8b, 30 cropped images according to one example are shown. Based on operations 710 and 720 described above with reference to FIG. 7, the optical flow values ​​of each of the 30 cropped images can be calculated. Based on the 30 optical flow values, the cropped images corresponding to Frame 01, Frame 24, Frame 25, and Frame 28 can be determined as error images.

[0119] The error image is described in detail below with reference to FIGS. 9 and FIGS. 10.

[0120] FIG. 9 is a flowchart illustrating the operation of determining alignment images according to one embodiment.

[0121] The following operations 910 and 920 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0122] According to one embodiment, the operation 630 described above with reference to FIG. 6 may include operations 910 and 920.

[0123] In operation 910, the electronic device can determine an error image among a plurality of cropped images based on a plurality of optical flow values. Since the movement of the electronic device and the target vehicle (e.g., the target vehicle (1) of FIG. 1) is continuous, the change in optical flow values ​​between the plurality of cropped images should not be large. For example, if the change in optical flow values ​​is large, the corresponding cropped image can be estimated as an error image.

[0124] With reference to Fig. 10 below, the operation for determining an error image is described in detail.

[0125] In operation 920, the electronic device may determine a plurality of aligned images based on an error image. A cropped image determined as an error image may be impossible or difficult to correct. In one example, the electronic device may determine the remaining cropped images, excluding the error image among the plurality of cropped images, as aligned images.

[0126] FIG. 10 is a flowchart illustrating the operation of determining an error image according to one embodiment.

[0127] The following operations 1010 to 1030 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0128] According to one embodiment, the operation 910 described above with reference to FIG. 9 may include operations 1010 to 1030.

[0129] In operation 1010, the electronic device can determine a first cropped image and a second cropped image adjacent to the first cropped image among a plurality of cropped images. For example, referring to FIG. 8a, the electronic device can determine a first cropped image (551) and a second cropped image (553) adjacent to the first cropped image (551) among a plurality of cropped images (551, 553, 555, 557, 559).

[0130] In operation 1020, the electronic device can calculate the difference between the first optical flow value of the first cropped image and the second optical flow value of the second cropped image. For example, referring to FIG. 8a, the first optical flow value of the first cropped image (551) can be calculated based on a first position change (e.g., a position change between (x3, y3) and (x1, y1)) between the first pixel (815) of the reference cropped image (555) and the second pixel (811) of the first cropped image (551) corresponding to the first pixel (815). For example, referring to FIG. 8a, the second optical flow value of the second cropped image (553) can be calculated based on a second position change (e.g., a position change between (x3, y3) and (x2, y2)) between the first pixel (815) of the reference cropped image (555) and the third pixel (813) of the second cropped image (553) corresponding to the first pixel (815). In one example, the electronic device can calculate the difference between the first optical flow value of the first cropped image (551) and the second optical flow value of the second cropped image (553) based on the first position change and the second position change.

[0131] In operation 1030, the electronic device can determine whether the first cropped image is an error image based on the difference. In one example, the electronic device can determine whether it is an error image based on the difference between adjacent cropped images. For example, the electronic device can calculate the absolute value of the difference between optical flow values ​​(e.g., first optical flow value and second optical flow value) between adjacent cropped images (e.g., first cropped image and second cropped image). The electronic device can obtain the average of the absolute values ​​of the differences between multiple optical flow values ​​between multiple adjacent cropped images. In one example, the electronic device can compare the average of the absolute values ​​with a preset threshold value as shown in [Equation 1] below.

[0132]

[0133] In [Mathematical Equation 1], f t may be the optical flow value of an adjacent crop image at an interval of t from the reference crop image. For example, may be a preset threshold value. The absolute value of the difference in optical flow values ​​between adjacent crop images before and after can be compared with the preset threshold value. For example, if the comparison results are all greater than the preset threshold value, the crop image may be determined to be an error image. The calculation method for determining whether a crop image is an error image is not limited to the described embodiments.

[0134] FIG. 11 is a flowchart illustrating the operation of generating a license plate image based on a plurality of alignment images according to one embodiment.

[0135] The following operations 1110 to 1130 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0136] According to one embodiment, the operation 640 described above with reference to FIG. 6 may include operations 1110 to 1130.

[0137] In operation 1110, the electronic device can determine a first alignment image that requires correction among a plurality of alignment images. A license plate area (e.g., license plate area (12) in FIG. 1) of a target vehicle (e.g., target vehicle (1) in FIG. 1) forms a plane and is attached to the target vehicle and moves together with it. For example, based on the results of optical flow, a non-planar portion of the license plate area may be determined to require correction. In one example, the electronic device can determine a first alignment image corresponding to a non-planar portion among a plurality of alignment images. The operation of determining the first alignment image that requires correction is described in detail below with reference to FIG. 12.

[0138] In operation 1120, the electronic device can generate a corrected first alignment image by correcting the first alignment image. Below, with reference to FIG. 13, the operation of generating the corrected first alignment image is described in detail.

[0139] In operation 1130, the electronic device may generate a license plate image based on a plurality of alignment images including a corrected first alignment image. In one example, the electronic device may generate a license plate image based on a plurality of existing alignment images that do not require correction and corrected alignment images.

[0140] FIG. 12 is a flowchart illustrating the operation of determining a first alignment image according to one embodiment.

[0141] The following operations 1210 and 1220 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0142] According to one embodiment, the operation 1110 described above with reference to FIG. 11 may include operations 1210 and 1220.

[0143] In operation 1210, the electronic device can determine whether the license plate area of ​​each of the plurality of alignment images satisfies a bilinear approximation. In one example, the electronic device can obtain coordinate values ​​corresponding to four corners (e.g., top-left, top-right, bottom-left, and bottom-right) for the license plate area of ​​each of the plurality of alignment images. The electronic device can determine whether the four corners of the license plate area of ​​each of the plurality of alignment images satisfy a bilinear approximation.

[0144] In operation 1220, the electronic device may determine that a first alignment image that does not satisfy the bilinear approximation requires correction. In one example, the electronic device may determine that an image in which four corners of the license plate area among the alignment images do not satisfy the bilinear approximation requires correction.

[0145] FIG. 13 is a flowchart illustrating the operation of generating a corrected first alignment image according to one embodiment.

[0146] The following operations 1310 to 1330 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0147] According to one embodiment, the operation 1120 described above with reference to FIG. 11 may include operations 1310 to 1330.

[0148] In operation 1310, the electronic device can calculate coordinate values ​​within the first alignment image for corners of the license plate area of ​​the first alignment image. In one example, the electronic device can calculate coordinate values ​​within the first alignment image for corners of the license plate area of ​​the first alignment image (e.g., top-left, top-right, bottom-left, and bottom-right) that are determined to require correction.

[0149] In operation 1320, the electronic device can calculate corrected coordinate values ​​for each of the corner coordinate values ​​based on a bilinear approximation. For example, the electronic device can calculate corrected coordinate values ​​satisfying the bilinear approximation based on the equation of a straight line in the vertical / horizontal direction passing through the center of the license plate area of ​​the first alignment image. For example, the electronic device can calculate the equation of a straight line in the vertical / horizontal direction passing through the center of the license plate area of ​​the first alignment image based on a median vector in the vertical / horizontal direction. In one example, the electronic device can calculate the equation of the straight line by excluding values ​​outside a preset range (e.g., values ​​corresponding to the upper / lower 15%, or values ​​corresponding to the upper / lower 7.5%) in order to reduce the error range. The method for reducing the error range and the preset range are not limited to the described embodiments.

[0150] In operation 1330, the electronic device can generate a corrected first alignment image by correcting the first alignment image based on corrected coordinate values. The electronic device can correct the first alignment image determined to require correction based on corrected coordinate values ​​satisfying a bilinear approximation. The electronic device can generate a corrected first alignment image by correcting the first alignment image.

[0151] FIG. 14 is a diagram illustrating the operation of correcting a first alignment image according to one example.

[0152] Referring to FIG. 14, a plurality of alignment images (551, 553, 555, 559) according to one example are shown.

[0153] According to one embodiment, the electronic device can determine an error image (e.g., error image (557) of FIG. 8a) among a plurality of cropped images (e.g., a plurality of cropped images (551, 553, 555, 557, 559) of FIG. 8a) in operation 910 described above with reference to FIG. 9. According to one embodiment, the electronic device can determine a plurality of aligned images (551, 553, 555, 559) based on the error image in operation 920 described above with reference to FIG. 9.

[0154] According to one embodiment, the electronic device can determine a first alignment image (559) that requires correction among a plurality of alignment images (551, 553, 555, 559) in the operation 1110 described above with reference to FIG. 11.

[0155] In the operation 1210 described above with reference to FIG. 12, the electronic device can determine whether the license plate area of ​​each of the plurality of alignment images (551, 553, 555, 559) satisfies the bilinear approximation. The electronic device can obtain coordinate values ​​corresponding to four corners for each of the license plate areas of the plurality of alignment images (551, 553, 555, 559). For example, for the license plate area of ​​image (551), coordinate values ​​(1411, 1412, 1413, 1414) corresponding to four corners can be obtained. The electronic device can determine whether the four corners of each license plate area satisfy the bilinear approximation.

[0156] The electronic device may determine that a first alignment image that does not satisfy the bilinear approximation requires correction in the operation 1220 described above with reference to FIG. 12. For example, the electronic device may determine that a first alignment image (559) that does not satisfy the bilinear approximation requires correction.

[0157] According to one embodiment, the electronic device can generate a corrected first alignment image (1459) by correcting the first alignment image (559) that requires correction in the operation 1120 described above with reference to FIG. 11.

[0158] Referring again to FIG. 13, in the aforementioned operation 1310, the electronic device can calculate the coordinate values ​​within the first alignment image (559) of the corners (1491, 1492, 1493, 1494) of the license plate area of ​​the first alignment image (559).

[0159] The electronic device can calculate corrected coordinate values ​​(1495, 1496, 1497, 1498) for each of the coordinate values ​​of the corners (1491, 1492, 1493, 1494) based on a bilinear approximation in operation 1320 described above with reference to FIG. 13.

[0160] The electronic device can generate a corrected first alignment image (1459) by correcting the first alignment image (559) based on the corrected coordinate values ​​(1495, 1496, 1497, 1498) in the operation 1330 described above with reference to FIG. 13.

[0161] According to one embodiment, the electronic device can generate a license plate image based on a plurality of alignment images (551, 553, 555, 1459) including a corrected first alignment image (1459) in the operation 1130 described above with reference to FIG. 11.

[0162] FIG. 15 is a flowchart illustrating the operation of generating a license plate image based on a plurality of alignment images according to one embodiment.

[0163] The following operations 1510 to 1530 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0164] According to one embodiment, the operation 640 described above with reference to FIG. 6 may include operations 1510 to 1530.

[0165] For example, a license plate area can be distinguished into a background area and an information area (e.g., vehicle number, such as numbers or letters). In the license plate area, the information area may be more important than the background area. According to one example, an electronic device can enhance the visibility of a license plate image by treating the background area and the information area differently.

[0166] In operation 1510, the electronic device can acquire intensity values ​​of pixels included in the license plate area of ​​each of the plurality of alignment images. For example, colors within the image may be defined on a pixel-by-pixel basis. Each pixel may have a specific intensity value (or color value). The intensity values ​​of the pixels may indicate how bright or dark the corresponding pixel is, or what color it represents. For example, the intensity of a bright color may be high, and the intensity of a dark color may be low. In one embodiment, the electronic device can acquire intensity values ​​for each of all pixels included in the license plate area of ​​each of the plurality of alignment images.

[0167] In operation 1520, the electronic device can determine pixels corresponding to preset intensity values ​​among the acquired intensity values. For example, in the case of a license plate where the background area is a bright color and the information area is a dark color, the electronic device can determine pixels with low intensity values ​​as the information area of ​​the license plate based on the preset intensity values. For example, in the case of a license plate where the background area is a dark color and the information area is a bright color, the electronic device can determine pixels with high intensity values ​​as the information area of ​​the license plate based on the preset intensity values. The preset intensity values ​​may vary depending on the color composition of the license plate and are not limited to the described embodiments.

[0168] In operation 1530, the electronic device can generate a license plate image based on determined pixels. In one example, the electronic device can determine pixels corresponding to an information area based on a preset intensity value. The electronic device can generate a license plate image by applying weights to the pixels corresponding to the information area.

[0169] According to one embodiment, the operation 640 described above with reference to FIG. 6 may further include an operation of post-processing a license plate image based on an image enhancement algorithm. In one example, the image enhancement algorithm may include a CLAHE algorithm. Based on the image enhancement algorithm, the visibility of the license plate image may be increased. According to one embodiment, the operation of post-processing the license plate image based on the image enhancement algorithm may be performed in parallel with or independently of operations 1510 to 1530.

[0170] FIG. 16 is a flowchart illustrating the operation of determining one or more sets of candidate numbers according to one embodiment.

[0171] The following operations 1610 and 1620 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0172] According to one embodiment, the operation 330 described above with reference to FIG. 3 may include operations 1610 and 1620.

[0173] In operation 1610, the electronic device can determine a group of candidate numbers for each number in a sequence of numbers appearing on the license plate image by processing the license plate image using a number recognition model. A number may refer to a number, letter, or other symbol appearing on a vehicle's license plate. A candidate number may be a number, letter, or other symbol that appears to correspond to the number appearing on the image. For example, the number recognition model can determine a group of candidate numbers for each number in a sequence of numbers by performing OCR on the sequence of numbers in the license plate image. For example, the number recognition model may be a Multi-Granularity Prediction for Scene Text Recognition (MGP-STR) model for extracting text on an image, or a model modified based thereon. If the visibility or readability of the numbers appearing on the license plate image is poor, multiple candidate numbers may be determined for a single number. The operation of determining a group of candidate numbers for a number is described in detail below with reference to FIGS. 17 and FIGS. 19.

[0174] In operation 1620, the electronic device may determine one or more sets of candidate numbers based on a group of candidate numbers determined for each number. The set of candidate numbers may represent the vehicle number of a target vehicle estimated based on the license plate image. If the visibility or readability of the license plate image is poor, one or more sets of candidate numbers may be determined corresponding to the license plate image as one or more candidate numbers are determined for each number in the sequence of numbers. The operation of determining one or more sets of candidate numbers is described in detail below with reference to FIGS. 18 and FIGS. 19.

[0175] FIG. 17 is a flowchart illustrating the operation of determining a group of candidate numbers according to one embodiment.

[0176] The following operations 1710 and 1720 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0177] According to one embodiment, the operation 1610 described above with reference to FIG. 16 may include operations 1710 and 1720.

[0178] In operation 1710, the electronic device may obtain license plate type information based on a license plate image. The license plate type information may be information regarding the format in which the license plate displays a sequence of numbers. For example, the license plate type information may include at least one of information regarding how many numbers the sequence of numbers on the license plate consists of, information regarding what type of symbol (e.g., number, Latin character, non-Latin character, or special symbol) appears in each place of the sequence of numbers on the license plate, information regarding what font the sequence of numbers is displayed in, or information regarding the location where the sequence of numbers is displayed within the license plate image.

[0179] According to one embodiment, an electronic device can acquire license plate type information in advance. Since vehicles driving in the same environment have common license plate type information, the electronic device can acquire license plate type information corresponding to the license plate image of a target vehicle in advance. For example, the license plate type information can be acquired based on country or region information entered by a user. For example, the license plate type information can be acquired by processing license plate images of multiple vehicles appearing in a video.

[0180] In operation 1720, the electronic device can determine a group of candidate numbers for each number in a sequence of numbers appearing on a license plate image based on license plate type information. The electronic device can accurately determine a group of candidate numbers for a number by performing OCR on the sequence of numbers in the license plate image based on license plate type information. For example, if it is known in advance based on license plate type information that the first digit in the license plate image is displayed as a number or a Latin character, special characters or non-Latin characters having a shape similar to the actual number on the license plate may be excluded from the group of candidate numbers.

[0181] FIG. 18 is a flowchart illustrating the operation of determining one or more sets of candidate numbers based on groups of candidate numbers according to one embodiment.

[0182] The following operations 1810 to 1830 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0183] According to one embodiment, the operation 1620 described above with reference to FIG. 16 may include operations 1810 to 1830.

[0184] In operation 1810, the electronic device may determine the confidence of each candidate number within a first group of candidate numbers determined for a first number. The confidence of a candidate number may represent the probability that the candidate number matches the actual number on the license plate to which it corresponds. For example, a number recognition model processing a license plate image may output one or more candidate numbers for each number in a sequence of numbers appearing on the license plate image and the confidence of each of the candidate numbers.

[0185] In operation 1820, the electronic device may exclude a first candidate number from a first candidate number group if the confidence level of the first candidate number within the first candidate number group is lower than a preset threshold.

[0186] In operation 1830, the electronic device may determine one or more sets of candidate numbers based on a first group of candidate numbers for a first number and a second group of candidate numbers for a second number. For example, the electronic device may determine one or more sets of candidate numbers for combinations of candidate numbers for each number included in a sequence of numbers.

[0187] According to one embodiment, the electronic device determines the reliability of each of one or more sets of candidate numbers and may exclude at least some of the sets of candidate numbers from the sets of candidate numbers based on the reliability of each of the sets of candidate numbers. The reliability of a set of candidate numbers may be determined by the product of the reliability of each of the candidate numbers included in the set of candidate numbers. For example, the electronic device may use n or n% of the sets of candidate numbers with high reliability among one or more sets of candidate numbers for the generation of a restored license plate image. For example, the electronic device may use some of the sets of candidate numbers with reliability higher than a preset threshold for the generation of a restored license plate image.

[0188] The electronic device can save resources for restoring license plate images by excluding candidate numbers that have a very low probability of matching the actual license plate.

[0189] FIG. 19 is a diagram illustrating the operation of determining one or more sets of candidate numbers according to one example.

[0190] Referring to FIG. 19, one or more sets of candidate numbers (1970) can be determined based on a license plate image (1910) according to one example.

[0191] If the quality of the multiple cropped images for generating the license plate image (1910) is low, the visibility and readability of the license plate image (1910) may not be sufficient to determine the vehicle number of the target vehicle, despite the operation for generating the license plate image (1910) described above with reference to FIGS. 3 through 15. According to one embodiment, the electronic device determines one or more sets of candidate numbers (1970) for the license plate image (1910) and may use the sets of candidate numbers (1970) for generating the restored license plate image.

[0192] The electronic device can determine a group of candidate numbers for the numbers (1911, 1912, 1913, 1914, 1915, 1916, 1917) of the sequence of numbers appearing on the number plate image (1910) by processing the number plate image (1910) using a number recognition model in the operation 1610 described above with reference to FIG. 16.

[0193] According to one embodiment, a number recognition model may determine one or more candidate numbers for each number (1911, 1912, 1913, 1914, 1915, 1916, 1917) of a number sequence. A table (1930) may display three candidate numbers determined for each number (1911, 1912, 1913, 1914, 1915, 1916, 1917) of a number sequence. A candidate number group corresponding to each number (1911, 1912, 1913, 1914, 1915, 1916, 1917) of a number sequence may include one or more candidate numbers determined for the corresponding number. For example, a first candidate number group corresponding to the first number (1911) may include 7, 1, and 2. For example, the second candidate number group corresponding to the second number (1912) may include 0, D, and O. For example, the fifth candidate number group corresponding to the fifth number (1915) may include C, O, and D. The number appearing in parentheses below the candidate number may be the confidence level corresponding to the candidate number.

[0194] According to one embodiment, a digit recognition model can determine a group of candidate digits for each digit in a sequence of digits appearing on a license plate image based on license plate type information. An electronic device can accurately determine a group of candidate digits for a digit by performing OCR on the sequence of digits of the license plate image based on license plate type information.

[0195] According to one embodiment, the number recognition model may exclude some of the candidate numbers shown in the table (1930) from the candidate number group. The table (1950) may display the result of some of the candidate numbers being excluded from the candidate number group. As in operations 1810 and 1820 described above with reference to FIG. 18, some of the candidate numbers may be excluded from the candidate number group based on the confidence level of each of the candidate numbers included in the candidate number group. For example, candidate numbers with a confidence level of 0.25 or less in each candidate number group may be excluded from each candidate number group. Some of the candidate numbers may be excluded from the candidate number group based on license plate type information. For example, if the second number (1912) is determined to be a number based on license plate type information, non-number candidate numbers (D, O) may be excluded from the candidate number group. For example, if the 7th digit (1917) is determined to be a Latin character based on the license plate type information, candidate digits (2, 7) that are not Latin characters may be excluded from the candidate digit group.

[0196] According to one embodiment, the electronic device determines the reliability of each of one or more sets of candidate numbers (1970) and, based on the reliability of each of the sets of candidate numbers (1970), may exclude at least some of the sets of candidate numbers (1970) from the sets of candidate numbers (1970). The electronic device may save resources for the restoration of the license plate image by excluding candidate numbers that have a very low probability of matching the actual license plate.

[0197] In the operation 1830 described above with reference to FIG. 18, the electronic device can determine combinations of candidate numbers for each number included in the sequence of numbers as one or more candidate number sets (1970). In the illustrated example, among the candidate number groups corresponding to each of the first number (1911) through the seventh number (1917), only the candidate number group corresponding to the fifth number (1915) includes multiple candidate numbers (C, O), so '7011 CDZ' of the first candidate number set (1971) and '7011 ODZ' of the second candidate number set (1972) can be determined as candidate number sets (1970) by combination with candidate numbers corresponding to other numbers.

[0198] FIG. 20 is a flowchart illustrating the operation of acquiring a restored license plate image according to one embodiment.

[0199] The following operations 2010 to 2030 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0200] According to one embodiment, the operation 340 described above with reference to FIG. 3 may include operations 2010 to 2030.

[0201] In operation 2010, the electronic device can acquire an image feature map for a license plate image. The image feature map may be a matrix representing features or patterns of an image. For example, the electronic device may determine the image feature map for a license plate image using an image encoder. For example, the image encoder may include a CNN model.

[0202] In operation 2020, the electronic device may acquire a first digit feature vector for a first set of candidate digits among one or more sets of candidate digits. The digit feature vector may be a matrix representing the features or patterns of the digits so that the digits can be processed together with an image feature map. For example, the electronic device may determine the digit feature vector for the set of candidate digits using a text encoder. For example, the text encoder may include an attention layer.

[0203] In operation 2030, the electronic device can obtain a first restored license plate image by restoring the license plate image based on a first set of candidate digits by processing an image feature map and a first digit feature vector using a conditional generative AI (artificial intelligence) model (e.g., a conditional diffusion model). The diffusion model may be a text conditional diffusion model (TCDM) including a UNet structure. The UNet structure is a CNN-based model including an encoder, skip connections, and a decoder, and can be used to generate an upscaled image while preserving information from the original image. For example, the UNet structure can learn the correlation between the image feature map and the digit feature vector using a cross-attention module. The diffusion model can generate the restored license plate image based on the set of candidate digits by repeatedly calling the UNet structure. For example, the diffusion model can be trained to reduce the difference between the data distribution of the license plate image generated using the diffusion model and the data distribution of the high-resolution license plate image.

[0204] FIG. 21 is a diagram illustrating the operation of acquiring a restored license plate image according to one example.

[0205] Referring to FIG. 21, one or more restored license plate images (2101, 2103) can be generated based on a license plate image (1910) according to one example and one or more sets of candidate numbers (1971, 1973).

[0206] A text-based image restoration model (2100) can receive a license plate image (1910) and one or more sets of candidate numbers (1971, 1973) as input and generate one or more restored license plate images (2101, 2103). The text-based image restoration model (2100) may include an image encoder for determining an image feature map for the license plate image (1910), a text encoder for determining a numeric feature vector for each of the sets of candidate numbers (1971, 1973), and a diffusion model for generating the restored license plate images (2101, 2103) based on the image feature map and the first numeric feature vector. The electronic device can obtain an image feature map for a license plate image (1910) using a text-based image restoration model (2100), obtain a first digit feature vector for a first set of candidate digits among one or more sets of candidate digits, and obtain a first restored license plate image (2101) by restoring the license plate image (1910) based on a first set of candidate digits (1971) by processing the image feature map and the first digit feature vector using a diffusion model.

[0207] The electronic device can obtain a second restored license plate image (2103) by restoring the license plate image (1910) based on a second set of candidate numbers (1973) by using operations similar to those used to obtain the first restored license plate image (2101). The operations for obtaining the first restored license plate image (2101) and the operations for obtaining the second restored license plate image (2103) may be performed sequentially or in parallel.

[0208] The restored license plate images (2101, 2103) generated based on candidate number sets (1971, 1973) may be images with higher resolution than the license plate image (1910). As the restored license plate images (2101, 2103) are generated based on different candidate number sets, the first restored license plate image (2101) and the second restored license plate image (2103) may be restored differently. For example, since the first restored license plate image (2101) is restored based on the first candidate number set (1971) which matches the sequence of numbers of the actual license plate, it may have higher visibility and readability compared to the second restored license plate image (2103). Even though the restored license plate images (2101, 2103) are generated based on different candidate number sets, each of the restored license plate images (2101, 2103) may represent the same sequence of numbers. For example, as the restored license plate images (2101, 2103) are generated based on different sets of candidate numbers as described above, each of the restored license plate images (2101, 2103) may represent a different sequence of numbers.

[0209] FIG. 22 is a flowchart illustrating the operation of determining the vehicle number of a target vehicle according to one embodiment.

[0210] The following operation 2210 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0211] According to one embodiment, operation 2210 may be performed after operation 340 described above with reference to FIG. 3 is performed.

[0212] In operation 2210, the electronic device can determine the vehicle number of a target vehicle based on one or more restored license plate images. For example, the electronic device can recognize or extract the vehicle number of the target vehicle as a character from a restored license plate image generated based on an optical character recognition (OCR) method. The method by which the electronic device determines the vehicle number of the target vehicle is not limited to the described embodiments. The vehicle number of the target vehicle can be accurately determined based on restored license plate images with improved visibility and recognition performance. A method for determining the vehicle number of a target vehicle according to one embodiment is described in detail with reference to FIG. 23.

[0213] FIG. 23 is a flowchart illustrating the operation of determining the vehicle number of a target vehicle based on sequences of numbers appearing on restored license plate images according to one embodiment.

[0214] The following operations 2310 to 2330 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0215] According to one embodiment, the operation 2210 described above with reference to FIG. 22 may include operations 2310 to 2330.

[0216] In operation 2310, the electronic device can determine a first sequence of numbers appearing on a first restored license plate image by processing the first restored license plate image (e.g., the first restored license plate image (2101) of FIG. 21) using a number recognition model. For example, the number recognition model may be an MGP-STR model for extracting text on an image or a model modified based thereon.

[0217] In operation 2320, the electronic device can determine a second sequence of numbers appearing on a second restored license plate image by processing a second restored license plate image (e.g., the second restored license plate image (2103) of FIG. 21) using a number recognition model.

[0218] In operation 2330, the electronic device can determine the first sequence of numbers as the vehicle number of the target vehicle if all sequences of numbers determined based on each of one or more restored license plate images match.

[0219] For example, as illustrated in FIG. 21, even though restored license plate images (2101, 2103) are generated based on different sets of candidate numbers, the first number sequence and the second number sequence may be the same. If the restored license plate images are generated based on different sets of candidate numbers but show the same number sequence, the electronic device can determine the number sequence as the vehicle number of the target vehicle because it is highly likely that the number sequence matches the actual vehicle number.

[0220] For example, as restored license plate images are generated based on different sets of candidate numbers, the first number sequence and the second number sequence may differ. If the restored license plate images generated based on different sets of candidate numbers represent different number sequences, the electronic device may not determine the vehicle number of the target vehicle as a single number sequence. The electronic device may output one or more restored license plate images or number sequences corresponding to the restored license plate images.

[0221] The method of determining the license plate number of a target vehicle based on one or more restored license plate images is not limited to the examples described. For example, an electronic device may determine the reliability of each of one or more sequences of numbers and output one or more sequences of numbers along with the reliability corresponding to each sequence of numbers.

[0222] FIG. 24 is a flowchart of an image generation method according to one embodiment.

[0223] The following operations 2410 to 2440 may be performed by an electronic device (e.g., the electronic device (10) of FIG. 1 or FIG. 2). For example, the electronic device may include a processor (e.g., the processor (210) of FIG. 2), a memory (e.g., the memory (220) of FIG. 2), and a communication unit (e.g., the communication unit (230) of FIG. 2).

[0224] For example, the electronic device can be a black box. For example, the electronic device can be a computing device that analyzes video or images captured by the black box.

[0225] In operation 2410, the electronic device may acquire a license plate image in which the license plate of the target vehicle appears. For example, the license plate image may be a cropped image of the area excluding the license plate area of ​​the target vehicle from an image in which the target vehicle appears. For example, the license plate image may be an image generated by creating a plurality of cropped images based on a plurality of basic images, as in operations 310 and 320 described above with reference to FIG. 3, and synthesizing at least some of the plurality of cropped images into a single image.

[0226] In operation 2420, the electronic device can determine one or more sets of candidate numbers for a sequence of numbers appearing on a license plate image. The description of operation 330 described above with reference to FIG. 3 may be similarly modified and applied.

[0227] In operation 2430, the electronic device may generate one or more restored license plate images corresponding to each of one or more sets of candidate numbers based on a license plate image and one or more sets of candidate numbers. One or more restored license plate images may be images with improved visibility and readability compared to the license plate image. The description of operation 340 described above with reference to FIG. 3 may be similarly modified and applied.

[0228] In operation 2440, the electronic device can determine the vehicle number of a target vehicle based on one or more restored license plate images. As the vehicle number of the target vehicle is determined based on an image with improved visibility and readability, the vehicle number of the target vehicle can be accurately determined. The description of operation 2210 described above with reference to FIG. 22 may be similarly modified and applied.

[0229] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. The devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0230] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave in order to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.

[0231] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0232] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0233] Although the embodiments described above have been explained with reference to limited drawings, those skilled in the art can apply various technical modifications and variations based thereon. For example, appropriate results can be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0234] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

1. A method for generating a vehicle license plate image performed by an electronic device, wherein The operation of generating a plurality of cropped images including a license plate area of ​​the target vehicle based on a plurality of basic images of a video in which the target vehicle is captured; The operation of generating a first license plate image by synthesizing at least some of the plurality of cropped images into a single image; The operation of determining one or more sets of candidate numbers for a sequence of numbers appearing on the first number plate image; and The operation of generating one or more restored license plate images corresponding to each of the one or more candidate number sets based on the first license plate image and the one or more candidate number sets. including, Method for generating vehicle license plate images.

2. In Paragraph 1, The operation of determining the above-mentioned set of one or more candidate numbers is, An operation to determine a group of candidate numbers for each number in a sequence of numbers appearing on the first number plate image by processing the first number plate image using a number recognition model; and The operation of determining one or more sets of candidate numbers based on a group of candidate numbers determined for each of the above numbers. including, Method for generating vehicle license plate images.

3. In Paragraph 2, The operation of determining one or more sets of candidate numbers based on the group of candidate numbers determined for each of the above numbers is, An operation to determine the reliability of each candidate number within the first group of candidate numbers determined for the first number; If the confidence level of the first candidate number within the first candidate number group is lower than a preset threshold, the operation of excluding the first candidate number from the first candidate number group; and The operation of determining one or more sets of candidate numbers based on the first group of candidate numbers for the first number and the second group of candidate numbers for the second number. including, Method for generating vehicle license plate images.

4. In Paragraph 2, The operation of determining the group of candidate numbers for each number of the above number sequence is, An operation to obtain license plate type information based on the first license plate image above; and The operation of determining the candidate number group for each number of the number sequence appearing on the first number plate image based on the above number plate type information. including, Method for generating vehicle license plate images.

5. In Paragraph 1, The operation of generating the one or more restored license plate images corresponding to each of the one or more candidate number sets is, The operation of obtaining a first image feature map for the first license plate image; The operation of obtaining a numeric feature vector for the upper set of candidate numbers among the above one or more sets of candidate numbers; and The operation of obtaining a first restored license plate image by restoring the first license plate image based on the top candidate number set by processing the first image feature map and the number feature vector using a conditional generative AI (artificial intelligence) model. including, Method for generating vehicle license plate images.

6. In Paragraph 1, The operation of generating the first license plate image by synthesizing at least some of the plurality of cropped images into a single image is, The operation of determining a reference crop image among the plurality of crop images above; The operation of calculating a plurality of optical flow values ​​between the reference crop image and each of the plurality of crop images; An operation of determining a plurality of aligned images among the plurality of cropped images based on the plurality of optical flow values; and The operation of generating the first number plate image based on the plurality of alignment images above; including, Method for generating vehicle license plate images.

7. In Paragraph 6, The cropped image above is, Determined as one of the plurality of cropped images based on the image generation time associated with each of the plurality of cropped images, Method for generating vehicle license plate images.

8. In Paragraph 6, The operation of determining the above plurality of alignment images is, An operation of determining an error image among the plurality of cropped images based on the plurality of optical flow values; and Operation of determining the plurality of alignment images based on the above error image including, Method for generating vehicle license plate images.

9. In Paragraph 6, The operation of generating the first license plate image above is, The operation of determining a first alignment image requiring correction among the plurality of alignment images above; The operation of generating the corrected first alignment image by correcting the first alignment image; and Operation of generating the first number plate image based on the plurality of alignment images including the corrected first alignment image. including, Method for generating vehicle license plate images.

10. In Paragraph 6, The operation of generating the first license plate image above is, An operation to obtain intensity values ​​of pixels included in the number plate area of ​​each of the plurality of alignment images; The operation of determining pixels corresponding to preset intensity values ​​among the above acquired intensity values; and The operation of generating the first license plate image based on the above-determined pixels including, Method for generating vehicle license plate images.

11. In Paragraph 1, The operation of determining the vehicle number of the target vehicle based on one or more restored license plate images. including more, Method for generating vehicle license plate images.

12. In Paragraph 11, The operation of determining the vehicle number of the target vehicle based on the one or more restored license plate images is, An operation to determine a first sequence of numbers appearing on a first restored license plate image by processing a first restored license plate image among the one or more restored license plate images using a number recognition model; The operation of determining a second sequence of numbers appearing on a second restored license plate image by processing a second restored license plate image among the one or more restored license plate images using the number recognition model; and When the sequences of numbers determined based on each of the one or more restored license plate images match, the operation of determining the first sequence of numbers as the vehicle number of the target vehicle. including, Method for generating vehicle license plate images.

13. A computer program stored on a computer-readable recording medium in combination with hardware to execute the method of claim 1.

14. In electronic devices, At least one processor including processing circuitry; and The memory includes one or more storage media for storing instructions, and When the above instructions are executed individually or collectively by the at least one processor, the electronic device causes at least: Based on multiple basic images of a video in which a target vehicle is filmed, multiple cropped images including the license plate area of ​​the target vehicle are generated, and A first license plate image is generated by synthesizing at least some of the above-mentioned plurality of cropped images into a single image, and One or more sets of candidate numbers for a sequence of numbers appearing on the first number plate image are determined, and Based on the first license plate image and the one or more sets of candidate numbers, one or more restored license plate images corresponding to each of the one or more sets of candidate numbers are generated. making, Electronic device.

15. A method for generating a vehicle license plate image performed by an electronic device, The operation of acquiring a second license plate image in which the license plate of the target vehicle appears; The operation of determining one or more sets of candidate numbers for a sequence of numbers appearing on the second number plate image; The operation of generating one or more restored license plate images corresponding to each of the one or more candidate number sets based on the second license plate image and the one or more candidate number sets; and The operation of determining the vehicle number of the target vehicle based on one or more restored license plate images. including, Method for generating vehicle license plate images.

16. In Paragraph 15, The operation of determining the above-mentioned set of one or more candidate numbers is, An operation to determine a group of candidate numbers for each number in a sequence of numbers appearing on the second license plate image by processing the second license plate image using a number recognition model; and The operation of determining one or more sets of candidate numbers based on a group of candidate numbers determined for each of the above numbers. including, Method for generating vehicle license plate images.

17. In Paragraph 16, The operation of determining one or more sets of candidate numbers based on the group of candidate numbers determined for each of the above numbers is, An operation to determine the reliability of each candidate number within the first group of candidate numbers determined for the first number; If the confidence level of the first candidate number within the first candidate number group is lower than a preset threshold, the operation of excluding the first candidate number from the first candidate number group; and The operation of determining one or more sets of candidate numbers based on the first group of candidate numbers for the first number and the second group of candidate numbers for the second number. including, Method for generating vehicle license plate images.

18. In Paragraph 16, The operation of determining the group of candidate numbers for each number of the above number sequence is, The operation of obtaining license plate type information based on the second license plate image above; and The operation of determining the candidate number group for each number of the number sequence appearing on the second number plate image based on the above number plate type information. including, Method for generating vehicle license plate images.

19. In Paragraph 15, The operation of generating the one or more restored license plate images corresponding to each of the one or more candidate number sets is, The operation of obtaining a second image feature map for the second license plate image above; The operation of obtaining a first number feature vector for a first set of candidate numbers among the above one or more sets of candidate numbers; and The operation of obtaining a first restored license plate image by restoring the second license plate image based on the first set of candidate numbers by processing the second image feature map and the first number feature vector using a diffusion model. including, Method for generating vehicle license plate images.

20. In Paragraph 15, The operation of acquiring the second license plate image in which the license plate of the target vehicle appears is, The operation of generating a plurality of cropped images including a license plate area of ​​the target vehicle based on a plurality of basic images of a video in which the target vehicle is captured; and The operation of generating the second license plate image by synthesizing at least some of the plurality of cropped images into a single image. including, Method for generating vehicle license plate images.