A method, system, device and medium for restoring a defective license plate image

Abstract

The present invention discloses a method, system, device and medium for restoring a damaged license plate image, relating to the technical field of image processing. The method includes: extracting a damaged license plate image from a real-time monitoring video; inputting the damaged license plate image into a pre-trained restoration model to obtain a restored license plate image; the pre-trained restoration model is trained in the following manner: generating an incomplete license plate image dataset according to a random segmentation of a complete license plate image dataset; constructing a first generator for generating a complete license plate image from an incomplete license plate image and a second generator for generating an incomplete license plate image from a complete license plate image; training the pre-trained restoration model according to the first generator and the second generator in combination with a generation loss, a cycle consistency loss and a license plate loss. The present invention combines a complete license plate image dataset and an incomplete license plate image dataset, greatly improving the accuracy and authenticity of the model output results.

Description

Background Art

[0002] The current license plate recognition mainly uses deep learning methods. First, object detection algorithms such as the YOLO series or line detection models are used on the image to detect the license plate area. Secondly, on the license plate area, OCR recognition models such as CRNN or LPRNet models are used to recognize the characters on the license plate. The existing technology has significant drawbacks when dealing with incomplete or blurred license plate areas, resulting in poor recognition effects. The specific drawbacks are as follows:

[0003] Limitations of the detection model: Existing license plate detection models sometimes cannot perfectly detect all license plates. This offset of the detection frame will lead to incomplete license plate images, thus affecting the recognition accuracy.

[0004] Image boundary problem: When the license plate is located at the image boundary, some characters may be cropped or occluded, resulting in incomplete license plate information. In this case, it is difficult for the recognition system to accurately obtain the complete license plate information.

[0005] Influence of occluders: The license plate may be partially occluded by objects such as branches and wires, further increasing the difficulty of recognition.

[0006] For the above three situations, the existing technology is difficult to effectively solve the restoration and recognition problems of incomplete license plate images, affecting the overall performance and reliability of the license plate recognition system. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a method, system, device and medium for restoring incomplete license plate images in view of the deficiencies of the existing technology, specifically as follows:

[0008] 1) In the first aspect, the present invention provides a method for restoring an incomplete license plate image, and the specific technical solution is as follows:

[0009] Obtain the real-time monitoring image collected by the monitoring device, and extract the incomplete license plate image from the real-time monitoring image;

[0010] Input the incomplete license plate image into a pre-trained restoration model to obtain the restored license plate image;

[0011] The pre-trained restoration model is trained in the following manner:

[0012] According to the random segmentation of the complete license plate image dataset, generate an incomplete license plate image dataset;

[0013] Construct a first generator for generating complete license plate images from incomplete license plate images and a second generator for generating incomplete license plate images from complete license plate images;

[0014] The training of the pre-trained recovery model is completed by combining the generation loss, cycle consistency loss, and license plate loss according to the first generator and the second generator.

[0015] The beneficial effects of a method for recovering a damaged license plate image provided by the present invention are as follows:

[0016] In the process of training the recovery model in this solution, a complete license plate image dataset and an incomplete license plate image dataset are combined. In addition, the total loss function includes three items, namely, generation loss, cycle consistency loss, and license plate loss, which greatly improves the accuracy and authenticity of the model output results.

[0017] On the basis of the above solution, the present invention can also be improved as follows.

[0018] Further, in the incomplete license plate image dataset, the images are divided into four categories: incomplete upper part of the image, incomplete lower part of the image, incomplete left part of the image, and incomplete right part of the image.

[0019] Further, the generation loss includes: the first loss difference between the complete license plate image generated from the incomplete license plate image and the real complete license plate image, and the generation loss also includes: the second loss difference between the incomplete license plate image generated from the complete license plate image.

[0020] Further, the generation loss includes: the first loss difference between the complete license plate image generated from the incomplete license plate image and the real complete license plate image, and the generation loss also includes: the second loss difference between the incomplete license plate image generated from the complete license plate image.

[0021] Further, the cycle consistency loss includes: the first consistency value between the incomplete license plate image and the original incomplete license plate image after generating the target complete license plate image from the incomplete license plate image and then generating the original incomplete license plate image from the target complete license plate image;

[0022] The cycle consistency loss also includes: the second consistency value between the complete license plate image and the original complete license plate image after generating the target incomplete license plate image from the complete license plate image and then generating the original complete license plate image from the target incomplete license plate image.

[0023] Further, the license plate loss includes: the first license plate loss obtained by calculating the loss of the first result of recognizing the complete license plate image;

[0024] The license plate loss also includes: the second license plate loss obtained by calculating the loss of the second result of recognizing the incomplete license plate image.

[0025] 2) Second aspect, the present invention also provides a recovery system for incomplete license plate images, and the specific technical solution is as follows:

[0026] The acquisition module is used to: acquire the real-time monitoring video collected by the monitoring device, and extract the incomplete license plate image from the real-time monitoring video;

[0027] The recovery module is used to: input the incomplete license plate image into the pre-trained recovery model to obtain the recovered license plate image;

[0028] The pre-trained recovery model is trained in the following manner:

[0029] According to the random segmentation of the complete license plate image dataset, an incomplete license plate image dataset is generated;

[0030] A first generator for generating a complete license plate image from an incomplete license plate image and a second generator for generating an incomplete license plate image from a complete license plate image are constructed;

[0031] According to the first generator and the second generator, combined with the generation loss, the cycle consistency loss, and the license plate loss, the training of the pre-trained recovery model is completed.

[0032] On the basis of the above solution, the present invention can also be improved as follows.

[0033] Further, in the incomplete license plate image dataset, the images are divided into four categories: the upper part of the image is incomplete, the lower part of the image is incomplete, the left part of the image is incomplete, and the right part of the image is incomplete.

[0034] Further, the generation loss includes: the first loss difference between the complete license plate image generated from the incomplete license plate image and the real complete license plate image, and the generation loss also includes: the second loss difference between the incomplete license plate image generated from the complete license plate image.

[0035] Further, the generation loss includes: the first loss difference between the complete license plate image generated from the incomplete license plate image and the real complete license plate image, and the generation loss also includes: the second loss difference between the incomplete license plate image generated from the complete license plate image.

[0036] Further, the cycle consistency loss includes: the first consistency value between the incomplete license plate image and the original incomplete license plate image after generating the target complete license plate image from the incomplete license plate image and then generating the original incomplete license plate image from the target complete license plate image;

[0037] The cycle consistency loss also includes: the second consistency value between the complete license plate image and the original complete license plate image after generating the target incomplete license plate image from the complete license plate image and then generating the original complete license plate image from the target incomplete license plate image.

[0038] Further, the license plate loss includes: a first license plate loss obtained by calculating the loss of the first result of recognizing the complete license plate image;

[0039] The license plate loss further includes: a second license plate loss obtained by calculating the loss of the second result of recognizing the incomplete license plate image.

[0040] 3) In a third aspect, the present invention further provides an electronic device, which includes a processor coupled to a memory. At least one computer program is stored in the memory and is loaded and executed by the processor to enable the electronic device to implement any one of the above methods.

[0041] 4) In a fourth aspect, the present invention further provides a computer-readable storage medium in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to enable a computer to implement any one of the above methods.

[0042] It should be noted that for the beneficial effects obtained by the technical solutions of the second to fourth aspects of the present invention and their corresponding possible implementation manners, reference may be made to the technical effects of the first aspect and its corresponding possible implementation manners described above, and details are not described herein again. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] By reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings, other features, objects, and advantages of the present invention will become more apparent:

[0044] Figure 1 is a flowchart of a method for restoring a mutilated license plate image according to an embodiment of the present invention;

[0045] Figure 2 is a structural framework diagram of an electronic device according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will further describe the embodiments of the present invention in detail with reference to the accompanying drawings.

[0047] As Figure 1 shown, a method for restoring a mutilated license plate image according to an embodiment of the present invention includes the following steps:

[0048] S1. Obtain real-time monitoring images collected by a monitoring device, and extract mutilated license plate images from the real-time monitoring images;

[0049] S2. Input the mutilated license plate images into a pre-trained restoration model to obtain restored license plate images;

[0050] The pre-trained recovery model is trained in the following manner:

[0051] Generate an incomplete license plate image dataset according to a random split of the complete license plate image dataset;

[0052] Construct a first generator for generating a complete license plate image from an incomplete license plate image and a second generator for generating an incomplete license plate image from a complete license plate image;

[0053] Complete the training of the pre-trained recovery model according to the first generator and the second generator in combination with a generation loss, a cycle consistency loss, and a license plate loss

[0054] The beneficial effects of a method for recovering a damaged license plate image provided by the present invention are as follows:

[0055] In the process of training the recovery model in this solution, the complete license plate image dataset and the incomplete license plate image dataset are combined. In addition, the total loss function includes three items, namely, a generation loss, a cycle consistency loss, and a license plate loss, which greatly improves the accuracy and authenticity of the model output results.

[0056] S1. Obtain a real-time monitoring image collected by a monitoring device, and extract a damaged license plate image from the real-time monitoring image. Among them:

[0057] The monitoring device can be: a camera at any traffic intersection, or a road monitoring camera.

[0058] The damaged license plate image refers to: an image corresponding to a license plate blocked by a foreign object or an image of the license plate information at the boundary of the monitoring image.

[0059] The process of extracting a damaged license plate image from the real-time monitoring image is as follows:

[0060] Perform noise reduction processing (noise filtering processing) and license plate positioning processing on each frame of the monitoring image to determine the damaged license plate image in the frame of the monitoring image;

[0061] Among them, the noise reduction processing includes:

[0062] Automatic white balance, automatic exposure, gamma correction, edge enhancement, and contrast adjustment processing.

[0063] The process of automatic white balance is: according to the light conditions collected by the white balance sensor set in the monitoring device, determine the color temperature value and chromaticity distance of the object being photographed in the current frame of the monitoring image, determine the adjustment value corresponding to the current frame according to the color temperature value and chromaticity distance, and perform adjustment based on the adjustment value.

[0064] Gamma correction includes: RGB brightness calculation and RGB darkness calculation, determining whether the current frame of monitored image needs to perform RGB brightness calculation or RGB darkness calculation, and performing corresponding processing according to the determination result;

[0065] The RGB brightness calculation formula is:

[0066]

[0067] The RGB darkness calculation formula is:

[0068]

[0069] The process of determining the current frame of monitored image is as follows:

[0070] Judge according to the contrast and brightness of the current frame of monitored image. When both the contrast and brightness exceed the upper limit of the preset contrast range and the upper limit of the preset brightness range, it is determined that the current frame of monitored image needs to perform RGB darkness calculation;

[0071] When both the contrast and brightness are lower than the lower limit of the preset contrast range and the lower limit of the preset brightness range, it is determined that the current frame of monitored image needs to perform RGB brightness calculation.

[0072] Among them, the contrast refers to: the difference between the brightest and darkest parts of various different colors in the image. The higher the contrast, the more dazzling and vivid the image feels. On the contrary, the smaller the contrast, the less obvious the change. High contrast means the image may be brighter, while low contrast may mean the image is darker.

[0073] Brightness refers to: for a grayscale image, the brightness is directly related to the grayscale value. The higher the grayscale value, the brighter the image. In a color image, adjusting the brightness of the image affects the overall brightness of the image, rather than just a specific area. If you need to adjust the brightness of the image, it can be achieved by adjusting the brightness.

[0074] Edge enhancement is implemented through a preset algorithm. The preset algorithm includes:

[0075] Feature extraction is performed on any frame of surveillance image through a sliding window algorithm to obtain a non-edge feature image set. Non-correlated feature suppression processing is performed on the non-edge feature image set through a non-edge suppression operator to obtain a suppressed feature image corresponding to each feature image in the non-edge feature image set. The corresponding feature image is replaced with the suppressed feature image to obtain an initial feature map corresponding to any frame of surveillance image. Through the East China window algorithm, feature extraction is performed on the initial feature map to obtain an edge feature image set. Through an edge enhancement operator, correlated feature enhancement processing is performed on the edge feature image set to obtain an enhanced feature image corresponding to each feature image in the edge feature image set. The corresponding feature image is replaced with the enhanced feature image to obtain an edge-enhanced feature map corresponding to the initial feature map. Among them:

[0076] The non-edge suppression operator determines non-edge sub-blocks in the training set, constructs a set of non-edge typical feature sub-blocks based on the non-edge sub-blocks, and performs matrix solution on the images in the set of non-edge typical feature sub-blocks to obtain the non-edge suppression operator.

[0077] The edge enhancement operator determines edge sub-blocks in the training set, constructs a set of edge typical feature sub-blocks based on the edge sub-blocks, and performs matrix solution on the images in the set of edge typical feature sub-blocks to obtain the edge enhancement operator.

[0078] License plate location: Through a preset model (the preset model is trained through a texture feature analysis and location algorithm), any frame of surveillance image is analyzed and processed, that is, row-column scanning is performed on the grayscale image after image preprocessing. The candidate area containing the license plate line segment in the column direction is determined through row scanning, the starting row coordinate and height of the area are determined, and then column scanning is performed on the area to determine its column coordinate and width, thereby determining a license plate area. This license plate area is the incomplete license plate image.

[0079] Character segmentation: After locating the license plate area in the image, through grayscale conversion, grayscale stretching, binarization, and edge processing, the character area is further accurately located, and then the dynamic template method is proposed according to the character size feature for character segmentation, and the character size is normalized.

[0080] Character recognition: The segmented characters are scaled and feature-extracted to obtain the expression form of specific characters, and then through classification discrimination and classification rules, they are matched and discriminated with the standard character expression forms in the character template database to recognize the input character image.

[0081] S2, input the incomplete license plate image into a pre-trained restoration model to obtain a restored license plate image.

[0082] Among them, the training process of the pre-trained restoration model is as follows:

[0083] (1) Make a data set

[0084] First, a dataset containing complete license plate images and incomplete license plate images needs to be created. The specific steps are as follows:

[0085] 1. Complete license plate image dataset (folder A):

[0086] Collect a dataset containing complete license plate images. These images should be as diverse as possible to cover different license plate styles and backgrounds.

[0087] 2. Incomplete license plate image dataset (folder B):

[0088] Randomly generate incomplete license plate images from the complete license plate images. The incomplete license plate images are divided into four categories according to the different directions of character loss: B1, B2, B3, B4, representing the following situations respectively:

[0089] B1: The characters in the upper part of the image are incomplete,

[0090] B2: The characters in the lower part of the image are incomplete,

[0091] B3: The characters in the left part of the image are incomplete,

[0092] B4: The characters in the right part of the image are incomplete.

[0093] (2) Build a Cycle - GAN model

[0094] Build a Cycle - GAN model for restoring incomplete license plate images. The main structure and loss function of the model are designed as follows:

[0095] 1. Model structure:

[0096] Build two generators G_AB and G_BA, which are used to generate complete license plate images from incomplete license plate images and generate incomplete license plate images from complete license plate images respectively.

[0097] Build two discriminators D A and D B , which are used to distinguish between the generated images and the real images.

[0098] 2. Loss function design:

[0099] 1) Generation loss: The loss of generating a complete license plate image from an incomplete license plate image: Measures the difference between the complete license plate image A' generated by the generator G_BA from the incomplete license plate image B and the real complete license plate image A.

[0100]

[0101] Among them, D Ais a discriminator for discriminating whether the image G generated by the generator G_BA is a real and complete license plate image. BA (B) is a real and complete license plate image.

[0102] Loss for generating an incomplete license plate image from a complete license plate image:

[0103]

[0104] where D B is a discriminator for discriminating whether the image G generated by the generator G_AB is a real and complete license plate image. AB (A) is a real and complete license plate image.

[0105] 2) Cycle consistency loss: Cycle consistency loss is one of the core concepts of Cycle-GAN, which is used to ensure the consistency of the generator in the two-way conversion. It ensures that when converting from one domain to another and then back, the original image can be restored. This loss is designed to prevent mode collapse, that is, the images generated by the generator lack diversity.

[0106] a. Cycle loss for generating a complete license plate image from an incomplete license plate image and then generating an incomplete license plate image again. Measure whether the combination of generators G BA and G AB can generate a complete license plate image A' from the incomplete license plate image B, then generate the original incomplete license plate image B' from A', and ensure the consistency between B' and B:

[0107] L cycle_B = ||B - G AB (G BA (B))||

[0108] b. Cycle loss for generating an incomplete license plate image from a complete license plate image and then generating a complete license plate image again. Measure whether the combination of generators G AB and G BA can generate an incomplete license plate image B' from the complete license plate image A, then generate the original complete license plate image A' from B', and ensure the consistency between A' and A:

[0109] L cycle_A = ||A - G BA (G AB (A))||

[0110] 3) License plate recognition loss:

[0111] The generated license plate image is recognized by the license plate recognition model, and the loss of the recognition result is calculated as an additional loss term:

[0112] L. recognition_B= CTCLoss(OCRNet(G BA (B)), Label_B)

[0113] L. recognition_A = CTCLoss(OCRNet(G AB (A)), Label_A)

[0114] Where OCRNet is a license plate recognition network, Label_B is the manually labeled license plate number of the incomplete license plate B, and OCRNet(G BA (B)) is the license plate recognition result of the generated complete license plate A, and Label_A is the manually labeled license plate number of the complete license plate B, and OCRNet(G AB (B)) is the license plate recognition result of the generated incomplete license plate B. CTCLoss (Connectionist Temporal Classification Loss) is a loss function designed specifically for handling sequence alignment problems, especially suitable for tasks of identifying variable-length sequences, such as speech recognition and character recognition, allowing the prediction results to not be strictly aligned with the target sequence in terms of time steps.

[0115] 4) Total loss function:

[0116] The final total loss function is expressed as:

[0117] L total = L GAB + L GBA + L cycle_A + L cycle_B + L. recognition_B + L. recognition_A

[0118] (3) Handling various situations of incomplete license plate images

[0119] Since there may be various situations for incomplete license plate images, such as incomplete characters at the upper, lower, left, and right edges of the image, the incomplete license plate images are divided into four categories: B1, B2, B3, and B4, representing different situations of incomplete characters in different directions.

[0120] (4) Generation model

[0121] When generating incomplete images from complete images, four generation models G_AB1, G_AB2, G_AB3, and G_AB4 are used, corresponding to generating B1, B2, B3, and B4 respectively. Each model is trained separately to generate images with specific directional losses.

[0122] (5) Restoration model

[0123] When generating a complete license plate image from an incomplete license plate image, since the actual usage situation does not allow the missing direction to be known in advance, a generation model G_BA is used. When training this model, data with missing characters in all directions is included so that the license plate images with missing characters in any direction can be effectively restored in practical applications.

[0124] The present solution has the following effects:

[0125] Detection box restoration ability: Compared with the prior art, the present solution can effectively restore license plate images that are missed due to detection box offset or the NMS (Non-Maximum Suppression) method. While reducing the threshold, the occurrence of false detections is reduced.

[0126] Handling of multi-directional missing characters: Compared with the prior art, the present solution conducts separate training for various incomplete situations of license plate images (partial characters missing at the upper, lower, left, and right edges), enabling the restoration model to more accurately restore the character information missing in different directions, and improving the integrity and credibility of license plate information.

[0127] Further, in the incomplete license plate image dataset, the images are divided into four categories: images incomplete at the upper part, images incomplete at the lower part, images incomplete at the left part, and images incomplete at the right part.

[0128] Further, the generation loss includes: the first loss difference between the complete license plate image generated from the incomplete license plate image and the real complete license plate image. The generation loss also includes: the second loss difference between the incomplete license plate image generated from the complete license plate image.

[0129] Further, the generation loss includes: the first loss difference between the complete license plate image generated from the incomplete license plate image and the real complete license plate image. The generation loss also includes: the second loss difference between the incomplete license plate image generated from the complete license plate image.

[0130] Further, the cycle consistency loss includes: the first consistency value between the incomplete license plate image and the original incomplete license plate image after generating the target complete license plate image from the incomplete license plate image and then generating the original incomplete license plate image from the target complete license plate image;

[0131] The cycle consistency loss also includes: the second consistency value between the complete license plate image and the original complete license plate image after generating the target incomplete license plate image from the complete license plate image and then generating the original complete license plate image from the target incomplete license plate image.

[0132] Further, the license plate loss includes: the first license plate loss obtained by calculating the loss of the first result of recognizing the complete license plate image;

[0133] The license plate loss further includes: a second license plate loss obtained by calculating the loss of the second result of recognizing an incomplete license plate image.

[0134] In the above embodiments, although the steps are numbered S1, S2, etc., they are only specific embodiments given by the present invention. Those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation, which is also within the protection scope of the present invention. It can be understood that in some embodiments, it may include some or all of the above embodiments.

[0135] A recovery system for a damaged license plate image, the specific technical solution is as follows:

[0136] The acquisition module is used to: acquire the real-time monitoring video collected by the monitoring device, and extract the damaged license plate image from the real-time monitoring video;

[0137] The recovery module is used to: input the damaged license plate image into a pre-trained recovery model to obtain a recovered license plate image;

[0138] The pre-trained recovery model is trained in the following manner:

[0139] According to the random segmentation of the complete license plate image dataset, an incomplete license plate image dataset is generated;

[0140] A first generator for generating a complete license plate image from an incomplete license plate image and a second generator for generating an incomplete license plate image from a complete license plate image are constructed;

[0141] According to the first generator and the second generator, combined with the generation loss, the cycle consistency loss, and the license plate loss, the training of the pre-trained recovery model is completed.

[0142] On the basis of the above solution, the present invention can be further improved as follows.

[0143] Further, in the incomplete license plate image dataset, the images are divided into four categories: the upper part of the image is incomplete, the lower part of the image is incomplete, the left part of the image is incomplete, and the right part of the image is incomplete.

[0144] Further, the generation loss includes: the first loss difference between the complete license plate image generated from the incomplete license plate image and the real complete license plate image, and the generation loss further includes: the second loss difference between the incomplete license plate image generated from the complete license plate image.

[0145] Further, the generation loss includes: the first loss difference between the complete license plate image generated from the incomplete license plate image and the real complete license plate image, and the generation loss further includes: the second loss difference between the incomplete license plate image generated from the complete license plate image.

[0146] Further, the cycle consistency loss includes: a first consistency value between an incomplete license plate image and the original incomplete license plate image after generating a target complete license plate image from the incomplete license plate image and then generating the original incomplete license plate image from the target complete license plate image;

[0147] The cycle consistency loss further includes: a second consistency value between a complete license plate image and the original complete license plate image after generating a target incomplete license plate image from the complete license plate image and then generating the original complete license plate image from the target incomplete license plate image.

[0148] Further, the license plate loss includes: a first license plate loss obtained by calculating a loss for a first result of recognizing a complete license plate image;

[0149] The license plate loss further includes: a second license plate loss obtained by calculating a loss for a second result of recognizing an incomplete license plate image.

[0150] It should be noted that the beneficial effects of the system for restoring a damaged license plate image provided in the above embodiments are the same as those of the method for restoring a damaged license plate image provided above, and will not be elaborated here. In addition, when the system provided in the above embodiments implements its functions, only the division of the above functional modules is used as an example for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. Additionally, the system and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments and will not be elaborated here.

[0151] As Figure 2 shown, an electronic device 300 according to an embodiment of the present invention includes a processor 320. The processor 320 is coupled to a memory 310. At least one computer program 330 is stored in the memory 310. The at least one computer program 330 is loaded and executed by the processor 320 so that the electronic device 300 implements any one of the above methods. Specifically:

[0152] The electronic device 300 may vary significantly due to different configurations or performances. It may include one or more processors 320 (Central Processing Units, CPUs) and one or more memories 310. Among them, at least one computer program 330 is stored in the one or more memories 310, and the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the electronic device 300 implements a method for restoring a mutilated license plate image provided in the above embodiments. Of course, the electronic device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input / output interface for input and output. The electronic device 300 may also include other components for implementing the functions of the device, which will not be elaborated here.

[0153] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements any one of the above methods.

[0154] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0155] In an exemplary embodiment, a computer program product or a computer program is also provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes any one of the above methods.

[0156] It should be noted that the terms "first", "second", etc. in the description and claims of this application are used to distinguish similar objects, and do not represent a specific order or sequence. In appropriate cases, the order of use of similar objects may be interchanged so that the embodiments of the present application described here can be implemented in an order other than the illustrated or described order.

[0157] Those skilled in the art of the present technology know that the present invention can be implemented as a system, method, or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, which is generally referred to as "circuit", "module", or "system" in this article. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, which contain computer-readable program code.

[0158] Any combination of one or more computer-readable media can be adopted. The computer-readable media can be computer-readable signal media or computer-readable storage media. The computer-readable storage media can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage media can be any tangible medium that contains or stores a program, which can be used by or in combination with an instruction execution system, apparatus, or device.

[0159] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for restoring an incomplete license plate image, characterized in that: include: Acquire a real-time monitoring image collected by a monitoring device, and extract a damaged license plate image from the real-time monitoring image; Inputting the incomplete license plate image into a pre-trained restoration model to obtain a restored license plate image; The pre-trained recovery model is trained in the following way: According to the random segmentation of the complete license plate image dataset, an incomplete license plate image dataset is generated; Constructing a first generator for generating a complete license plate image from an incomplete license plate image and a second generator for generating an incomplete license plate image from a complete license plate image; According to the first generator and the second generator, the training of the pre-trained recovery model is completed in combination with the generation loss, the cycle consistency loss and the license plate loss; The generation loss includes: a first loss difference between a complete license plate image generated from an incomplete license plate image and a real complete license plate image, and the generation loss also includes: a second loss difference between an incomplete license plate image generated from a complete license plate image; The cycle consistency loss includes: generating a target complete license plate image from an incomplete license plate image, and then generating an original incomplete license plate image from the target complete license plate image, a first consistency value between the incomplete license plate image and the original incomplete license plate image; The cycle consistency loss also includes: generating a target incomplete license plate image from the complete license plate image, and then generating the original complete license plate image from the target incomplete license plate image, a second consistency value between the complete license plate image and the original complete license plate image; The license plate loss includes: a first license plate loss obtained by performing loss calculation on a first result of recognizing a complete license plate image; The license plate loss also includes: a second license plate loss obtained by performing loss calculation on a second result of recognizing the incomplete license plate image.

2. The method for restoring a defective license plate image according to claim 1, characterized in that: In the incomplete license plate image data set, images are divided into four categories: incomplete upper image, incomplete lower image, incomplete left image and incomplete right image.

3. A system for restoring incomplete license plate images, characterized in that: include: The acquisition module is used to: acquire the real-time monitoring image collected by the monitoring device, and extract the incomplete license plate image from the real-time monitoring image; The restoration module is used to: input the incomplete license plate image into a pre-trained restoration model to obtain a restored license plate image; The pre-trained recovery model is trained in the following way: According to the random segmentation of the complete license plate image dataset, an incomplete license plate image dataset is generated; Constructing a first generator for generating a complete license plate image from an incomplete license plate image and a second generator for generating an incomplete license plate image from a complete license plate image; Complete the training of the pre-trained recovery model according to the first generator and the second generator in combination with the generation loss, the cycle consistency loss and the license plate loss; The generation loss includes: a first loss difference between a complete license plate image generated from an incomplete license plate image and a real complete license plate image, and the generation loss also includes: a second loss difference between an incomplete license plate image generated from a complete license plate image; The cycle consistency loss includes: generating a target complete license plate image from an incomplete license plate image, and then generating an original incomplete license plate image from the target complete license plate image, a first consistency value between the incomplete license plate image and the original incomplete license plate image; The cycle consistency loss also includes: generating a target incomplete license plate image from the complete license plate image, and then generating the original complete license plate image from the target incomplete license plate image, a second consistency value between the complete license plate image and the original complete license plate image; The license plate loss includes: a first license plate loss obtained by performing loss calculation on a first result of recognizing a complete license plate image; The license plate loss also includes: a second license plate loss obtained by performing loss calculation on a second result of recognizing the incomplete license plate image.

4. A system for restoring incomplete license plate images according to claim 3, characterized in that: In the incomplete license plate image data set, images are divided into four categories: incomplete upper image, incomplete lower image, incomplete left image and incomplete right image.

5. An electronic device, characterized in that: The electronic device comprises a processor, the processor is coupled to a memory, at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the electronic device implements the method as claimed in claim 1 or 2.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that the computer implements the method according to claim 1 or 2.

Images