Method, system and device for recovering incomplete license plate image and medium

Abstract

The invention discloses an incomplete license plate image recovery method, system and device and a medium, and relates to the technical field of image processing, and the method comprises the steps: extracting an incomplete license plate image from a real-time monitoring image; inputting the incomplete license plate image into a pre-trained recovery model to obtain a recovered license plate image; the pre-trained recovery model is trained through the following steps: generating an incomplete license plate image data set according to random segmentation of a complete license plate image data set; constructing a first generator for generating a complete license plate image from the incomplete license plate image and a second generator for generating an incomplete license plate image from the complete license plate image; and completing training of a pre-trained recovery model according to the first generator and the second generator in combination with the generation loss, the cyclic consistency loss and the license plate loss. According to the method, the complete license plate image data set and the incomplete license plate image data set are combined, so that the accuracy and authenticity of a model output result are greatly improved.

Description

Background Art

[0002] The current license plate recognition mainly uses deep learning methods. First, the target detection algorithm such as the YOLO series or line detection model is used on the image to detect the license plate area. Then, an OCR recognition model such as CRNN or LPRNet model is used on the license plate area to recognize the characters on the license plate. The existing technology has significant shortcomings when dealing with incomplete or blurred license plate areas, resulting in poor recognition results. The specific shortcomings are as follows:

[0003] Detection model limitations: Existing license plate detection models sometimes fail to perfectly detect all license plates. This offset in the detection frame can lead to incomplete license plate images, thus affecting recognition accuracy.

[0004] Image boundary problem: When the license plate is at the edge of the image, some characters may be cropped or blocked, resulting in incomplete license plate information. In this case, it is difficult for the recognition system to accurately obtain the complete license plate information.

[0005] Impact of obstructions: The license plate may be partially obscured by objects such as branches and wires, further increasing the difficulty of recognition.

[0006] For the above three situations, it is difficult for the existing technology to effectively solve the restoration and recognition problems of incomplete license plate images, which affects the overall performance and reliability of the license plate recognition system. Summary of the invention

[0007] The technical problem to be solved by the present invention is to address the deficiencies of the prior art and specifically provide a method, system, device and medium for restoring a defective license plate image, as follows:

[0008] 1) In the first aspect, the present invention provides a method for restoring a defective license plate image, and the specific technical solution is as follows:

[0009] Acquire a real-time monitoring image collected by a monitoring device, and extract a damaged license plate image from the real-time monitoring image;

[0010] Inputting the incomplete license plate image into a pre-trained restoration model to obtain a restored license plate image;

[0011] The pre-trained recovery model is trained in the following way:

[0012] According to the random segmentation of the complete license plate image dataset, an incomplete license plate image dataset is generated;

[0013] Constructing a first generator for generating a complete license plate image from an incomplete license plate image and a second generator for generating an incomplete license plate image from a complete license plate image;

[0014] The training of the pre-trained recovery model is completed according to the first generator and the second generator combined with the generation loss, cycle consistency loss and license plate loss.

[0015] The beneficial effects of the method for restoring a defective license plate image provided by the present invention are as follows:

[0016] During the restoration model training process, this scheme combines the complete license plate image dataset and the incomplete license plate image dataset. In addition, the total loss function includes three items, namely, generation loss, cycle consistency loss, and license plate loss, which greatly improves the accuracy and authenticity of the model output results.

[0017] Based on the above solution, the present invention can also be improved as follows.

[0018] Furthermore, in the incomplete license plate image data set, the images are divided into four categories: incomplete upper image, incomplete lower image, incomplete left image and incomplete right image.

[0019] Further, the generation loss includes: a first loss difference between a complete license plate image generated from an incomplete license plate image and a real complete license plate image, and the generation loss also includes: a second loss difference between an incomplete license plate image generated from a complete license plate image.

[0020] Further, the generation loss includes: a first loss difference between a complete license plate image generated from an incomplete license plate image and a real complete license plate image, and the generation loss also includes: a second loss difference between an incomplete license plate image generated from a complete license plate image.

[0021] Further, the cycle consistency loss includes: a first consistency value between the incomplete license plate image and the original incomplete license plate image after generating a target complete license plate image from the incomplete license plate image and then generating the original incomplete license plate image from the target complete license plate image;

[0022] The cycle consistency loss also includes: generating a target incomplete license plate image from the complete license plate image, and then generating the original complete license plate image from the target incomplete license plate image, a second consistency value between the complete license plate image and the original complete license plate image.

[0023] Further, the license plate loss includes: a first license plate loss obtained by performing loss calculation on a first result of recognizing a complete license plate image;

[0024] The license plate loss also includes: a second license plate loss obtained by performing loss calculation on a second result of recognizing the incomplete license plate image.

[0025] 2) In the second aspect, the present invention also provides a system for restoring an incomplete license plate image, and the specific technical solution is as follows:

[0026] The acquisition module is used to: acquire the real-time monitoring image collected by the monitoring device, and extract the incomplete license plate image from the real-time monitoring image;

[0027] The restoration module is used to: input the incomplete license plate image into a pre-trained restoration model to obtain a restored license plate image;

[0028] The pre-trained recovery model is trained in the following way:

[0029] According to the random segmentation of the complete license plate image dataset, an incomplete license plate image dataset is generated;

[0030] Constructing a first generator for generating a complete license plate image from an incomplete license plate image and a second generator for generating an incomplete license plate image from a complete license plate image;

[0031] The training of the pre-trained recovery model is completed according to the first generator and the second generator combined with the generation loss, cycle consistency loss and license plate loss.

[0032] Based on the above solution, the present invention can also be improved as follows.

[0033] Furthermore, in the incomplete license plate image data set, the images are divided into four categories: incomplete upper image, incomplete lower image, incomplete left image and incomplete right image.

[0034] Further, the generation loss includes: a first loss difference between a complete license plate image generated from an incomplete license plate image and a real complete license plate image, and the generation loss also includes: a second loss difference between an incomplete license plate image generated from a complete license plate image.

[0035] Further, the generation loss includes: a first loss difference between a complete license plate image generated from an incomplete license plate image and a real complete license plate image, and the generation loss also includes: a second loss difference between an incomplete license plate image generated from a complete license plate image.

[0036] Further, the cycle consistency loss includes: a first consistency value between the incomplete license plate image and the original incomplete license plate image after generating a target complete license plate image from the incomplete license plate image and then generating the original incomplete license plate image from the target complete license plate image;

[0037] The cycle consistency loss also includes: generating a target incomplete license plate image from the complete license plate image, and then generating the original complete license plate image from the target incomplete license plate image, a second consistency value between the complete license plate image and the original complete license plate image.

[0038] Further, the license plate loss includes: a first license plate loss obtained by performing loss calculation on a first result of recognizing a complete license plate image;

[0039] The license plate loss also includes: a second license plate loss obtained by performing loss calculation on a second result of recognizing the incomplete license plate image.

[0040] 3) In a third aspect, the present invention further provides an electronic device, comprising a processor, wherein the processor is coupled to a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the electronic device implements any of the above methods.

[0041] 4) In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor so that a computer implements any of the above methods.

[0042] It should be noted that the beneficial effects achieved by the technical solutions of the second to fourth aspects of the present invention and the corresponding possible implementation methods can be found in the above-mentioned technical effects of the first aspect and its corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0044] Figure 1 A schematic diagram of a process for restoring a defective license plate image according to an embodiment of the present invention;

[0045] Figure 2 The present invention is a structural framework diagram of an electronic device. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0047] like Figure 1 As shown, a method for restoring a defective license plate image according to an embodiment of the present invention comprises the following steps:

[0048] S1, obtaining a real-time monitoring image collected by a monitoring device, and extracting a defective license plate image from the real-time monitoring image;

[0049] S2, inputting the incomplete license plate image into a pre-trained restoration model to obtain a restored license plate image;

[0050] The pre-trained recovery model is trained in the following way:

[0051] According to the random segmentation of the complete license plate image dataset, an incomplete license plate image dataset is generated;

[0052] Constructing a first generator for generating a complete license plate image from an incomplete license plate image and a second generator for generating an incomplete license plate image from a complete license plate image;

[0053] The training of the pre-trained recovery model is completed according to the first generator and the second generator combined with the generation loss, the cycle consistency loss and the license plate loss

[0054] The beneficial effects of the method for restoring a defective license plate image provided by the present invention are as follows:

[0055] During the restoration model training process, this scheme combines the complete license plate image dataset and the incomplete license plate image dataset. In addition, the total loss function includes three items, namely, generation loss, cycle consistency loss, and license plate loss, which greatly improves the accuracy and authenticity of the model output results.

[0056] S1, obtaining a real-time monitoring image collected by a monitoring device, and extracting a damaged license plate image from the real-time monitoring image.

[0057] The monitoring device can be: a camera at any traffic intersection, or a road monitoring camera.

[0058] The incomplete license plate image refers to: the image corresponding to the license plate blocked by foreign objects or the image of the license plate information at the boundary of the surveillance image.

[0059] The process of extracting incomplete license plate images from real-time monitoring images is as follows:

[0060] Perform noise reduction (noise filtering) and license plate location processing on each frame of surveillance image to determine the incomplete license plate image in the frame of surveillance image;

[0061] Among them, the noise reduction processing includes:

[0062] Automatic white balance, automatic exposure, gamma correction, edge enhancement, and contrast adjustment processing.

[0063] The process of automatic white balance is as follows: according to the light conditions collected by the white balance sensor installed in the monitoring device, the color temperature value and chromaticity distance of the object in the current frame of the monitoring image are determined, the adjustment value corresponding to the current frame is determined according to the color temperature value and chromaticity distance, and adjustments are made based on the adjustment value.

[0064] Gamma correction includes: RGB brightness calculation and RGB darkness calculation, judging whether the current frame monitoring image needs RGB brightness calculation or RGB darkness calculation, and performing corresponding processing according to the judgment result;

[0065] The RGB brightness calculation formula is:

[0066]

[0067] The RGB darkness calculation formula is:

[0068]

[0069] The process required to determine the current frame surveillance image is:

[0070] The contrast and brightness of the current frame surveillance image are judged. When both the contrast and the brightness exceed the upper limit of the preset contrast range and the upper limit of the preset brightness range, it is determined that the current frame surveillance image needs to perform RGB darkness calculation;

[0071] When both the contrast and the brightness are lower than the preset contrast range lower limit and the preset brightness range lower limit, it is determined that the current frame monitoring image needs to perform RGB brightness calculation.

[0072] Contrast refers to the difference between the brightest and darkest parts of the various colors in the image. The higher the contrast, the more dazzling and bright the image will feel. Conversely, the smaller the contrast, the less obvious the change. High contrast means that the image may be brighter, while low contrast means that the image may be darker.

[0073] Brightness refers to: For grayscale images, brightness is directly related to the grayscale value. The higher the grayscale value, the brighter the image. In color images, adjusting the brightness of an image affects the brightness of the entire image, not just a specific area. If you need to adjust the brightness of an image, you can do so by adjusting the brightness.

[0074] Edge enhancement is achieved through preset algorithms, including:

[0075] The feature extraction of any frame of monitoring image is performed through the sliding window algorithm to obtain a non-edge feature image set. The non-edge suppression operator is used to suppress non-correlated features on the non-edge feature image set to obtain a suppressed feature image corresponding to each feature image in the non-edge feature image set. The suppressed feature image is used to replace the corresponding feature image to obtain the initial feature map corresponding to any frame of monitoring image. The East China window algorithm is used to extract features from the initial feature map to obtain an edge feature image set. The edge enhancement operator is used to enhance related features on the edge feature image set to obtain an enhanced feature image corresponding to each feature image in the edge feature image set. The enhanced feature image is used to replace the corresponding feature image to obtain an edge enhanced feature map corresponding to the initial feature map. Where:

[0076] The non-edge suppression operator determines the non-edge sub-blocks in the training set, constructs a non-edge typical feature sub-block set based on the non-edge sub-blocks, and performs matrix solution on the image in the non-edge typical feature sub-block set to obtain the non-edge suppression operator.

[0077] The edge enhancement operator determines the edge sub-blocks in the training set, constructs the edge typical feature sub-block set based on the edge sub-blocks, performs matrix solution on the image in the edge typical feature sub-block set, and obtains the edge enhancement operator.

[0078] License plate location: Through the preset model (the preset model is trained by the texture feature analysis and location algorithm), any frame of surveillance image is analyzed and processed, that is, row and column scanning is performed on the grayscale image after image preprocessing, and the candidate area containing the license plate line segment in the column direction is determined by row scanning, the starting row coordinates and height of the area are determined, and then the area is scanned by column to determine its column coordinates and width, thereby determining a license plate area. The license plate area is the incomplete license plate image.

[0079] Character segmentation: After locating the license plate area in the image, the character area is further accurately located through grayscale, grayscale stretching, binarization, and edge processing. Then, a dynamic template method is proposed for character segmentation based on the character size characteristics, and the character size is normalized.

[0080] Character recognition: The segmented characters are scaled and features extracted to obtain the expression of specific characters. Then, through classification and classification rules, they are matched and identified with the standard character expressions in the character template database to recognize the input character image.

[0081] S2, inputting the incomplete license plate image into a pre-trained restoration model to obtain a restored license plate image.

[0082] Among them, the training process of the pre-trained recovery model is as follows:

[0083] (1) Create a dataset

[0084] First, you need to create a dataset containing complete license plate images and incomplete license plate images. The specific steps are as follows:

[0085] 1. Complete license plate image dataset (folder A):

[0086] Collect a dataset of complete license plate images. These images should be as diverse as possible to cover different license plate styles and backgrounds.

[0087] 2. Incomplete license plate image dataset (folder B):

[0088] Incomplete license plate images are randomly generated from complete license plate images. Incomplete license plate images are divided into four categories according to the different directions of missing characters: B1, B2, B3, and B4, representing the following situations respectively:

[0089] B1: The characters on the upper part of the image are incomplete.

[0090] B2: The characters at the bottom of the image are incomplete.

[0091] B3: The characters on the left side of the image are incomplete.

[0092] B4: The characters on the right side of the image are incomplete.

[0093] (2) Building the Cycle-GAN model

[0094] A Cycle-GAN model is constructed for incomplete license plate image restoration. The main structure and loss function of the model are designed as follows:

[0095] 1. Model structure:

[0096] Construct two generators G_AB and G_BA, which are used to generate a complete license plate image from an incomplete license plate image and to generate an incomplete license plate image from a complete license plate image.

[0097] Construct two discriminators D A and D B , which is used to distinguish generated images from real images.

[0098] 2. Loss function design:

[0099] 1) Generation loss: The loss of generating a complete license plate image from an incomplete license plate image: measures the difference between the complete license plate image A' generated by the generator G_BA from the incomplete license plate image B and the real complete license plate image A.

[0100]

[0101] Among them, D Ais a discriminator, used to discriminate the image G generated by the generator G_BA BA (B) Whether it is a real complete license plate image.

[0102] The loss of generating an incomplete license plate image from a complete license plate image:

[0103]

[0104] Among them, D B is a discriminator used to discriminate the image G generated by the generator G_AB AB (A) Whether it is a real complete license plate image.

[0105] 2) Cycle consistency loss: Cycle consistency loss is one of the core concepts of Cycle-GAN and is used to ensure the consistency of the generator in bidirectional conversion. It ensures that the original image can be restored when converting from one domain to another and back. This loss is designed to prevent mode collapse, that is, the lack of diversity in the images generated by the generator.

[0106] a. The cycle loss of generating a complete license plate image from an incomplete license plate image and then generating an incomplete license plate image. BA and G AB Can the combination of generate a complete license plate image A' from an incomplete license plate image B, and then generate the original incomplete license plate image B' from A', and ensure the consistency of B' with B:

[0107] L cycle_B =||BG AB (G BA (B))||

[0108] b. The cycle loss of generating an incomplete license plate image from a complete license plate image and then generating a complete license plate image. AB and G BA Can the combination of generate an incomplete license plate image B' from the complete license plate image A, and then generate the original complete license plate image A' from B', and ensure the consistency of A' and A:

[0109] L cycle_A =||AG BA (G AB (A))||

[0110] 3) License plate recognition loss:

[0111] The generated license plate image is recognized by the license plate recognition model, and the loss of the recognition result is calculated as an additional loss term:

[0112] L. recognition_B=CTCLoss(OCRNet(G BA (B)),Label_B)

[0113] L. recognition_A =CTCLoss(OCRNet(G AB (A)),Label_A)

[0114] Among them, OCRNet is the license plate recognition network, Label_B is the manually labeled license plate number of the incomplete license plate B, and OCRNet (G BA (B)) is the license plate recognition result of the generated complete license plate A, Label_A is the manually labeled license plate number of the complete license plate B, OCRNet (G AB (B)) is the license plate recognition result of the generated incomplete license plate B. CTCLoss (Connectionist Temporal Classification Loss) is a loss function designed for handling sequence alignment problems, especially for tasks of recognizing indefinite length sequences, such as speech recognition and character recognition, allowing the prediction results to be not strictly aligned with the target sequence in terms of time steps.

[0115] 4) Total loss function:

[0116] The final total loss function is expressed as:

[0117] L total =L GAB +L GBA +L cycle_A +L cycle_B +L. recognition_B +L. recognition_A

[0118] (3) Various situations of handling incomplete license plate images

[0119] Since there may be many situations of incomplete license plate images, such as incomplete characters at the upper, lower, left and right edges of the image, the incomplete license plate images are divided into four categories: B1, B2, B3 and B4, representing the incomplete characters in different directions.

[0120] (4) Generative Model

[0121] When generating an incomplete image from a complete image, four generative models G_AB1, G_AB2, G_AB3, and G_AB4 are used to generate B1, B2, B3, and B4 respectively. Each model is trained separately to generate images with missing information in a specific direction.

[0122] (5) Restoration model

[0123] When generating a complete license plate image from an incomplete license plate image, since the missing direction cannot be known in advance in actual use, a generative model G_BA is used. The model contains data on all missing directions during training, so that it can effectively restore license plate images with any missing direction in practical applications.

[0124] This solution has the following effects:

[0125] Detection frame restoration capability: Compared with the existing technology, this solution can effectively restore the license plate image that is missed due to detection frame offset or NMS (non-maximum suppression) method, while lowering the threshold and reducing the occurrence of false detection.

[0126] Multi-directional missing information processing: Compared with the existing technology, this solution conducts training for various incomplete situations of license plate images (incomplete characters on the upper, lower, left and right edges), so that the restoration model can more accurately restore the missing character information in different directions, thereby improving the integrity and credibility of the license plate information.

[0127] Furthermore, in the incomplete license plate image data set, the images are divided into four categories: incomplete upper image, incomplete lower image, incomplete left image and incomplete right image.

[0128] Further, the generation loss includes: a first loss difference between a complete license plate image generated from an incomplete license plate image and a real complete license plate image, and the generation loss also includes: a second loss difference between an incomplete license plate image generated from a complete license plate image.

[0129] Further, the generation loss includes: a first loss difference between a complete license plate image generated from an incomplete license plate image and a real complete license plate image, and the generation loss also includes: a second loss difference between an incomplete license plate image generated from a complete license plate image.

[0130] Further, the cycle consistency loss includes: a first consistency value between the incomplete license plate image and the original incomplete license plate image after generating a target complete license plate image from the incomplete license plate image and then generating the original incomplete license plate image from the target complete license plate image;

[0131] The cycle consistency loss also includes: generating a target incomplete license plate image from the complete license plate image, and then generating the original complete license plate image from the target incomplete license plate image, a second consistency value between the complete license plate image and the original complete license plate image.

[0132] Further, the license plate loss includes: a first license plate loss obtained by performing loss calculation on a first result of recognizing a complete license plate image;

[0133] The license plate loss also includes: a second license plate loss obtained by performing loss calculation on a second result of recognizing the incomplete license plate image.

[0134] In the above embodiments, although the steps are numbered S1, S2, etc., these are only specific embodiments given by the present invention. Those skilled in the art may adjust the execution order of S1, S2, etc. according to actual conditions, which is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0135] A system for restoring incomplete license plate images, the specific technical solution is as follows:

[0136] The acquisition module is used to: acquire the real-time monitoring image collected by the monitoring device, and extract the incomplete license plate image from the real-time monitoring image;

[0137] The restoration module is used to: input the incomplete license plate image into a pre-trained restoration model to obtain a restored license plate image;

[0138] The pre-trained recovery model is trained in the following way:

[0139] According to the random segmentation of the complete license plate image dataset, an incomplete license plate image dataset is generated;

[0140] Constructing a first generator for generating a complete license plate image from an incomplete license plate image and a second generator for generating an incomplete license plate image from a complete license plate image;

[0141] The training of the pre-trained recovery model is completed according to the first generator and the second generator combined with the generation loss, cycle consistency loss and license plate loss.

[0142] Based on the above solution, the present invention can also be improved as follows.

[0143] Furthermore, in the incomplete license plate image data set, the images are divided into four categories: incomplete upper image, incomplete lower image, incomplete left image and incomplete right image.

[0144] Further, the generation loss includes: a first loss difference between a complete license plate image generated from an incomplete license plate image and a real complete license plate image, and the generation loss also includes: a second loss difference between an incomplete license plate image generated from a complete license plate image.

[0145] Further, the generation loss includes: a first loss difference between a complete license plate image generated from an incomplete license plate image and a real complete license plate image, and the generation loss also includes: a second loss difference between an incomplete license plate image generated from a complete license plate image.

[0146] Further, the cycle consistency loss includes: a first consistency value between the incomplete license plate image and the original incomplete license plate image after generating a target complete license plate image from the incomplete license plate image and then generating the original incomplete license plate image from the target complete license plate image;

[0147] The cycle consistency loss also includes: generating a target incomplete license plate image from the complete license plate image, and then generating the original complete license plate image from the target incomplete license plate image, a second consistency value between the complete license plate image and the original complete license plate image.

[0148] Further, the license plate loss includes: a first license plate loss obtained by performing loss calculation on a first result of recognizing a complete license plate image;

[0149] The license plate loss also includes: a second license plate loss obtained by performing loss calculation on a second result of recognizing the incomplete license plate image.

[0150] It should be noted that the beneficial effects of the incomplete license plate image recovery system provided in the above embodiment are the same as the beneficial effects of the incomplete license plate image recovery method provided in the above embodiment, which will not be described in detail here. In addition, when the system provided in the above embodiment realizes its functions, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be described in detail here.

[0151] like Figure 2 As shown, an electronic device 300 according to an embodiment of the present invention includes a processor 320, the processor 320 is coupled to a memory 310, and the memory 310 stores at least one computer program 330, and the at least one computer program 330 is loaded and executed by the processor 320, so that the electronic device 300 implements any of the above methods, specifically:

[0152] The electronic device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors 320 (Central Processing Units, CPU) and one or more memories 310, wherein the one or more memories 310 store at least one computer program 330, and the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the electronic device 300 implements a method for restoring a damaged license plate image provided in the above embodiment. Of course, the electronic device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the electronic device 300 may also include other components for realizing the functions of the device, which will not be described in detail here.

[0153] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements any one of the above methods.

[0154] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0155] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs any of the above methods.

[0156] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and represent the definition of a specific order or sequence. The order of use of similar objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0157] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0158] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0159] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for restoring incomplete license plate images. It is characterized in that include: Acquire a real-time monitoring image collected by a monitoring device, and extract a damaged license plate image from the real-time monitoring image; Inputting the incomplete license plate image into a pre-trained restoration model to obtain a restored license plate image; The pre-trained recovery model is trained in the following way: According to the random segmentation of the complete license plate image dataset, an incomplete license plate image dataset is generated; Constructing a first generator for generating a complete license plate image from an incomplete license plate image and a second generator for generating an incomplete license plate image from a complete license plate image; According to the first generator and the second generator, the training of the pre-trained recovery model is completed by combining the generation loss, the cycle consistency loss and the license plate loss.

2. A method for restoring a defective license plate image according to claim 1, It is characterized in that In the incomplete license plate image data set, images are divided into four categories: incomplete upper image, incomplete lower image, incomplete left image and incomplete right image.

3. The method for restoring a defective license plate image according to claim 1, It is characterized in that The generation loss includes: a first loss difference between a complete license plate image generated from an incomplete license plate image and a real complete license plate image, and the generation loss also includes: a second loss difference between an incomplete license plate image generated from a complete license plate image.

4. The method for restoring a defective license plate image according to claim 1, It is characterized in that The cycle consistency loss includes: generating a target complete license plate image from an incomplete license plate image, and then generating an original incomplete license plate image from the target complete license plate image, a first consistency value between the incomplete license plate image and the original incomplete license plate image; The cycle consistency loss also includes: generating a target incomplete license plate image from the complete license plate image, and then generating the original complete license plate image from the target incomplete license plate image, a second consistency value between the complete license plate image and the original complete license plate image.

5. The method for restoring a defective license plate image according to claim 1, It is characterized in that The license plate loss includes: a first license plate loss obtained by performing loss calculation on a first result of recognizing a complete license plate image; The license plate loss also includes: a second license plate loss obtained by performing loss calculation on a second result of recognizing the incomplete license plate image.

6. A system for restoring incomplete license plate images. It is characterized in that include: The acquisition module is used to: acquire the real-time monitoring image collected by the monitoring device, and extract the incomplete license plate image from the real-time monitoring image; The restoration module is used to: input the incomplete license plate image into a pre-trained restoration model to obtain a restored license plate image; The pre-trained recovery model is trained in the following way: According to the random segmentation of the complete license plate image dataset, an incomplete license plate image dataset is generated; Constructing a first generator for generating a complete license plate image from an incomplete license plate image and a second generator for generating an incomplete license plate image from a complete license plate image; The training of the pre-trained recovery model is completed according to the first generator and the second generator combined with the generation loss, cycle consistency loss and license plate loss.

7. A system for restoring an incomplete license plate image according to claim 6, It is characterized in that In the incomplete license plate image data set, images are divided into four categories: incomplete upper image, incomplete lower image, incomplete left image and incomplete right image.

8. A system for restoring an incomplete license plate image according to claim 6, It is characterized in that The generation loss includes: a first loss difference between a complete license plate image generated from an incomplete license plate image and a real complete license plate image, and the generation loss also includes: a second loss difference between an incomplete license plate image generated from a complete license plate image.

9. An electronic device, It is characterized in that The electronic device comprises a processor, the processor is coupled to a memory, at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the electronic device implements the method according to any one of claims 1 to 5.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements the method according to any one of claims 1 to 5.

Images