A method for generating a sprayed license plate image and related methods and devices

Abstract

This invention provides a method for generating painted license plate images, as well as related methods and equipment. The method includes: acquiring a license plate content template image and a painted license plate background image; extracting license plate content features from the license plate content template image and extracting painted license plate background features from the painted license plate background image; and generating a painted license plate image based on the license plate content features and the painted license plate background features. The method provided by this invention can automatically generate painted license plates based on the license plate content template and the painted license plate background. Compared to collecting painted license plates from actual scenes, this method significantly reduces manpower and resource consumption.

Description

Technical Field

[0001] This invention relates to the field of license plate recognition technology, and in particular to a method for generating painted license plate images and related methods and equipment. Background Technology

[0002] In recent years, license plate recognition has played an important role in intelligent transportation systems, making a significant contribution to maintaining traffic safety, intelligent vehicle management, and improving urban security.

[0003] However, in some complex environments, such as low light, obstruction, or blurriness, the recognition rate of standard license plates drops significantly. To overcome this difficulty, Article 13 of the "Regulations for the Implementation of the Road Traffic Safety Law of the People's Republic of China" stipulates that "the rear of the body or cargo compartment of heavy and medium-sized trucks and their trailers, tractors and their trailers shall be painted with enlarged license plate numbers, and the characters shall be upright and clear." In some scenarios, painted license plates are easier to recognize than standard license plates.

[0004] The current approach to achieving painted license plate recognition involves acquiring a large number of painted license plate images, training a license plate recognition model using these images, and then using the trained model to recognize the painted license plate images to be identified. Currently, acquiring a large number of painted license plate images involves collecting them from real-world scenarios. However, collecting these images from real-world scenarios incurs a significant manpower and resource consumption. Summary of the Invention

[0005] In view of this, the present invention provides a method for generating painted license plate images, as well as related methods and equipment, to solve the problem that current methods of obtaining painted license plates cause a huge manpower burden and resource consumption. The technical solution is as follows:

[0006] A method for generating images of painted license plates includes:

[0007] Obtain the license plate content template image and the license plate background image for spraying;

[0008] Extract license plate content features from the license plate content template image, and extract sprayed license plate background features from the sprayed license plate background image;

[0009] Based on the license plate content features and the background features of the painted license plate, a painted license plate image is generated.

[0010] Optionally, obtaining the license plate content template image and the sprayed license plate background image includes:

[0011] Obtain license plate content template images from a content dataset containing multiple license plate content template images with different character attributes;

[0012] The background images of painted license plates are obtained from a background dataset that includes multiple background images of painted license plates, wherein the multiple background images of painted license plates are extracted from multiple real painted license plate images.

[0013] Optionally, the step of extracting license plate content features from the license plate content template image and extracting sprayed license plate background features from the sprayed license plate background image includes:

[0014] Based on a pre-built image generation model for painted license plates, license plate content features are extracted from the license plate content template image, and background features of the painted license plate are extracted from the background image of the painted license plate.

[0015] The step of generating a painted license plate image based on the license plate content features and the painted license plate background features includes:

[0016] The license plate content features are fused with the painted license plate background features to obtain the fused features;

[0017] Based on the aforementioned painted license plate image generation model, and using the fused features as a basis, a painted license plate image is generated.

[0018] Optionally, the license plate image generation model uses the first generator network in the trained adversarial generative network;

[0019] The training data for the Generative Adversarial Network includes: training license plate content template images and training painted license plate background images;

[0020] The training objective of the Generative Adversarial Network (GAN) includes: enabling the discriminator network in the GAN to determine the painted license plate image generated by the first generator network based on the training license plate content template image and the training painted license plate background image as a real painted license plate image.

[0021] Optionally, the training objectives for adversarial generative networks may also include:

[0022] The structural information of the characters in the painted license plate image generated by the first generation network based on the training license plate content template image and the training painted license plate background image is made to be consistent with the structural information of the characters in the training license plate content template image.

[0023] Optionally, the training objectives for adversarial generative networks may also include:

[0024] The license plate content in the sprayed license plate image generated by the first generation network based on the training license plate content template image and the training sprayed license plate background image is made to be consistent with the license plate content in the training license plate content template image.

[0025] Optionally, the training data may also include: real painted license plate images;

[0026] The training process for generative adversarial networks includes:

[0027] The training license plate content template image and the training painted license plate background image are input into the first generation network to obtain the painted license plate image generated by the first generation network based on the input.

[0028] The painted license plate image generated by the first generator network and the real painted license plate image are respectively input into the discriminant network in the adversarial generator network to obtain the probability that the discriminant network will identify the input image as a real painted license plate;

[0029] Determine the first prediction loss based on the obtained probabilities;

[0030] The parameters of the adversarial generative network are updated based on the first predicted loss.

[0031] Optionally, the training process for adversarial generative networks may also include:

[0032] Based on the training license plate content template image and the training painted license plate background image, character structure supervision information is constructed, wherein the character structure supervision information is used to constrain the character structure of the painted license plate generated by the first generation network;

[0033] Based on the character structure supervision information and the painted license plate image generated by the first generation network, the second prediction loss is determined;

[0034] The step of updating the parameters of the adversarial generative network based on the first predicted loss includes:

[0035] The parameters of the adversarial generative network are updated based on the first prediction loss and the second prediction loss.

[0036] Optionally, the step of constructing character structure supervision information based on the training license plate content template image and the training painted license plate background image includes:

[0037] The background of the training license plate content template image is processed to black and the characters are processed to white to obtain the processed image;

[0038] The processed image is added to the corresponding pixels in the training spray-painted license plate background image, and the result is used as character structure supervision information.

[0039] Optionally, the adversarial generative network may further include a second generative network, and the training data may further include a training template background image;

[0040] The training process for adversarial generative networks also includes:

[0041] The training license plate content template image and the training template background image are input into the second generation network to obtain the license plate template image generated by the second generation network based on the input;

[0042] The third prediction loss is determined based on the training license plate content template and the license plate template image;

[0043] Based on the painted license plate image generated by the second generator network and the first generator network and the training template background image, the license plate content template image is reconstructed, and the fourth prediction loss is determined according to the reconstructed license plate content template image and the training license plate content template image.

[0044] The step of updating the parameters of the adversarial generative network based on the first predicted loss and the second predicted loss includes:

[0045] The parameters of the adversarial generative network are updated based on the first prediction loss, the second prediction loss, the third prediction loss, and the fourth prediction loss.

[0046] A method for constructing a license plate recognition model, comprising:

[0047] Obtain a painted license plate dataset, wherein the painted license plate dataset includes multiple painted license plate images generated using any of the painted license plate image generation methods described above;

[0048] A license plate recognition model is trained based on the painted license plate images in the painted license plate dataset, and the trained license plate recognition model is used as the constructed painted license plate recognition model.

[0049] A method for recognizing painted license plates includes:

[0050] Acquire the image of the painted license plate to be identified;

[0051] The painted license plate image is input into the painted license plate recognition model constructed using the above-described painted license plate recognition model construction method to obtain the painted license plate recognition result output by the painted license plate recognition model.

[0052] A device for generating painted license plate images includes: an image acquisition module, a content feature extraction module, a background feature extraction module, and a painted license plate image generation module;

[0053] The image acquisition module is used to acquire the license plate content template image and the sprayed license plate background image;

[0054] The content feature extraction module is used to extract license plate content features from the license plate content template image;

[0055] The background feature extraction module is used to extract the background features of the painted license plate from the painted license plate background image;

[0056] The painted license plate image generation module is used to generate a painted license plate image based on the license plate content features and the painted license plate background features.

[0057] A device for generating license plate images by spraying paint includes: a memory and a processor;

[0058] The memory is used to store programs;

[0059] The processor is used to execute the program to implement each step of the above-described method for generating license plate images.

[0060] A readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for generating painted license plate images as described above.

[0061] The method for generating painted license plate images provided by this invention first acquires a license plate content template image and a painted license plate background image. Then, it extracts license plate content features from the license plate content template image and extracts painted license plate background features from the painted license plate background image. Finally, based on the license plate content features and painted license plate background features, it generates a painted license plate image. This method can automatically generate painted license plate images based on a license plate content template and a painted license plate background. Compared to collecting painted license plates from actual scenes, this method significantly reduces manpower and resource consumption. Based on the above method, this invention also provides a method for constructing a painted license plate recognition model. This method can construct a high-performance painted license plate recognition model. Furthermore, this invention provides a method for recognizing painted license plates. Using this method to recognize painted license plate images yields good recognition results. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0063] Figure 1 This is a flowchart illustrating the method for generating painted license plate images provided in an embodiment of the present invention.

[0064] Figure 2 This is a schematic diagram of multiple license plate content template images with different character attributes provided in an embodiment of the present invention;

[0065] Figure 3 A schematic diagram illustrating the process of generating a painted license plate image based on a pre-built painted license plate image generation model, provided in an embodiment of the present invention;

[0066] Figure 4 This is a schematic diagram of a process for training an adversarial generative network according to an embodiment of the present invention;

[0067] Figure 5 A schematic diagram of the structure of an adversarial generative network provided in an embodiment of the present invention;

[0068] Figure 6 This is another schematic diagram of the training process for the generative adversarial network provided in an embodiment of the present invention;

[0069] Figure 7 This is a schematic diagram of the structure of the spray-painted license plate image generation device provided in an embodiment of the present invention;

[0070] Figure 8 This is a schematic diagram of the structure of the license plate image generation device provided in an embodiment of the present invention. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] To address the significant manpower and resource consumption associated with collecting images of painted license plates from real-world scenarios, the applicant conceived of an automated method for generating such images. To this end, the applicant conducted research and discovered that while several license plate image generation methods exist, these are all designed for generating standard license plate images. Since the characteristics of standard license plates differ from those of painted license plates, existing methods are not suitable for generating painted license plate images. Therefore, the applicant sought to propose a method specifically for generating painted license plate images and conducted research in this area. Through continuous research, a method for generating painted license plate images was ultimately proposed, which can produce high-quality painted license plates.

[0073] The method for generating painted license plate images provided by this invention can be applied to electronic devices with data processing capabilities. These electronic devices can be user-side terminals, such as smartphones, PCs, laptops, tablets, and PDAs. The electronic device can also be a server, which can be a single server, a server cluster consisting of multiple servers, or a cloud computing server center. The server may include a processor, memory, and network interfaces. The aforementioned terminals or servers can generate painted license plates according to the method provided by this invention. Those skilled in the art should understand that the aforementioned terminals and servers are merely examples. Other existing or future terminals or servers that are applicable to this invention should also be included within the scope of protection of this invention, and are hereby incorporated by reference.

[0074] The following embodiments will describe the method for generating painted license plate images provided by the present invention.

[0075] Please see Figure 1 The diagram illustrates a flowchart of a method for generating license plate images by spraying paint according to an embodiment of the present invention. This method may include:

[0076] Step S101: Obtain the license plate content template image and the license plate background image.

[0077] The license plate content template image and the license plate background image are used to generate the license plate image. To facilitate the subsequent generation of the license plate, the license plate content template image and the license plate background image are preferably images of the same size.

[0078] Considering that painted license plates in real-world scenarios do not have fixed background styles and font styles, meaning that painted license plates in real-world scenarios have a high degree of diversity, this embodiment pre-constructs a content dataset and a background dataset. The content dataset includes multiple license plate content template images with different character attributes (i.e., the content dataset includes diverse license plate content template images), and the background dataset includes multiple painted license plate background images (i.e., the background dataset includes diverse painted license plate backgrounds). When generating painted license plate images, the license plate content template images are obtained from the content dataset, and the painted license plate background images are obtained from the background dataset.

[0079] The character attributes can include, but are not limited to, character content, character size, character position, character tilt angle, character spacing, character font style, etc. It should be noted that if at least one character attribute of two license plate content template images is different, then the character attributes of the two license plate content template images are considered to be different.

[0080] Optionally, multiple license plate content template images with different character attributes can be generated via script. Figure 2The diagram illustrates multiple license plate content template images with different character attributes. When generating a license plate content template image, it is possible to randomly generate the image based on a set range of character parameters and / or multiple sets of character parameters. For example, if the character attributes include attributes A, B, and C, multiple sets of character parameters can be set for attribute A, a range of character parameters can be set for attribute B, and a range of character parameters can be set for attribute C. When generating the license plate content template image, one set of character parameters can be randomly selected from the multiple sets of character parameters set for attribute A, one character parameter can be randomly selected from the range of character parameters set for attribute B, and one character parameter can be randomly selected from the range of character parameters set for attribute C. The license plate content template image is then generated based on the selected character parameters. This method can generate diverse license plate content template images.

[0081] Optionally, multiple real painted license plate images (real painted license plate images refer to images of painted license plates in real scenes) can be acquired, and painted license plate background images can be extracted from multiple real painted license plate images to obtain multiple painted license plate background images. For example, a large number (e.g., two thousand) of real painted license plate images can be collected from surveillance cameras in real scenes, and then painted license plate background images can be extracted from the collected real painted license plate images.

[0082] It should be noted that although this invention also requires collecting real painted license plate images, the number required is significantly less than that needed to train a license plate recognition model. Generally, training a license plate recognition model requires approximately 100,000 images, while this invention requires approximately 10,000 or even fewer. Furthermore, collecting the real painted license plate images required for this invention is far less difficult than collecting them for training a license plate recognition model. For example, training a license plate recognition model requires collecting real painted license plate images from multiple locations, while this invention only requires collecting images from one location (e.g., the local area). License plates from other locations can be generated using the method provided by this invention.

[0083] Step S102: Extract license plate content features from the license plate content template image and extract sprayed license plate background features from the sprayed license plate background image.

[0084] Among them, the license plate content features are features that can characterize the license plate content in the license plate content template image, and the license plate background features are features that can characterize the background of the sprayed license plate.

[0085] Step S103: Generate an image of the painted license plate based on the license plate content features and the background features of the painted license plate.

[0086] Specifically, the features of the license plate content can be fused with the features of the painted license plate background, and the painted license plate image can be generated based on the fused features.

[0087] The method for generating painted license plate images provided in this invention first acquires a license plate content template image and a painted license plate background image. Then, it extracts license plate content features from the license plate content template image and extracts painted license plate background feature images from the painted license plate background image. Finally, based on the license plate content features and painted license plate background features, it generates a painted license plate image. This method can automatically generate painted license plate images based on the license plate content template image and the painted license plate background image. Compared to collecting painted license plate images from actual scenes, this method significantly reduces the human burden and resource consumption associated with acquiring painted license plate images. Furthermore, since this invention constructs a content dataset including diverse license plate content template images and a background dataset including diverse painted license plate background images, diverse painted license plate images can be generated based on the content dataset and background dataset and the method for generating painted license plate images provided by this invention (the license plate content template images are diverse, and the painted license plate background images are diverse, which means that the combination of license plate content template images and painted license plate background images is more diverse, that is, the generated painted license plate images are more diverse).

[0088] In one possible implementation, the painted license plate image can be generated based on a pre-built painted license plate image generation model. The following section will introduce the implementation process of generating the painted license plate image based on the pre-built painted license plate image generation model.

[0089] Please see Figure 3 The diagram illustrates a process for generating painted license plate images based on a pre-built painted license plate image generation model, which may include:

[0090] Step S301: Obtain the license plate content template image and the license plate background image.

[0091] Specifically, license plate content template images can be obtained from a content dataset containing multiple license plate content template images with different character attributes, and painted license plate background images can be obtained from a background dataset containing various painted license plate background images. The construction methods for the content dataset and background dataset can be found in the relevant parts of the above embodiments, and will not be repeated here.

[0092] Step S302: Based on the pre-built sprayed license plate image generation model, extract license plate content features from the license plate content template image and extract sprayed license plate background features from the sprayed license plate background image.

[0093] Specifically, the license plate image generation model can include two feature extraction parts: a license plate content feature extraction part and a license plate background feature extraction part. The license plate content feature extraction part can be used to extract license plate content features from the obtained license plate content template image, and the license plate background feature extraction part can be used to extract license plate background features from the obtained license plate background image.

[0094] Step S303: Fuse the license plate content features with the painted license plate background features to obtain the fused features.

[0095] Optionally, the method for fusing the license plate content features with the painted license plate background features can be to concatenate the license plate content features with the painted license plate background features (i.e., perform a concat operation on the license plate content features and the painted license plate background features). For example, if both the license plate content features and the painted license plate background features are features with dimensions of 64×256×256, then fusing the license plate content features with the painted license plate background features will result in a feature of 128×256×256.

[0096] Step S304: Based on the painted license plate image generation model, generate a painted license plate image using the fused features.

[0097] In addition to the aforementioned license plate content feature extraction and license plate background feature extraction parts, the painted license plate image generation model also includes a painted license plate image generation part. The fused features are input into the painted license plate image generation part of the model, and the painted license plate image generation part generates a painted license plate image based on the input features.

[0098] The following section describes the process of constructing a model for generating painted license plate images.

[0099] In one possible implementation, the license plate image generation model can employ the generative network within a trained adversarial generative network.

[0100] The following section focuses on the training process of Generative Adversarial Networks (GANs). There are several ways to train a GAN; this embodiment provides two optional implementation methods.

[0101] First implementation method:

[0102] Generative Adversarial Networks (GANs) consist of a generator network and a discriminator network. The training data for GANs includes: training license plate content template images, training painted license plate background images, and real painted license plate images. The training objective of GANs is to enable the discriminator network to classify painted license plate images generated by the generator network based on the training license plate content template images and training painted license plate background images as real painted license plate images.

[0103] Please see Figure 4 The diagram illustrates the process of training an adversarial generative network, which may include:

[0104] Step S401: Input the training license plate content template image and the training painted license plate background image into the generator network to obtain the painted license plate image generated by the generator network based on the input.

[0105] Optionally, the generative network may include a first encoder, a second encoder, and a generator. After obtaining the training license plate content template image and the training painted license plate background image, the training license plate content template image is input into the first encoder for encoding (the process of the first encoder encoding the training license plate content template image is the process of extracting license plate content features from the training license plate content template image), and the training painted license plate background image is input into the second encoder for encoding (the process of the second encoder encoding the training painted license plate background image is the process of extracting painted license plate background features from the training painted license plate background image). The encoding result output by the first encoder (i.e., license plate content features) and the encoding result output by the second encoder (i.e., painted license plate background features) are fused and input into the generator to obtain the painted license plate image generated by the generator based on the input.

[0106] Step S402: Input the painted license plate image generated by the generator network and the real painted license plate image into the discriminator network in the generative adversarial network, respectively, to obtain the probability that the discriminator network will identify the input image as a real painted license plate.

[0107] Step S402 can obtain the probability that the discriminator network will identify the sprayed license plate image generated by the generator network (i.e., the fake sprayed license plate image) as a real sprayed license plate image, and the probability that the discriminator network will identify the input real sprayed license plate image as a real sprayed license plate.

[0108] Step S403: Determine the predicted loss based on the obtained probabilities.

[0109] Specifically, the predicted loss can be determined using the following formula:

[0110] L = E y～pdata(Y) [logD Y (y)]+E x～pdata(X) [log(1-D Y (G(x))] (1)

[0111] Where y represents the actual painted license plate image, and D Y (y) represents the probability that the discriminator network identifies a real painted license plate image as a real painted license plate, G(x) represents the painted license plate image generated by the generator network, and D Y(G(x)) represents the probability that the discriminator network classifies the painted license plate image generated by the generator network as a real painted license plate, E y～pdata(Y) E represents a real license plate image randomly selected from the distribution of real license plate image data. x～pdata(X) This indicates that a license plate content template image is randomly selected from the license plate content template image data distribution. It should be noted that the first half of Equation (1) is used to optimize the discriminator network, and the second half is used to optimize the generator network. For the discriminator network, it is hoped that it can identify all the input painted license plate images as true, and for the generator network, it is hoped that it can generate painted license plate images that the discriminator network identifies as true.

[0112] Step S403: Update the parameters of the adversarial generative network based on the determined predicted loss.

[0113] The adversarial generative network is trained multiple times using multiple different training data in the manner described above until the training termination conditions are met (e.g., reaching the preset number of training iterations, or the model performance meeting the requirements). The generative network in the trained adversarial generative network is then used as the constructed license plate image generation model.

[0114] In order to construct a higher-performance model for generating painted license plate images, capable of generating painted license plate images with controllable license plate content and character structure consistent with the character structure of the input license plate content template, this invention provides a second implementation method (preferred implementation method):

[0115] like Figure 5 As shown, the Generative Adversarial Network (GAN) includes a first generator network 501, a second generator network 502, and a discriminator network 503. The training data for the GAN includes: training license plate content template images, training painted license plate background images, training template background images, and real painted license plate images. Preferably, all images in the training data are of the same size. It should be noted that the training template background image in a training dataset is the background image of the training license plate content template image in that training dataset. For example, if the background of the training license plate content template image in a training dataset is blue, then the training template background image in that training dataset is blue.

[0116] The training objectives of the Generative Adversarial Network (GAN) include: first, enabling the discriminator network in the GAN to classify the license plate images generated by the first generator network based on the training license plate content template image and the training license plate background image as genuine license plate images; second, ensuring that the structural information of the characters in the license plate images generated by the first generator network based on the training license plate content template image and the training license plate background image is consistent with the structural information of the characters in the training license plate content template image; and third, ensuring that the license plate content in the license plate images generated by the first generator network based on the training license plate content template and the training license plate background image is consistent with the license plate content in the training license plate content template. It should be noted that the structural information of the characters may include, but is not limited to, the structural information of each character itself, the relative positional information between characters, etc.

[0117] Please see Figure 6 The diagram illustrates the process of training an adversarial generative network, which may include:

[0118] Step S601: Input the training license plate content template image and the training painted license plate background image into the first generation network to obtain the painted license plate image generated by the first generation network based on the input.

[0119] Optional, such as Figure 5 As shown, the first generator network 501 may include a first encoder, a second encoder, and a first generator. For example, both the first encoder and the second encoder may employ multiple (e.g., two) stacked 4×4 convolutional kernels and pooling layers. After obtaining the training license plate content template image and the training painted license plate background image, the training license plate content template image is input into the first encoder for encoding (the process of the first encoder encoding the training license plate content template image is the process of extracting license plate content features from the training license plate content template image), and the training painted license plate background image is input into the second encoder for encoding (the process of the second encoder encoding the training painted license plate background image is the process of extracting painted license plate background features from the training painted license plate background image). The encoding result output by the first encoder (i.e., license plate content features) and the encoding result output by the second encoder (i.e., painted license plate background features) are fused and input into the first generator to obtain the painted license plate image generated by the first generator based on the input.

[0120] Step S602: Input the training license plate content template image and the training template background image into the second generation network to obtain the license plate template image generated by the second generation network based on the input.

[0121] Optional, such as Figure 5As shown, the second generator network 502 may include the first encoder (i.e., the second generator network shares the first encoder with the first generator network), a third encoder (exemplarily, the third encoder may employ multiple (e.g., two) stacked 4×4 convolutional kernels and pooling layers), and a second generator. The training license plate content template image is input to the first encoder for encoding, and the training template background image is input to the third encoder for encoding (the process of the third encoder encoding the training template background image is the process of extracting template background features from the training template background image). The encoding results output by the first encoder and the third encoder are fused and then input to the second generator, which generates the license plate template image based on the input.

[0122] It should be noted that this embodiment does not limit the execution order of steps S601 and S602. For example, step S601 can be executed first and then step S602 can be executed, or step S602 can be executed first and then step S601 can be executed, or steps S601 and S602 can be executed in parallel.

[0123] Step S603: Input the painted license plate image generated by the first generator network and the real painted license plate image into the discriminator network in the adversarial generator network, respectively, to obtain the probability that the discriminator network will identify the input image as a real painted license plate, and determine the first prediction loss based on the obtained probability.

[0124] Step S603 can obtain the probability that the discriminator network will identify the painted license plate image generated by the first generator network (i.e., the fake painted license plate image) as a real painted license plate image, and the probability that the discriminator network will identify the input real painted license plate image as a real painted license plate.

[0125] Specifically, the first prediction loss L1 can be determined according to the following formula:

[0126] L1 = E y～pdata(Y) [logD Y (y)]+E x～pdata(X) [log(1-D Y (G(x))] (2)

[0127] In equation (2), y and D Y (y), G(x), D Y (G(x)), E y～pdata(Y) E x～pdata(X) For further explanation, please refer to the explanation of the corresponding part in (1). This embodiment will not repeat the explanation here.

[0128] Step S604: Construct character structure supervision information based on the training license plate content template image and the training painted license plate background image, and determine the second prediction loss based on the character structure supervision information and the painted license plate image generated by the first generator network.

[0129] Among them, the character structure supervision information is used to constrain the character structure of the painted license plate image generated by the first generator network. That is, the character structure supervision information is fine-grained character supervision information, which is used to further constrain the generated painted license plate image at a fine-grained level.

[0130] Optionally, the process of constructing character structure supervision information based on the training license plate content template image and the training painted license plate background image may include: processing the background of the training license plate content template image to black and the characters to white to obtain a processed image; adding the processed image to the corresponding pixels in the training painted license plate background image, and using the result of the addition as the character structure supervision information.

[0131] Specifically, the second prediction loss L2 can be determined according to the following formula:

[0132] L2 = |mask-fake x2y |1 (3)

[0133] Where mask represents character structure supervision information, and fake x2y This represents the painted license plate image generated by the first generator network, where |*|1 indicates that absolute loss is performed pixel by pixel on the image.

[0134] Step S605: Determine the third prediction loss based on the license plate content template image trained by the training network and the license plate template image generated by the second generation network.

[0135] Specifically, the third prediction loss L3 can be determined according to the following formula:

[0136]

[0137] in, This represents the encoding result of the training license plate content template image, that is, the license plate content features extracted from the training license plate content template image, which are obtained through the first encoder. This represents the encoding result of the training template background image, that is, the template background features extracted from the training template background image, which is obtained through the third encoder. Indicates the second generator is based on and The generated license plate template image.

[0138] Step S606: Reconstruct the license plate content template image based on the painted license plate image generated by the second generator network and the first generator network and the training template background image, and determine the fourth prediction loss based on the reconstructed license plate content template image and the training license plate content template image.

[0139] Specifically, the fourth prediction loss L4 can be determined according to the following formula:

[0140]

[0141] Among them, and For the description, please refer to the relevant part of Equation (4). represents the encoded result of the training spray license plate background image, that is, the spray license plate background features extracted from the training spray license plate background image, which are obtained through the second encoder. represents that the first generator in the first generation network is based on and to generate a spray license plate image. represents the encoded result of the spray license plate image generated by the first generator, that is, the license plate content features extracted from the spray license plate image generated by the first generator, which are obtained by encoding the spray license plate image generated by the first encoder through the first encoder. represents the license plate content template image reconstructed by the second generator in the second generation network based on the encoded result of the spray license plate image generated by the first generator and the encoded result of the training template background image.

[0142] It should be noted that the above L3 and L4 are used to make the license plate content in the spray license plate image generated by the first generator consistent with the license plate content in the training license plate content template image. That is, L3 and L4 are used to enable the first generator to controllably generate a spray license plate image with a specified license plate content.

[0143] In addition, the above L3 can solve the problem of unpaired data. For example, if the content of the spray license plate image generated by the first generator is "Beijing A23456", if there is a real spray license plate image with the content of "Beijing A23456" in the training dataset, it can be used to supervise the quality of the generated spray license plate image. However, in the actual training dataset, there cannot be real spray license plate images with the same content as all the generated spray license plate images. In view of this, the present invention uses the reconstructed license plate content template image to supervise the quality of the generated spray license plate image.

[0144] It should be noted that this embodiment does not limit the execution order of steps S603 to S606. Steps S603 to S606 can be executed in any order or in parallel.

[0145] Step S607: Update the parameters of the adversarial generation network according to the first prediction loss, the second prediction loss, the third prediction loss, and the fourth prediction loss.

[0146] Specifically, the first prediction loss, the second prediction loss, the third prediction loss, and the fourth prediction loss can be fused to obtain the total prediction loss, and the parameters of the adversarial generation network are updated according to the total prediction loss.

[0147] There are several ways to fuse the first, second, third, and fourth prediction losses. In one possible implementation, the first, second, third, and fourth prediction losses can be directly summed. In another possible implementation, the first, second, third, and fourth prediction losses can be weighted and summed. The total prediction loss Loss obtained by weighted summing of the first, second, third, and fourth prediction losses can be expressed as:

[0148] Loss=α1·L1+α2·L2+α3·L3+α4·L4 (6)

[0149] Wherein, α1, α2, α3, and α4 represent the weights corresponding to the first prediction loss L1, the second prediction loss L2, the third prediction loss L3, and the fourth prediction loss L4, respectively. α1, α2, α3, and α4 are hyperparameters, which can be set according to specific circumstances.

[0150] Multiple different training data sets (each training data set includes training license plate content template data, training painted license plate background image, training template background image, and real painted license plate image) are used to train the adversarial generative network multiple times in the manner described above until the training termination conditions are met (e.g., reaching the preset number of training times, model performance meeting requirements, etc.). The first generative network in the trained adversarial generative network is then used as the constructed painted license plate image generation model.

[0151] The above method can be used to construct a model for generating painted license plate images that can produce images of relatively good quality.

[0152] Based on the method for generating painted license plate images provided in the above embodiments, this invention also provides a method for constructing a painted license plate recognition model, which may include:

[0153] Step a1: Obtain the dataset of painted license plates.

[0154] The painted license plate dataset includes multiple painted license plate images generated using the painted license plate image generation method provided in the above embodiments. Preferably, the painted license plate dataset may also include multiple real painted license plate images. It should be noted that when generating painted license plate images using the painted license plate image generation method provided in the above embodiments, diverse painted license plate images can be generated according to the specific application scenario's requirements for training data.

[0155] Step a2: Train a license plate recognition model based on the painted license plate images in the painted license plate dataset. The trained license plate recognition model is used as the constructed painted license plate recognition model.

[0156] Since the method for generating painted license plate images provided in the above embodiments can generate a large number of high-quality and diverse painted license plate images, the painted license plate dataset contains a sufficient number of high-quality and diverse painted license plate images. Therefore, a high-performance painted license plate recognition model can be trained based on the painted license plate images in the painted license plate dataset.

[0157] Based on the method for constructing a painted license plate recognition model provided in the above embodiments, this invention also provides a method for recognizing painted license plates, which may include:

[0158] Step b1: Obtain the image of the painted license plate to be identified.

[0159] Step b2: Input the image of the painted license plate to be identified into the painted license plate recognition model constructed using the painted license plate recognition model construction method provided in the above embodiment, and obtain the painted license plate recognition result output by the painted license plate recognition model.

[0160] Since the method for constructing a painted license plate recognition model provided in the above embodiments can construct a painted license plate recognition model with better performance, using the painted license plate recognition model with better performance to perform license plate recognition on the painted license plate image to be recognized can obtain a more accurate recognition result.

[0161] This invention also provides a spray-painted license plate image generation device. The spray-painted license plate image generation device provided in this invention will be described below. The spray-painted license plate image generation device described below can be referred to in correspondence with the spray-painted license plate image generation method described above.

[0162] Please see Figure 7 The diagram illustrates the structure of a license plate image generation device provided in an embodiment of the present invention, which may include: an image acquisition module 701, a content feature extraction module 702, a background feature extraction module 703, and a license plate image generation module 704. Wherein:

[0163] The image acquisition module 701 is used to acquire the license plate content template image and the sprayed license plate background image.

[0164] The content feature extraction module 702 is used to extract license plate content features from the license plate content template image.

[0165] Background feature extraction module 703 is used to extract background features of the painted license plate from the painted license plate background image.

[0166] The license plate image generation module 704 is used to generate a license plate image based on the license plate content features and the license plate background features.

[0167] Optionally, the image acquisition module 701 includes: a license plate content template image acquisition submodule and a sprayed license plate background image acquisition submodule.

[0168] The license plate content template image acquisition submodule is used to acquire license plate content template images from a content dataset containing multiple license plate content template images with different character attributes.

[0169] The license plate background image acquisition submodule is used to acquire the license plate background image from a background dataset that includes multiple license plate background images, wherein the multiple license plate background images are extracted from multiple real license plate images.

[0170] Optionally, the content feature extraction module 702 is specifically used to extract license plate content features from the license plate content template image based on a pre-built sprayed license plate image generation model.

[0171] Background feature extraction module 703 is specifically used to extract background features of the painted license plate from the painted license plate background image based on a pre-built painted license plate image generation model.

[0172] The license plate image generation module 704 is specifically used to fuse the license plate content features with the license plate background features to obtain fused features, and generate a license plate image based on the license plate image generation model and the fused features.

[0173] Optionally, the painted license plate image generation model adopts the first generator network in the trained adversarial generative network; the training data of the adversarial generative network includes: training license plate content template image and training painted license plate background image.

[0174] Optionally, the training objective of the adversarial generative network includes: enabling the discriminator network in the adversarial generative network to determine the painted license plate image generated by the first generator network based on the training license plate content template image and the training painted license plate background image as a real painted license plate image.

[0175] Optionally, the training objectives for adversarial generative networks may also include:

[0176] The structural information of the characters in the painted license plate image generated by the first generation network based on the training license plate content template image and the training painted license plate background image is made to be consistent with the structural information of the characters in the training license plate content template image.

[0177] Optionally, the training objectives for adversarial generative networks may also include:

[0178] The license plate content in the sprayed license plate image generated by the first generation network based on the training license plate content template image and the training sprayed license plate background image is made to be consistent with the license plate content in the training license plate content template image.

[0179] Optionally, the training data further includes: real painted license plate images; the painted license plate image generation device provided in this embodiment of the invention further includes a model training module, which is used to train an adversarial generative network. Specifically, when training the adversarial generative network, the model training module is used for:

[0180] The training license plate content template image and the training painted license plate background image are input into the first generation network to obtain the painted license plate image generated by the first generation network based on the input.

[0181] The painted license plate image generated by the first generator network and the real painted license plate image are respectively input into the discriminant network in the adversarial generator network to obtain the probability that the discriminant network will identify the input image as a real painted license plate;

[0182] Determine the first prediction loss based on the obtained probabilities;

[0183] The parameters of the adversarial generative network are updated based on the first predicted loss.

[0184] Optionally, the model training module is also used when training the generative adversarial network to:

[0185] Based on the training license plate content template image and the training painted license plate background image, character structure supervision information is constructed, wherein the character structure supervision information is used to constrain the character structure of the painted license plate generated by the first generation network;

[0186] Based on the character structure supervision information and the painted license plate image generated by the first generation network, the second prediction loss is determined;

[0187] The step of updating the parameters of the adversarial generative network based on the first predicted loss includes:

[0188] The parameters of the adversarial generative network are updated based on the first prediction loss and the second prediction loss.

[0189] Optionally, when constructing character structure supervision information based on the training license plate content template image and the training painted license plate background image, the model training module is specifically used for:

[0190] The background of the training license plate content template image is processed to black and the characters are processed to white to obtain the processed image;

[0191] The processed image is added to the corresponding pixels in the training spray-painted license plate background image, and the result is used as character structure supervision information.

[0192] Optionally, the adversarial generative network further includes a second generative network, and the training data further includes a training template background image; the model training module, when training the adversarial generative network, is also used for:

[0193] The training license plate content template image and the training template background image are input into the second generation network to obtain the license plate template image generated by the second generation network based on the input;

[0194] The third prediction loss is determined based on the training license plate content template and the license plate template image;

[0195] Based on the painted license plate image generated by the second generator network and the first generator network and the training template background image, the license plate content template image is reconstructed, and the fourth prediction loss is determined according to the reconstructed license plate content template image and the training license plate content template image.

[0196] The step of updating the parameters of the adversarial generative network based on the first predicted loss and the second predicted loss includes:

[0197] The parameters of the adversarial generative network are updated based on the first prediction loss, the second prediction loss, the third prediction loss, and the fourth prediction loss.

[0198] The license plate image generation device provided in this invention first acquires a license plate content template image and a license plate background image. Then, it extracts license plate content features from the license plate content template image and extracts background feature images from the license plate background image. Finally, it generates a license plate image based on the license plate content features and the license plate background features. The license plate image generation method provided by this invention can automatically generate license plate images based on the license plate content template image and the license plate background image. Compared to collecting license plate images from actual scenes, the license plate image generation device provided in this invention can significantly reduce the human burden and resource consumption of acquiring license plate images. Furthermore, since this invention constructs a content dataset including diverse license plate content template images and a background dataset including diverse license plate background images, diverse license plate images can be generated based on the content dataset, the background dataset, and the method for generating license plate images provided by this invention.

[0199] This invention also provides a device for generating license plate images by spraying paint. Please refer to [link / reference]. Figure 8The diagram shows the structure of the sprayed license plate image generation device, which may include: at least one processor 801, at least one communication interface 802, at least one memory 803 and at least one communication bus 804.

[0200] In this embodiment of the invention, the number of processor 801, communication interface 802, memory 803, and communication bus 804 is at least one, and processor 801, communication interface 802, and memory 803 communicate with each other through communication bus 804.

[0201] The processor 801 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0202] The memory 803 may include high-speed RAM, or it may also include non-volatile memory, such as at least one disk storage device;

[0203] The memory stores a program, which the processor can call. The program is used for:

[0204] Obtain the license plate content template image and the license plate background image for spraying;

[0205] Extract license plate content features from the license plate content template image, and extract sprayed license plate background features from the sprayed license plate background image;

[0206] Based on the license plate content features and the background features of the painted license plate, a painted license plate image is generated.

[0207] Optionally, the refined and extended functions of the program can be found in the description above.

[0208] This invention also provides a readable storage medium that stores a program suitable for execution by a processor, the program being used for:

[0209] Obtain the license plate content template image and the license plate background image for spraying;

[0210] Extract license plate content features from the license plate content template image, and extract sprayed license plate background features from the sprayed license plate background image;

[0211] Based on the license plate content features and the background features of the painted license plate, a painted license plate image is generated.

[0212] Optionally, the refined and extended functions of the program can be found in the description above.

[0213] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0214] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0215] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating images of painted license plates, characterized in that, include: Obtain the license plate content template image and the license plate background image for spraying; Extract license plate content features from the license plate content template image and extract sprayed license plate background features from the sprayed license plate background image; Based on the license plate content features and the background features of the painted license plate, a painted license plate image is generated; The process of obtaining the license plate content template image and the sprayed license plate background image includes: From a content dataset containing multiple license plate content template images with different character attributes, obtain license plate content template images; from a background dataset containing multiple sprayed license plate background images, obtain sprayed license plate background images, wherein the multiple sprayed license plate background images are extracted from multiple real sprayed license plate images; The step of extracting license plate content features from the license plate content template image and extracting sprayed license plate background features from the sprayed license plate background image, and generating a sprayed license plate image based on the license plate content features and the sprayed license plate background features, includes: Based on a pre-built painted license plate image generation model, license plate content features are extracted from the license plate content template image, and painted license plate background features are extracted from the painted license plate background image; the license plate content features and the painted license plate background features are fused to obtain fused features; based on the painted license plate image generation model and using the fused features as a basis, a painted license plate image is generated. The sprayed license plate image generation model adopts the first generator network in the trained adversarial generative network. The training data of the adversarial generative network includes training license plate content template images, training sprayed license plate background images, and real sprayed license plate images. The training process for generative adversarial networks includes: The training license plate content template image and the training painted license plate background image are input into the first generation network to obtain the painted license plate image generated by the first generation network based on the input; the painted license plate image generated by the first generation network and the real painted license plate image are respectively input into the discriminant network in the generative adversarial network to obtain the probability that the discriminant network will identify the input image as a real painted license plate; the first prediction loss is determined based on the obtained probability. Based on the training license plate content template image and the training painted license plate background image, character structure supervision information is constructed, wherein the character structure supervision information is used to constrain the character structure of the painted license plate generated by the first generation network; based on the character structure supervision information and the painted license plate image generated by the first generation network, a second prediction loss is determined; The parameters of the adversarial generative network are updated based on the first prediction loss and the second prediction loss.

2. The method for generating a license plate image according to claim 1, characterized in that, The training objective of the Generative Adversarial Network (GAN) includes: enabling the discriminator network in the GAN to determine the painted license plate image generated by the first generator network based on the training license plate content template image and the training painted license plate background image as a real painted license plate image.

3. The method for generating a license plate image according to claim 2, characterized in that, The training objectives for generative adversarial networks also include: The structural information of the characters in the painted license plate image generated by the first generation network based on the training license plate content template image and the training painted license plate background image is made to be consistent with the structural information of the characters in the training license plate content template image.

4. The method for generating a license plate image according to claim 2 or 3, characterized in that, The training objectives for generative adversarial networks also include: The license plate content in the sprayed license plate image generated by the first generation network based on the training license plate content template image and the training sprayed license plate background image is made to be consistent with the license plate content in the training license plate content template image.

5. The method for generating a license plate image according to claim 1, characterized in that, The step of constructing character structure supervision information based on the training license plate content template image and the training painted license plate background image includes: The background of the training license plate content template image is processed to black and the characters are processed to white to obtain the processed image; The processed image is added to the corresponding pixels in the training spray-painted license plate background image, and the result is used as character structure supervision information.

6. The method for generating a license plate image according to claim 1, characterized in that, The adversarial generative network also includes a second generative network, and the training data also includes a training template background image; The training process for adversarial generative networks also includes: The training license plate content template image and the training template background image are input into the second generation network to obtain the license plate template image generated by the second generation network based on the input; The third prediction loss is determined based on the training license plate content template and the license plate template image; Based on the painted license plate image generated by the second generator network and the first generator network and the training template background image, the license plate content template image is reconstructed, and the fourth prediction loss is determined according to the reconstructed license plate content template image and the training license plate content template image. The step of updating the parameters of the adversarial generative network based on the first predicted loss and the second predicted loss includes: The parameters of the adversarial generative network are updated based on the first prediction loss, the second prediction loss, the third prediction loss, and the fourth prediction loss.

7. A method for constructing a license plate recognition model, characterized in that, include: Obtain a painted license plate dataset, wherein the painted license plate dataset includes multiple painted license plate images generated using the painted license plate image generation method as described in any one of claims 1 to 6; A license plate recognition model is trained based on the painted license plate images in the painted license plate dataset, and the trained license plate recognition model is used as the constructed painted license plate recognition model.

8. A method for recognizing painted license plates, characterized in that, include: Acquire the image of the painted license plate to be identified; The painted license plate image is input into the painted license plate recognition model constructed using the painted license plate recognition model construction method as described in claim 7, and the painted license plate recognition result output by the painted license plate recognition model is obtained.

9. A device for generating license plate images by spraying paint, characterized in that, include: Image acquisition module, content feature extraction module, background feature extraction module, and painted license plate image generation module; The image acquisition module is used to acquire the license plate content template image and the sprayed license plate background image; The content feature extraction module is used to extract license plate content features from the license plate content template image; The background feature extraction module is used to extract the background features of the painted license plate from the painted license plate background image; The painted license plate image generation module is used to generate a painted license plate image based on the license plate content features and the painted license plate background features; The image acquisition module, when acquiring the license plate content template image and the sprayed license plate background image, is specifically used for: From a content dataset containing multiple license plate content template images with different character attributes, obtain license plate content template images; from a background dataset containing multiple sprayed license plate background images, obtain sprayed license plate background images, wherein the multiple sprayed license plate background images are extracted from multiple real sprayed license plate images; When extracting license plate content features from the license plate content template image, the content feature extraction module is specifically used to extract license plate content features from the license plate content template image based on a pre-built sprayed license plate image generation model. When the background feature extraction module extracts the background features of the painted license plate from the painted license plate background image, it is specifically used to extract the background features of the painted license plate from the painted license plate background image based on the painted license plate image generation model. When generating a painted license plate image based on the license plate content features and the painted license plate background features, the painted license plate image generation module is specifically used to fuse the license plate content features and the painted license plate background features to obtain fused features, and generate a painted license plate image based on the painted license plate image generation model and the fused features. The sprayed license plate image generation model adopts the first generator network in the trained adversarial generative network. The training data of the adversarial generative network includes training license plate content template images, training sprayed license plate background images, and real sprayed license plate images. The spray-painted license plate image generation device further includes a model training module, which is used to train an adversarial generative network. Specifically, when training the adversarial generative network, the model training module is used for: The training license plate content template image and the training painted license plate background image are input into the first generation network to obtain the painted license plate image generated by the first generation network based on the input; the painted license plate image generated by the first generation network and the real painted license plate image are respectively input into the discriminant network in the generative adversarial network to obtain the probability that the discriminant network will identify the input image as a real painted license plate; the first prediction loss is determined based on the obtained probability. Based on the training license plate content template image and the training painted license plate background image, character structure supervision information is constructed, wherein the character structure supervision information is used to constrain the character structure of the painted license plate generated by the first generation network; based on the character structure supervision information and the painted license plate image generated by the first generation network, a second prediction loss is determined; The parameters of the adversarial generative network are updated based on the first prediction loss and the second prediction loss.

10. A device for generating license plate images by spraying paint, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the method for generating license plate images as described in any one of claims 1 to 6.

11. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the method for generating a painted license plate image as described in any one of claims 1 to 6.

Images