License plate image generation method, device and computer storage medium

By generating diverse license plate images through a virtual three-dimensional model, the method addresses the limitations of traditional training sample diversity, enhancing neural network recognition accuracy for license plate recognition.

CN112396076BActive Publication Date: 2024-12-03HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN201910755413.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-08-15
Publication Date
2024-12-03
Estimated Expiration
2039-08-15

AI Technical Summary

Technical Problem

Existing license plate recognition systems face limitations due to the lack of diversity in training samples, as they rely on manually annotated images from cameras, which are limited in variety, hindering the effectiveness of neural network training.

Method used

A method and device for generating license plate images using a three-dimensional model of a virtual license plate, simulating various attributes and conditions to create diverse training samples, including shape, color, characters, and surface properties, without relying on real-world camera captures.

Benefits of technology

This approach enhances the efficiency and diversity of training samples, improving the recognition accuracy of neural networks by simulating a wide range of license plate scenarios, thus optimizing the training process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112396076B_ABST
    Figure CN112396076B_ABST
Patent Text Reader

Abstract

The present application discloses a license plate image generation method, device and computer storage medium, and belongs to the field of neural network training technology. The method includes: constructing a three-dimensional model of a virtual license plate, and determining the license plate surface attribute information of the virtual license plate to generate a license plate image for the virtual license plate. Therefore, when determining the training sample for the neural network model used for license plate recognition, the license plate image can be directly generated according to actual needs through the embodiment of the present application, and there is no need to collect the real license plate through a camera to obtain the license plate image, which improves the efficiency of obtaining the license plate image. In addition, since the virtual license plate is a license plate simulated according to actual needs, the license plate images corresponding to different types of virtual license plates can be generated through the embodiment of the present application, which improves the diversity of the license plate images in the training samples, thereby improving the recognition accuracy of the neural network model trained according to the training samples.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of neural network training technology, and in particular to a license plate generation method, device and computer storage medium. Background Art

[0002] With the development of big data technology, license plate recognition can be performed through neural networks. Before realizing license plate recognition through neural networks, it is necessary to first train the initialized neural network using training samples. The training samples include multiple license plate images, each of which is annotated with a label indicating the license plate identification. The license plate identification includes multiple characters.

[0003] In the related art, the image captured by the camera for the license plate can be obtained first. For each captured image, a label indicating the license plate identification is manually annotated, and each annotated image is called a license plate image. Each annotated license plate image is used as a training sample to train the initialized neural network. That is, the related art generates license plate images for training by means of camera acquisition and manual annotation.

[0004] Since the types of license plates covered in the images captured by the camera for license plates are limited, the types of license plate images generated for training are relatively single, which is not conducive to training the neural network model. Summary of the invention

[0005] The embodiments of the present application provide a license plate image generation method, device and computer storage medium, which can improve the recognition accuracy of a neural network model trained based on the generated license plate image. The technical solution is as follows:

[0006] In one aspect, a method for generating a license plate image is provided, the method comprising:

[0007] Constructing a three-dimensional model of a virtual license plate, wherein the virtual license plate is a license plate simulated according to demand;

[0008] Determining license plate surface attribute information of the virtual license plate;

[0009] A license plate image for the virtual license plate is generated according to the license plate surface attribute information and the three-dimensional model.

[0010] Optionally, constructing a three-dimensional model of a virtual license plate includes:

[0011] Determining the shape of the border of the virtual license plate;

[0012] Determining the background color of the virtual license plate;

[0013] Determining a plurality of characters on the virtual license plate for indicating a license plate identification and an arrangement order of the plurality of characters;

[0014] A three-dimensional model of the virtual license plate is generated according to the shape of the frame, the background color, the multiple characters, and the arrangement order of the multiple characters.

[0015] Optionally, determining the shape of the border of the virtual license plate includes:

[0016] The shape of the border of the virtual license plate is determined according to the deformation of the virtual license plate.

[0017] Optionally, generating the three-dimensional model of the virtual license plate according to the shape of the frame, the background color, the multiple characters, and the arrangement order of the multiple characters includes:

[0018] Generating a theoretical three-dimensional view of the virtual license plate according to the shape of the frame, the background color, the multiple characters, and the arrangement order of the multiple characters;

[0019] According to the crease condition of the virtual license plate, the theoretical three-dimensional view of the virtual license plate is adjusted to obtain a three-dimensional model of the virtual license plate.

[0020] Optionally, the license plate surface attribute information includes at least one or more of the following attribute information:

[0021] The metal property of the virtual license plate;

[0022] The diffuse reflection property of the virtual license plate;

[0023] The flatness of the virtual license plate;

[0024] The specular reflection property of the virtual license plate;

[0025] Dust condition on the virtual license plate;

[0026] The stain condition, coverage condition and occlusion condition on the virtual license plate.

[0027] Optionally, generating a license plate image for the virtual license plate according to the license plate attribute information and the three-dimensional model includes:

[0028] Determining shooting parameters of a virtual shooting scene;

[0029] generating a virtual photo for the virtual license plate according to the shooting parameters, the license plate attribute information and the three-dimensional model;

[0030] The virtual photo is post-processed to obtain the license plate image.

[0031] Optionally, the shooting parameters of the virtual shooting scene include at least one or more of the following parameters:

[0032] The color of the light in the virtual shooting scene, the brightness of the light, the exposure parameters of the virtual camera in the virtual shooting scene, and the positions of the virtual camera and the virtual license plate in the virtual shooting scene.

[0033] Optionally, after generating a license plate image for the virtual license plate according to the license plate attribute information and the three-dimensional model, the method further includes:

[0034] The license plate recognition result on the virtual license plate is marked in the license plate image.

[0035] On the other hand, a license plate image generating device is provided, the device comprising:

[0036] A construction module, used to construct a three-dimensional model of a virtual license plate, where the virtual license plate is a license plate simulated according to requirements;

[0037] A determination module, used to determine the license plate surface attribute information of the virtual license plate;

[0038] A generation module is used to generate a license plate image for the virtual license plate based on the license plate surface attribute information and the three-dimensional model.

[0039] Optionally, the building module is specifically used to:

[0040] Determining the shape of the border of the virtual license plate;

[0041] Determining the background color of the virtual license plate;

[0042] Determining a plurality of characters on the virtual license plate for indicating a license plate identification and an arrangement order of the plurality of characters;

[0043] A three-dimensional model of the virtual license plate is generated according to the shape of the frame, the background color, the multiple characters, and the arrangement order of the multiple characters.

[0044] Optionally, the construction module is specifically used to determine the shape of the border of the virtual license plate according to the deformation of the virtual license plate.

[0045] Optionally, the building module is specifically used to:

[0046] Generating a theoretical three-dimensional view of the virtual license plate according to the shape of the frame, the background color, the multiple characters, and the arrangement order of the multiple characters;

[0047] According to the crease condition of the virtual license plate, the theoretical three-dimensional view of the virtual license plate is adjusted to obtain a three-dimensional model of the virtual license plate.

[0048] Optionally, the license plate surface attribute information includes at least one or more of the following attribute information:

[0049] The metal property of the virtual license plate;

[0050] The diffuse reflection property of the virtual license plate;

[0051] The flatness of the virtual license plate;

[0052] The virtual license plate mirror reflection property;

[0053] Dust condition on the virtual license plate;

[0054] The stain condition, coverage condition and occlusion condition on the virtual license plate.

[0055] Optionally, the generating module is specifically used to:

[0056] Determining shooting parameters of a virtual shooting scene;

[0057] generating a virtual photo for the virtual license plate according to the shooting parameters, the license plate attribute information and the three-dimensional model;

[0058] The virtual photo is post-processed to obtain the license plate image.

[0059] Optionally, the shooting parameters of the virtual shooting scene include at least one or more of the following parameters:

[0060] The color of the light in the virtual shooting scene, the brightness of the light, the exposure parameters of the virtual camera in the virtual shooting scene, and the positions of the virtual camera and the virtual license plate in the virtual shooting scene.

[0061] Optionally, the device further comprises:

[0062] The marking module is used to mark the license plate recognition result on the virtual license plate in the license plate image.

[0063] On the other hand, a license plate image generating device is provided, the device comprising:

[0064] processor;

[0065] a memory for storing processor-executable instructions;

[0066] Wherein, the processor is configured to execute the steps of the method for generating a license plate image provided in any of the above aspects.

[0067] The beneficial effects brought by the technical solution provided in the embodiments of the present application include:

[0068] In an embodiment of the present application, a virtual license plate can be simulated according to actual needs, and then a three-dimensional model of the virtual license plate can be constructed, and the license plate surface attribute information of the virtual license plate can be determined to generate a license plate image for the virtual license plate. Therefore, when determining the training sample for the neural network model used for license plate recognition, the license plate image can be directly generated through the embodiment of the present application according to actual needs, and there is no need to collect the real license plate through a camera to obtain the license plate image, which improves the efficiency of obtaining the license plate image. In addition, since the virtual license plate is a license plate simulated according to actual needs, the license plate images corresponding to different types of virtual license plates can be generated through the embodiment of the present application, which improves the diversity of the license plate images in the training samples, thereby improving the recognition accuracy of the neural network model trained based on the training samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0070] Figure 1 This is a flow chart of a license plate image generation method provided in an embodiment of the present application.

[0071] Figure 2 This is a block diagram of a license plate image generating device provided in an embodiment of the present application.

[0072] Figure 3 It is a structural diagram of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0074] Before explaining the embodiments of the present application, the application scenarios involved in the embodiments of the present application are explained. License plate recognition refers to: identifying the characters used to indicate the license plate identification in the license plate image to obtain the license plate identification corresponding to the license plate image. The license plate identification includes multiple characters, and the multiple characters can be one or more of numbers, English letters, Chinese characters and other characters. The license plate identification can also be called a license plate number. At present, fast license plate recognition can be achieved through neural network technology. In the process of realizing license plate recognition through neural networks, the diversity of training samples used to train the neural network directly affects the recognition accuracy of the trained neural network. Therefore, it is necessary to obtain a large number of different types of training samples to improve the recognition accuracy of the trained neural network model. For the neural network model for license plate recognition, the training sample refers to multiple license plate images, each of which is marked with a label for indicating the license plate identification. The license plate image generation method adopted in the embodiment of the present application is applied to the scenario of obtaining training samples for the neural network model for license plate recognition.

[0075] Figure 1 : is a flow chart of a license plate image generation method provided in an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0076] Step 101: construct a three-dimensional model of a virtual license plate, where the virtual license plate is simulated according to demand.

[0077] The embodiment of the present application can simulate a virtual license plate according to actual needs, and then generate a license plate image for the virtual license plate through steps 101 to 103. Therefore, when determining the training sample for the neural network model for license plate recognition, the license plate image can be directly generated through steps 101 to 103 according to actual needs, without the need to collect the real license plate through a camera to obtain the license plate image, thereby improving the efficiency of obtaining the license plate image. In addition, since the virtual license plate is a license plate simulated according to actual needs, license plate images corresponding to different types of virtual license plates can be generated through steps 101 to 103, thereby improving the diversity of the license plate images in the training samples, thereby improving the recognition accuracy of the neural network model subsequently trained according to the training samples.

[0078] The three-dimensional model is used to indicate the view of the virtual license plate in the three-dimensional space. In a possible implementation, step 101 may specifically include: determining the shape of the frame of the virtual license plate; determining the background color of the virtual license plate; determining multiple characters on the virtual license plate used to indicate the license plate identification and the arrangement order of the multiple characters; generating the three-dimensional model of the virtual license plate according to the shape of the frame, the background color, the multiple characters and the arrangement order of the multiple characters.

[0079] The shape of the frame of the virtual license plate may include the shape of an outer frame and the shape of an inner frame. The shape of the outer frame of the virtual license plate may be determined by: a plurality of outer frame options are displayed in the current display interface, and each outer frame option indicates an outer frame shape. When a selection operation for any of the plurality of outer frame options is detected, the shape of the outer frame of the virtual license plate is determined based on the outer frame option corresponding to the selection operation.

[0080] The selection operation can be triggered by the management personnel according to actual needs. That is, the shape of the outer frame of the virtual license plate is determined according to actual needs. For example, if a license plate image of a virtual license plate for a military license plate needs to be generated, then the outer frame option corresponding to the military license plate can be selected from multiple outer frame options to determine that the shape of the outer frame of the virtual license plate has the shape of the outer frame of the military license plate.

[0081] In addition, the above-mentioned determination of the shape of the inner frame of the license plate and the background color of the virtual license plate can refer to the above-mentioned implementation method of determining the shape of the outer frame of the virtual license plate, and will not be elaborated here one by one.

[0082] In addition, when determining the shape of the border of the virtual license plate, the deformation of the virtual license plate may also be considered. The deformation includes deformation caused by damage, deformation caused by folding, deformation caused by wrinkles and twisting, and so on. For example, the shape of the outer frame of a normal license plate is a rectangle. When the license plate is damaged, a corner of the outer frame of the license plate may be worn away. At this time, the shape of the outer frame of the license plate is not a rectangle, but may be a trapezoid. Therefore, when determining the shape of the outer frame through the outer frame option displayed in the display interface, the shape of the outer frame of the virtual license plate may be adjusted according to the deformation of the virtual license plate after determining the shape of the outer frame of the virtual license plate based on the outer frame option corresponding to the selection operation, and the adjusted shape is used as the shape of the outer frame of the virtual license plate.

[0083] The above-mentioned multiple characters used to indicate the license plate identification may include any characters such as Chinese province abbreviations, English letters, numbers, etc. The arrangement order of multiple characters refers to the order in which multiple characters can indicate a license plate identification after being arranged. For example, when applying the embodiment of the present application, the arrangement order of the multiple characters can be determined according to the type of virtual license plate to be generated. The types of virtual license plates include ordinary license plates, military license plates, and other special license plates.

[0084] After determining the shape of the frame, the background color, the multiple characters, and the order of the multiple characters by any of the above methods, a three-dimensional model of the virtual license plate can be generated according to the shape of the frame, the background color, the multiple characters, and the order of the multiple characters. The three-dimensional model of the virtual license plate is used to indicate a three-dimensional view of the virtual license plate.

[0085] In addition, if a license plate has creases, the three-dimensional view of the license plate relative to a normal license plate will change. Therefore, in an embodiment of the present application, the implementation method of generating a three-dimensional model of a virtual license plate according to the shape of the frame, the background color, and the arrangement order of multiple characters can be: generating a theoretical three-dimensional view of the virtual license plate according to the shape of the frame, the background color, and the arrangement order of multiple characters; adjusting the theoretical three-dimensional view of the virtual license plate according to the creases of the virtual license plate to obtain a three-dimensional model of the virtual license plate. That is, the final three-dimensional model of the virtual license plate is a three-dimensional model that can indicate the existence of folds, wrinkles or creases on the license plate.

[0086] For example, according to the shape of the frame, the background color, and the arrangement order of multiple characters, the theoretical three-dimensional view of the generated virtual license plate may be a flat regular hexahedron. However, if a corner of the virtual license plate is creased, the corresponding corner of the regular hexahedron will also bend. Therefore, the theoretical three-dimensional view of the virtual license plate can be adjusted according to the crease of the virtual license plate, so that the obtained three-dimensional model of the virtual license plate conforms to the actual three-dimensional view of the license plate with the crease.

[0087] Step 102: Determine the license plate surface attribute information of the virtual license plate.

[0088] Based on step 101, the three-dimensional model of the virtual license plate can be determined. However, for license plates with the same three-dimensional model, if the surfaces of these license plates have different attributes, the license plate images captured by the camera for these license plates are very likely to be different. Therefore, in the embodiment of the present application, in order to make the license plate image of the virtual license plate finally generated close to the license plate image captured by the camera, the license plate surface attribute information of the virtual license plate can also be considered when generating the license plate image of the virtual license plate.

[0089] In a possible implementation, the license plate surface attribute information includes at least one or more of the following attribute information:

[0090] The metal property of the virtual license plate is used to indicate the metal property of the virtual license plate. The metal property includes the intensity of light reflected by the virtual license plate when light shines on the surface of the virtual license plate, and the color of the reflection formed by the light reflected by the virtual license plate. The metal property can also be called metallicity. For example, the metallicity can be set between 0.0-1.0. The closer the value corresponding to the metallicity is to 1, the stronger the metallic sense of the surface of the virtual license plate is. The closer the value corresponding to the metallicity is to 0, the weaker the metallic sense of the virtual license plate is.

[0091] The diffuse reflection property of the virtual license plate is used to indicate the degree to which light forms diffuse reflection on the virtual license plate when the light shines on the surface of the virtual license plate. The diffuse reflection property can also be called roughness.

[0092] The flatness of the virtual license plate.

[0093] The specular reflection property of the virtual license plate is used to indicate the degree to which the light forms a specular reflection on the virtual license plate when the light is projected onto the surface of the virtual license plate. The diffuse reflection property can also be called reflectivity.

[0094] Dust on the virtual license plate.

[0095] The condition of stains, coverage and occlusion on the virtual license plate.

[0096] The above-mentioned license plate surface attribute information is only used for illustration. When using the embodiments of the present application, the license plate surface attribute information may include any license plate surface factors that can affect the license plate image captured by the camera, and will not be given examples one by one here.

[0097] Step 103: Generate a license plate image for the virtual license plate based on the license plate surface attribute information and the three-dimensional model.

[0098] After the license plate surface attribute information and the three-dimensional model of the virtual license plate are determined according to step 101 and step 102 , a license plate image for the virtual license plate can be generated through step 103 .

[0099] In a possible implementation, step 103 may specifically include: determining shooting parameters of a virtual shooting scene; generating a virtual photo for the virtual license plate according to the shooting parameters, license plate attribute information and a three-dimensional model; and post-processing the virtual photo to obtain a license plate image.

[0100] For the same license plate, the license plate image captured by the camera is different in different shooting scenes. Therefore, in the embodiment of the present application, in order to make the license plate image finally generated close to the license plate image captured by the camera, a virtual shooting scene can also be determined to obtain the license plate image through the above implementation method.

[0101] When the color and brightness of the light in the shooting scene change, the license plate image collected by the camera for the same license plate is very likely to be different. In addition, when the exposure parameters of the camera change, the license plate image collected by the camera for the same license plate is very likely to be different. In addition, if the license plate is within the camera shooting area, but the distance between the license plate and the camera changes, the license plate image collected by the camera for the same license plate is very likely to be different. In addition, if the license plate is at the edge of the camera shooting area, the license plate image collected by the camera for the license plate may be lens distorted. Therefore, in an embodiment of the present application, the shooting parameters of the above-mentioned virtual shooting scene include at least one or more of the following parameters: the color of the light in the virtual shooting scene, the brightness of the light, the exposure parameters of the virtual camera in the virtual shooting scene, and the position of the virtual camera and the virtual license plate in the virtual shooting scene. The exposure parameters of the virtual camera may include the exposure time and exposure intensity of the virtual camera.

[0102] The above-mentioned shooting parameters are only used for illustration. When using the embodiments of the present application, the shooting parameters may include any shooting factors that may affect the license plate image captured by the camera, and they will not be given examples one by one here.

[0103] In addition, the above process of generating a virtual photo for a virtual license plate based on shooting parameters, license plate attribute information and a three-dimensional model can be achieved through ray tracing technology, which will not be elaborated in detail here.

[0104] Since there are many noise points in the virtual photo directly generated for the virtual license plate, in the embodiment of the present application, the virtual photo can also be post-processed, and the post-processed image can be used as the license plate image. The post-processing can include noise reduction, white balance and other processing. Among them, the noise reduction processing can be achieved by AI (artificial intelligence, artificial intelligence) technology, which is also not elaborated in detail here.

[0105] In addition, the virtual license plate constructed in the embodiment of the present application is used for subsequent training of the recognition model. Therefore, after generating a license plate image for the virtual license plate based on the license plate attribute information and the three-dimensional model through step 103, the license plate recognition result on the virtual license plate can also be marked in the license plate image to facilitate the subsequent use of the license plate recognition result and the license plate image as a training sample.

[0106] The license plate recognition result may be an attribute of the license plate used for recognition, such as the color of the license plate.

[0107] For another example, the license plate recognition result may be a license plate logo. At this time, when constructing a three-dimensional model of a virtual license plate, multiple characters on the virtual license plate are required to indicate the license plate logo. That is, in the process of generating a license plate image of a virtual license plate, the multiple characters used to indicate the license plate logo are known. Therefore, in the implementation of the present application, after generating a license plate image of a virtual license plate, the license plate logo on the virtual license plate may be directly annotated in the license plate image, so as to facilitate subsequent training of a neural network model directly based on the annotated license plate image. This avoids the need to manually annotate the license plate logo in the license plate image, thereby improving the efficiency of determining training samples. For another example, the license plate recognition result may include the license plate number and occlusion conditions.

[0108] In an embodiment of the present application, a virtual license plate can be simulated according to actual needs, and then a three-dimensional model of the virtual license plate can be constructed, and the license plate surface attribute information of the virtual license plate can be determined to generate a license plate image for the virtual license plate. Therefore, when determining the training sample for the neural network model used for license plate recognition, the license plate image can be directly generated through the embodiment of the present application according to actual needs, and there is no need to collect the real license plate through a camera to obtain the license plate image, which improves the efficiency of obtaining the license plate image. In addition, since the virtual license plate is a license plate simulated according to actual needs, the license plate images corresponding to different types of virtual license plates can be generated through the embodiment of the present application, which improves the diversity of the license plate images in the training samples, thereby improving the recognition accuracy of the neural network model trained based on the training samples.

[0109] Figure 2 Schematic diagram of a license plate image generation device provided in an embodiment of the present application. Figure 2 As shown, the device 200 includes:

[0110] A construction module 201 is used to construct a three-dimensional model of a virtual license plate, where the virtual license plate is a license plate simulated according to requirements;

[0111] A determination module 202, used to determine the license plate surface attribute information of the virtual license plate;

[0112] The generation module 203 is used to generate a license plate image for the virtual license plate according to the license plate surface attribute information and the three-dimensional model.

[0113] Optionally, construct a module specifically for:

[0114] Determine the shape of the border of the virtual license plate;

[0115] Determine the background color of the virtual license plate;

[0116] Determining a plurality of characters on a virtual license plate for indicating a license plate identification and an arrangement order of the plurality of characters;

[0117] A three-dimensional model of a virtual license plate is generated according to the shape of the frame, the background color, the multiple characters and the arrangement order of the multiple characters.

[0118] Optionally, the construction module is specifically used to determine the shape of the border of the virtual license plate according to the deformation of the virtual license plate.

[0119] Optionally, construct a module specifically for:

[0120] Generate a theoretical three-dimensional view of the virtual license plate according to the shape of the frame, the background color, the multiple characters, and the arrangement order of the multiple characters;

[0121] According to the crease condition of the virtual license plate, the theoretical three-dimensional view of the virtual license plate is adjusted to obtain a three-dimensional model of the virtual license plate.

[0122] Optionally, the license plate surface attribute information includes at least one or more of the following attribute information:

[0123] The metal properties of the virtual license plate;

[0124] Diffuse properties of virtual license plates;

[0125] The flatness of the virtual license plate;

[0126] Specular properties of virtual license plates;

[0127] Dust condition on the virtual license plate;

[0128] The condition of stains, coverage and occlusion on the virtual license plate.

[0129] Optionally, generate a module specifically for:

[0130] Determining shooting parameters of a virtual shooting scene;

[0131] Generate a virtual photo for the virtual license plate according to the shooting parameters, the license plate attribute information and the three-dimensional model;

[0132] The virtual photo is post-processed to obtain the license plate image.

[0133] Optionally, the shooting parameters of the virtual shooting scene include at least one or more of the following parameters:

[0134] The color of the light in the virtual shooting scene, the brightness of the light, the exposure parameters of the virtual camera in the virtual shooting scene, and the positions of the virtual camera and the virtual license plate in the virtual shooting scene.

[0135] Optionally, the device further comprises:

[0136] The labeling module is used to label the license plate recognition result on the virtual license plate in the license plate image.

[0137] In an embodiment of the present application, a virtual license plate can be simulated according to actual needs, and then a three-dimensional model of the virtual license plate can be constructed, and the license plate surface attribute information of the virtual license plate can be determined to generate a license plate image for the virtual license plate. Therefore, when determining the training sample for the neural network model used for license plate recognition, the license plate image can be directly generated through the embodiment of the present application according to actual needs, and there is no need to collect the real license plate through a camera to obtain the license plate image, which improves the efficiency of obtaining the license plate image. In addition, since the virtual license plate is a license plate simulated according to actual needs, the license plate images corresponding to different types of virtual license plates can be generated through the embodiment of the present application, which improves the diversity of the license plate images in the training samples, thereby improving the recognition accuracy of the neural network model trained based on the training samples.

[0138] It should be noted that: the device for generating a license plate image provided in the above embodiment only uses the division of the above functional modules as an example when generating a license plate image. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device for generating a license plate image provided in the above embodiment and the embodiment of the method for generating a license plate image belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0139] Figure 3 This is a block diagram of a terminal 300 provided in an embodiment of the present application. The terminal 300 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The terminal 300 may also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0140] Typically, the terminal 300 includes a processor 301 and a memory 302 .

[0141] The processor 301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 301 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 301 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 301 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 301 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0142] The memory 302 may include one or more computer-readable storage media, which may be non-transitory. The memory 302 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 302 is used to store at least one instruction, which is used to be executed by the processor 301 to implement the method for generating a license plate image provided in the method embodiment of the present application.

[0143] In some embodiments, the terminal 300 may further optionally include: a peripheral device interface 303 and at least one peripheral device. The processor 301, the memory 302 and the peripheral device interface 303 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 303 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 304, a touch display screen 305, a camera 306, an audio circuit 307, a positioning component 308 and a power supply 309.

[0144] The peripheral device interface 303 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 301 and the memory 302. In some embodiments, the processor 301, the memory 302, and the peripheral device interface 303 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 301, the memory 302, and the peripheral device interface 303 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0145] The radio frequency circuit 304 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 304 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 304 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 304 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 304 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 304 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0146] The display screen 305 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen 305 is a touch display screen, the display screen 305 also has the ability to collect touch signals on the surface or above the surface of the display screen 305. The touch signal can be input to the processor 301 as a control signal for processing. At this time, the display screen 305 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 305 can be one, and the front panel of the terminal 300 is set; in other embodiments, the display screen 305 can be at least two, which are respectively set on different surfaces of the terminal 300 or are folded; in some other embodiments, the display screen 305 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal 300. Even, the display screen 305 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 305 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode) and the like.

[0147] The camera assembly 306 is used to capture images or videos. Optionally, the camera assembly 306 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 306 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0148] The audio circuit 307 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 301 for processing, or input them into the radio frequency circuit 304 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 300. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 301 or the radio frequency circuit 304 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 307 may also include a headphone jack.

[0149] The positioning component 308 is used to locate the current geographical location of the terminal 300 to implement navigation or LBS (Location Based Service). The positioning component 308 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, Russia's Grenas system or the European Union's Galileo system.

[0150] The power supply 309 is used to power various components in the terminal 300. The power supply 309 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 309 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0151] In some embodiments, the terminal 300 further includes one or more sensors 310 , including but not limited to: an acceleration sensor 311 , a gyroscope sensor 312 , a pressure sensor 313 , a fingerprint sensor 314 , an optical sensor 315 , and a proximity sensor 316 .

[0152] The acceleration sensor 311 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 300. For example, the acceleration sensor 311 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 301 can control the touch display screen 305 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 311. The acceleration sensor 311 can also be used for collecting game or user motion data.

[0153] The gyro sensor 312 can detect the body direction and rotation angle of the terminal 300, and the gyro sensor 312 can cooperate with the acceleration sensor 311 to collect the user's 3D actions on the terminal 300. The processor 301 can implement the following functions based on the data collected by the gyro sensor 312: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0154] The pressure sensor 313 can be set on the side frame of the terminal 300 and / or the lower layer of the touch display screen 305. When the pressure sensor 313 is set on the side frame of the terminal 300, it can detect the user's holding signal of the terminal 300, and the processor 301 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 313. When the pressure sensor 313 is set on the lower layer of the touch display screen 305, the processor 301 controls the operability controls on the UI interface according to the user's pressure operation on the touch display screen 305. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0155] The fingerprint sensor 314 is used to collect the user's fingerprint, and the processor 301 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 314, or the fingerprint sensor 314 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 301 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 314 can be set on the front, back, or side of the terminal 300. When a physical button or a manufacturer logo is set on the terminal 300, the fingerprint sensor 314 can be integrated with the physical button or the manufacturer logo.

[0156] The optical sensor 315 is used to collect the ambient light intensity. In one embodiment, the processor 301 can control the display brightness of the touch display screen 305 according to the ambient light intensity collected by the optical sensor 315. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 305 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 305 is reduced. In another embodiment, the processor 301 can also dynamically adjust the shooting parameters of the camera assembly 306 according to the ambient light intensity collected by the optical sensor 315.

[0157] The proximity sensor 316, also called a distance sensor, is usually arranged on the front panel of the terminal 300. The proximity sensor 316 is used to collect the distance between the user and the front of the terminal 300. In one embodiment, when the proximity sensor 316 detects that the distance between the user and the front of the terminal 300 is gradually decreasing, the processor 301 controls the touch display screen 305 to switch from the screen-on state to the screen-off state; when the proximity sensor 316 detects that the distance between the user and the front of the terminal 300 is gradually increasing, the processor 301 controls the touch display screen 305 to switch from the screen-off state to the screen-on state.

[0158] Those skilled in the art will understand that Figure 3 The structure shown in the figure does not constitute a limitation on the terminal 300, and the terminal 300 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0159] The embodiment of the present application also provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by the processor of the terminal, the terminal can execute the method for generating the license plate image provided in the above embodiment.

[0160] The embodiment of the present application also provides a computer program product containing instructions, which, when executed on a terminal, enables the terminal to execute the method for generating a license plate image provided in the above embodiment.

[0161] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0162] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for generating a license plate image, characterized in that: The method comprises: Determine the shape of the border of the virtual license plate, the background color of the virtual license plate, multiple characters on the virtual license plate used to indicate the license plate identification, and the arrangement order of the multiple characters; generate a theoretical three-dimensional view of the virtual license plate based on the shape of the border, the background color, the multiple characters, and the arrangement order of the multiple characters, the theoretical three-dimensional view is a flat regular hexahedron, and according to the crease of the virtual license plate, the theoretical three-dimensional view of the virtual license plate is adjusted so that the corresponding corners of the regular hexahedron are bent to obtain a three-dimensional model of the virtual license plate; the shape of the border includes the shape of the outer frame and the shape of the inner frame, wherein the shape of the outer frame of the virtual license plate is determined. Shape, including: a plurality of frame options are displayed in the current display interface, each frame option indicates a frame shape, when a selection operation for any of the plurality of frame options is detected, a frame shape is determined based on the frame option corresponding to the selection operation, and the determined shape is adjusted according to the deformation of the virtual license plate, the deformation including the deformation caused by damage, the deformation caused by folding, and the deformation caused by wrinkles and twists, and the adjusted shape is used as the shape of the frame of the virtual license plate; wherein the obtained three-dimensional model of the virtual license plate is a three-dimensional model that can indicate the existence of folds, wrinkles or creases on the license plate; Determine the license plate surface attribute information of the virtual license plate, the license plate surface attribute information at least including one or more of the following attribute information: the metal attribute of the virtual license plate; the diffuse reflection attribute of the virtual license plate; the flatness of the virtual license plate; the mirror reflection attribute of the virtual license plate; the dust condition on the virtual license plate; the stain condition, coverage condition and occlusion condition on the virtual license plate; Generate a license plate image for the virtual license plate by ray tracing technology according to the license plate surface attribute information, the three-dimensional model and shooting parameters of the virtual scene, wherein the shooting parameters are used to indicate shooting factors that can affect the license plate image captured by the camera; The license plate recognition result on the virtual license plate is marked in the license plate image so that the license plate image and the license plate recognition result are used as a training sample for training a neural network model. The license plate recognition result includes multiple characters for indicating a license plate identification, a license plate color, and a license plate occlusion condition.

2. The method according to claim 1, characterized in that The step of generating a license plate image for the virtual license plate according to the license plate attribute information, the three-dimensional model and the shooting parameters of the virtual scene includes: Generate a virtual photo for the virtual license plate by ray tracing technology according to the shooting parameters, the license plate attribute information and the three-dimensional model; The virtual photo is post-processed to obtain the license plate image.

3. The method according to claim 2, characterized in that The shooting parameters of the virtual shooting scene include at least one or more of the following parameters: The color of the light in the virtual shooting scene, the brightness of the light, the exposure parameters of the virtual camera in the virtual shooting scene, and the positions of the virtual camera and the virtual license plate in the virtual shooting scene.

4. A license plate image generating device, characterized in that: The device comprises: A construction module, used to construct a three-dimensional model of a virtual license plate; A determination module, used to determine the license plate surface attribute information of the virtual license plate; A generation module, used to generate a license plate image for the virtual license plate by ray tracing technology according to the license plate surface attribute information, the three-dimensional model and the shooting parameters of the virtual scene, wherein the shooting parameters are used to indicate shooting factors that can affect the license plate image collected by the camera; Wherein, the building block is specifically used for: Determine the shape of the border of the virtual license plate, the frame shape of the frame includes the shape of the outer frame and the shape of the inner frame, wherein determining the shape of the outer frame of the virtual license plate includes: a plurality of outer frame options are displayed in the current display interface, each outer frame option indicates an outer frame shape, when a selection operation for any of the plurality of outer frame options is detected, determine a shape of an outer frame based on the outer frame option corresponding to the selection operation, and adjust the determined shape according to the deformation of the virtual license plate, the deformation including the deformation caused by damage, the deformation caused by folding, and the deformation caused by wrinkles and twists, and use the adjusted shape as the shape of the outer frame of the virtual license plate; Determining the background color of the virtual license plate; Determining a plurality of characters on the virtual license plate for indicating a license plate identification and an arrangement order of the plurality of characters; generating a three-dimensional model of the virtual license plate according to the shape of the frame, the background color, the multiple characters, and the arrangement order of the multiple characters; Wherein, the building block is specifically used for: Generating a theoretical three-dimensional view of the virtual license plate according to the shape of the frame, the background color, the multiple characters, and the arrangement order of the multiple characters, wherein the theoretical three-dimensional view is a flat regular hexahedron; According to the crease of the virtual license plate, the theoretical three-dimensional view of the virtual license plate is adjusted so that the corners corresponding to the regular hexahedron are bent to obtain a three-dimensional model of the virtual license plate; The license plate surface attribute information includes at least one or more of the following attribute information: The metal property of the virtual license plate; The diffuse reflection property of the virtual license plate; The flatness of the virtual license plate; The specular reflection property of the virtual license plate; Dust condition on the virtual license plate; The condition of stains, coverage and occlusion on the virtual license plate; The device also includes: A labeling module is used to label the license plate recognition result on the virtual license plate in the license plate image so that the license plate image and the license plate recognition result are used as a training sample for training the neural network model. The license plate recognition result includes multiple characters for indicating the license plate identification, the license plate color and the license plate occlusion condition.

5. The device according to claim 4, characterized in that The generating module is specifically used for: Generate a virtual photo for the virtual license plate by ray tracing technology according to the shooting parameters, the license plate attribute information and the three-dimensional model; The virtual photo is post-processed to obtain the license plate image.

6. The device according to claim 5, characterized in that The shooting parameters of the virtual shooting scene include at least one or more of the following parameters: The color of the light in the virtual shooting scene, the brightness of the light, the exposure parameters of the virtual camera in the virtual shooting scene, and the positions of the virtual camera and the virtual license plate in the virtual shooting scene.

7. A license plate image generating device, characterized in that: The device comprises: processor; a memory for storing processor-executable instructions; The processor is configured to execute the steps of the method described in any one of claims 1 to 3.

Citation Information

Patent Citations

  • Method of artificial-intelligence training data set generation used for object recognition

    CN108596259A

  • License plate image generation method, device, apparatus and medium based on style transfer

    CN109255772A