A method and device for recognizing license plate characters

Abstract

The present invention relates to the field of image processing technology, and discloses a method and device for recognizing license plate characters, obtaining a video image data set of vehicles in a preset area, wherein the video image data set includes an image frame of a target vehicle, and the image frame of the target vehicle includes a license plate character image and a license plate character text; preprocessing the video image data set; recognizing the preprocessed video image data set by constructing a preset license plate image recognition model, and completing the recognition of the license plate characters according to a preset loss function when the loss calculation result meets a preset condition. The method provided by the present invention reconstructs a more realistic and clear license plate character image, thereby improving the accuracy of license plate character recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for recognizing license plate characters. Background Art

[0002] License plate recognition technology plays an important role in many tasks such as urban traffic management, vehicle identification, parking fee management, and violation handling. Affected by many factors such as lighting conditions, weather conditions, image clarity, license plate shooting angle, and license plate color, license plate recognition is still a challenging task.

[0003] License plate recognition includes two tasks: one is the detection of the license plate position, that is, obtaining the position of the license plate area from the captured image; the other is the recognition of license plate characters, that is, recognizing the visible characters in the detected license plate area. At present, the main method is to install high-position video cameras on the roadside to capture roadside scene images and perform license plate recognition on the vehicles in the images. However, due to insufficient light at night, exposure of the license plate area due to reflections, and blurred license plate images due to high vehicle speeds, the image quality of the license plate area is low, making it difficult to accurately recognize other license plate characters.

[0004] Image super-resolution is a technology that improves image resolution through certain methods. It can reconstruct low-quality images into high-quality images. It is currently mainly achieved by using deep learning algorithms. For low-quality roadside parking scene images in dark light and other scenes, the resolution of the entire parking scene image is improved by using super-resolution algorithms, but the image of the license plate area is not processed separately. The image of the license plate area contains a large amount of text information, including Chinese abbreviations of different provinces with complex strokes, and characters of different colors or fonts for different car models. Using general super-resolution algorithms, it is difficult to reconstruct the detailed information of the license plate characters, and there is a defect of low accuracy in license plate character recognition in special scenes such as dark light. Summary of the invention

[0005] In view of this, the present invention provides a method and device for recognizing license plate characters to overcome the problem of low accuracy of license plate character recognition in special scenes such as dark light.

[0006] In a first aspect, the present invention provides a method for recognizing license plate characters, the method comprising:

[0007] Acquire a video image dataset of vehicles in a preset area, wherein the video image dataset includes an image frame of a target vehicle, and the image frame of the target vehicle includes a license plate character image and a license plate character text;

[0008] Preprocess the video image dataset;

[0009] By constructing a preset license plate image recognition model, the preprocessed video image data set is recognized. According to the preset loss function, when the loss calculation result meets the preset conditions, the recognition of the license plate characters is completed.

[0010] In an optional implementation, preprocessing the video image data set includes:

[0011] According to the preset screening resolution, images that meet the preset resolution conditions are screened out from the video image data set;

[0012] The screened images are subjected to image quality degradation processing to complete the preprocessing of the video image data set.

[0013] In an optional implementation, performing quality degradation processing on the image includes: downsampling the image and performing random mixed noise processing on the image.

[0014] In an optional implementation, the preset license plate image recognition model includes: a preset image diffusion model, a preset license plate character diffusion model and a preset multimodal hybrid model, wherein:

[0015] The preset image diffusion model is used to encode the image data and generate the feature representation Z after adding Gaussian white noise. T and low-resolution feature representation Z LR , where Z represents the feature representation encoded into the preset space. For Z, Gaussian noise is gradually added at each diffusion time step t to obtain Z T , t=1,...T-1,T;

[0016] The license plate character diffusion model is preset to predict the license plate character text and generate the license plate character text prediction result c T ;

[0017] A preset multimodal hybrid model is used to fuse the image features in the preset image diffusion model and the text information in the preset license plate character diffusion model. According to the preset loss function, when the loss calculation result meets the preset conditions, the recognition of the license plate characters is completed.

[0018] In an optional implementation, the image features in the preset image diffusion model and the text information in the preset license plate character diffusion model are fused, and according to the preset loss function, when the loss calculation result meets the preset conditions, the recognition of the license plate characters is completed, including:

[0019] The image features in the preset image diffusion model and the text information in the preset license plate character diffusion model are fused to generate text-guided image features I const-t and image-guided text features C const-t ;

[0020] At each diffusion time step t, the fused image feature representation I const-t , use the preset denoising model to gradually remove the noise and generate Z T-1 , until Z is obtained 0 ;

[0021] For the fused text feature representation C const-t , using the transformer-based decoder to gradually denoise the text features and generate c T-1 , until c is obtained 0 ;

[0022] Using the preset image feature decoder to decode Z 0 Perform image reconstruction to generate an image X that meets the preset resolution HR , using the preset text feature decoder to decode c 0 Perform text reconstruction to generate a license plate character text sequence that meets the preset resolution;

[0023] According to the preset loss function, when the image loss calculation result of the preset resolution and the license plate character text sequence loss calculation result of the preset resolution meet the preset conditions, the recognition of the license plate characters is completed.

[0024] In an optional implementation, the preset conditions include:

[0025] The image loss calculation result of the preset resolution and the license plate character text sequence loss calculation result of the preset resolution are both less than the first preset value,

[0026] or,

[0027] The image loss calculation result of the preset resolution is less than the second preset value,

[0028] or,

[0029] The calculation result of the license plate character text sequence loss at the preset resolution is less than the third preset value.

[0030] In an optional implementation, the preset loss function includes: an image reconstruction loss function and a license plate character text recognition loss function, wherein:

[0031] Image reconstruction loss function, used for calculation of vehicle images;

[0032] License plate character text recognition loss function is used to calculate the vehicle's license plate character text sequence.

[0033] In a second aspect, the present invention provides a device for recognizing license plate characters, the device comprising:

[0034] A data acquisition module, used to acquire a video image data set of vehicles in a preset area, wherein the video image data set includes an image frame of a target vehicle, and the image frame of the target vehicle includes a license plate character image and a license plate character text;

[0035] A preprocessing module, used for preprocessing the video image data set;

[0036] The license plate character recognition module is used to recognize the preprocessed video image data set by building a preset license plate image recognition model. According to the preset loss function, when the loss calculation result meets the preset conditions, the license plate character recognition is completed.

[0037] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the license plate character recognition method of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0038] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the license plate character recognition method of the first aspect or any corresponding embodiment thereof.

[0039] The license plate character recognition method and device provided by the present invention guide the preset image diffusion model by constructing a preset license plate image recognition model, so that the image diffusion model can reconstruct a more realistic license plate character image, and promote the mutual cooperation between text information and image information through the preset multimodal hybrid model, so as to reconstruct a more realistic and clear license plate character image. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 is a flow chart of a method for recognizing license plate characters according to an embodiment of the present invention;

[0042] Figure 2 is a structural block diagram of a device for recognizing license plate characters according to an embodiment of the present invention;

[0043] Figure 3 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0045] According to an embodiment of the present invention, an embodiment of a method for recognizing license plate characters is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0046] In this embodiment, a method for recognizing license plate characters is provided, which can be used in the above-mentioned mobile terminals, such as mobile phones, tablet computers, etc. Figure 1 is a flow chart of a method for recognizing license plate characters according to an embodiment of the present invention, such as Figure 1 As shown, the process includes the following steps:

[0047] Step S101, obtaining a video image dataset of vehicles in a preset area, wherein the video image dataset includes an image frame of a target vehicle, and the image frame of the target vehicle includes a license plate character image and a license plate character text.

[0048] In an embodiment of the present invention, video data of surveillance videos of preset areas are collected, such as different surveillance videos and image data in different cities, different streets, different time periods and different seasons, etc. This is only an example and is not limited to this. In actual applications, corresponding surveillance videos are selected according to actual conditions.

[0049] Specifically, the video data set includes at least one short video clip, each short video clip includes multiple frames of continuous images, the images contain information such as the target vehicle, and the image frames of the target vehicle include license plate character images and license plate character texts.

[0050] Step S102: pre-processing the video image data set.

[0051] In an embodiment of the present invention, preprocessing a video image data set includes: selecting images that meet preset resolution conditions from the video image data set according to a preset screening resolution; performing image quality degradation processing on the selected images to complete the preprocessing of the video image data set. The degradation processing includes: downsampling the image and performing random mixed noise processing on the image. This is only an example and is not intended to be limiting. In actual applications, corresponding preprocessing means and preset resolution conditions are selected according to actual conditions. For example, the preset resolution condition can be greater than a certain value, and the value is selected according to actual conditions.

[0052] Specifically, an image frame containing a vehicle license plate is extracted from the surveillance video, and an image of the license plate area is obtained using a license plate detection algorithm or a manual labeling method. The license plate detection algorithm includes but is not limited to using two-stage or single-stage target detection methods such as Faster-RCNN, YOLO, SSD, etc. to obtain the coordinate information of the license plate detection frame. Manual labeling refers to using a rectangular frame tool to obtain the coordinates of the license plate area, and then cropping the license plate image from the original surveillance image according to the license plate area coordinate information.

[0053] Specifically, vehicle images cover images of the same vehicle captured from real road traffic scene monitoring under different perspectives, different backgrounds, and different lighting intensities, wherein the content covered by the vehicle images includes: license plate appearance information, license plate character information, etc.

[0054] Specifically, the license plate character text information is obtained by manual annotation.

[0055] In a specific embodiment, high-resolution images in the data set are screened out, including but not limited to meeting the following conditions: (1) image resolution is greater than h*w, such as 64*192; (2) image aspect ratio is greater than 2.5. Data that meet the above conditions are high-resolution images, and image quality degradation processing is performed on these images, including but not limited to the following methods: (1) using a bilinear interpolation algorithm to perform image downsampling processing of 2 or 4 times; (2) adding random mixed noise processing to the image; (3) using an image degradation algorithm to perform image processing.

[0056] Step S103, by constructing a preset license plate image recognition model, the preprocessed video image data set is recognized, and according to a preset loss function, when the loss calculation result meets the preset conditions, the recognition of the license plate characters is completed.

[0057] In the embodiment of the present invention, the preset license plate image recognition model includes: a preset image diffusion model, a preset license plate character diffusion model and a preset multimodal hybrid model, wherein the preset image diffusion model is used to perform feature encoding on the image data to generate a feature representation Z after adding Gaussian white noise. Tand low-resolution feature representation Z LR , where Z represents the feature representation encoded into the preset space. For Z, Gaussian noise is gradually added at each diffusion time step t to obtain Z T , t=1,...T-1,T. The license plate character diffusion model is preset to predict the license plate character text and generate the license plate character text prediction result c T A preset multimodal hybrid model is used to fuse the image features in the preset image diffusion model and the text information in the preset license plate character diffusion model, and according to a preset loss function, when the loss calculation result meets the preset conditions, the recognition of the license plate characters is completed.

[0058] In an embodiment of the present invention, the image features in the preset image diffusion model and the text information in the preset license plate character diffusion model are fused, and according to the preset loss function, when the loss calculation result meets the preset condition, the recognition of the license plate character is completed, including: fusing the image features in the preset image diffusion model and the text information in the preset license plate character diffusion model to generate a text-guided image feature I const-t and image-guided text features C const-t At each diffusion time step t, the fused image feature representation I const-t , use the preset denoising model to gradually remove the noise and generate Z T-1 , until Z is obtained 0 For the fused text feature representation C const-t , using the transformer-based decoder to gradually denoise the text features and generate c T-1 , until c is obtained 0 . Use the preset image feature decoder to decode Z 0 Perform image reconstruction to generate an image X that meets the preset resolution HR , using the preset text feature decoder to decode c 0 Perform text reconstruction to generate a license plate character text sequence that meets the preset resolution. According to the preset loss function, when the image loss calculation result of the preset resolution and the license plate character text sequence loss calculation result of the preset resolution meet the preset conditions, the license plate character recognition is completed.

[0059] In the embodiment of the present invention, the preset condition includes: the image loss calculation result of the preset resolution and the license plate character text sequence loss calculation result of the preset resolution are both less than the first preset value, or the image loss calculation result of the preset resolution is less than the second preset value, or the license plate character text sequence loss calculation result of the preset resolution is less than the third preset value. The preset condition, the first preset value, the second preset value, and the third preset value are not limited here, and the corresponding values ​​and conditions are selected according to the actual situation.

[0060] In the embodiment of the present invention, the preset loss function includes: an image reconstruction loss function and a license plate character text recognition loss function, wherein the image reconstruction loss function is used for vehicle image calculation and the license plate character text recognition loss function is used for vehicle license plate character text sequence calculation.

[0061] In a specific embodiment, the preset image diffusion model performs the reverse denoising process of the diffusion model based on the low-resolution image and the character sequence to achieve image super-resolution. For example, for the image diffusion model, the mainstream diffusion model Stable diffusion model is adopted, which includes two parts, an encoder E and a decoder D, which are used to perform the forward process and the reverse diffusion process respectively. First, the above high-resolution image is encoded into the latent space, which is expressed as:

[0062] Z=E(X),

[0063] Among them, X represents the input high-resolution image and Z is the feature representation encoded into the latent space.

[0064] Then, at each diffusion time step t, Gaussian noise is gradually added to Z to obtain Z T , t=1,...T-1,T, in the reverse diffusion process, the noise is used to predict the network μ θ Used to remove noise.

[0065] At the same time, the low-resolution image X LR Perform feature encoding, expressed as:

[0066] Z LR =E(X LR ),

[0067] For the encoder E and the decoder D, a variational autoencoder (VAE) structure is used.

[0068] It is difficult for the preset image diffusion model to restore the license plate character image with high text character fidelity under inaccurate text priors. Therefore, the preset license plate character diffusion model is added to guide the preset image diffusion model.

[0069] The preset license plate character diffusion model is different from the preset image diffusion model. The preset image diffusion model is a continuous diffusion model, and the added noise conforms to the Gaussian distribution; while the preset license plate character text diffusion model is discrete, assuming that it conforms to the category distribution in the forward process.

[0070] Specifically, the preset license plate character text diffusion model includes two parts. First, the text prediction model is used to predict the license plate character text of the low-resolution license plate image, which is defined as P. The low-resolution license plate character image is input and the output is cT , the loss license plate character text prediction model includes but is not limited to the use of a connectionist temporal classification algorithm (Connectionist Temporal Classification, CTC).

[0071] Then c T The Z obtained in the above steps T and Z LR The preset multimodal hybrid module is fed into the system so that the license plate character text generation and image generation can be guided in a relevant manner, thereby generating a more reasonable and accurate license plate character image. For the license plate character text features output by the preset multimodal hybrid module, the transformer-based decoder is used to perform reverse denoising.

[0072] The role of the preset multimodal hybrid model is to promote the joint optimization of the image diffusion model and the text diffusion model, which is defined as MoM and includes two modules.

[0073] For the first module, the input is two parts, the image features Z T and Z LR The feature concatenation and character text prediction c T This module uses a UNet-based network structure to perform image feature Z T and Z LR The features of are concatenated for feature extraction, and the character text is predicted c T Mapped to the same feature size, the fused multimodal information is obtained, which is defined as the text-guided image feature, defined as I const-t ; For the downsampling and upsampling process of UNet, convolution and deconvolution modules with a step size of 2 are used. Each convolution and deconvolution module is connected to a normalization layer and a nonlinear activation function layer. The normalization layer includes but is not limited to an instance normalization layer, an adaptive instance normalization layer, etc.; the nonlinear activation layer includes but is not limited to nonlinear activation functions such as ReLU and Leaky ReLU.

[0074] For the second module, the input is the character text prediction c T , this module uses a transformer-based encoder to obtain image-guided text features, defined as C const-t; For the transformer-based encoder, the transformer network structure is adopted, with a total of 3 layers. The feature dimension of each layer can be set to 512 dimensions. For each layer of transformer, it is divided into two sub-layers. The first sub-layer is a multi-head attention module, where the multi-head can be set to 8-head attention; the second sub-layer is a standard feedforward neural network; for each transformer layer, the output of each multi-head attention is residually connected with the input of the current encoder layer, and the output result is sent to the normalization layer, which adopts a layer normalization layer; then the normalized result is sent to the feedforward neural network layer, and then the residual connection calculation and the layer normalization calculation are performed again, and the result is sent to the next transformer layer for repetition.

[0075] For the multimodal hybrid module, it is specifically expressed as:

[0076] I const-t , C const-t =MoM([Z T ,Z LR ], c T , t).

[0077] In another specific embodiment, the overall license plate character image super-resolution is achieved according to a preset image diffusion model, a preset license plate character diffusion model, and a preset multimodal hybrid model:

[0078] (1) First, for image features, the VAE-based feature encoder E is used to encode the high-resolution image and the low-resolution image, respectively, to obtain Z T ,Z LR ; For the license plate character text features, the license plate character text prediction model P is used to perform text prediction and obtain c T ;

[0079] (2) Then, at each diffusion step t, for Z T ,Z LR , c T , the multimodal fusion model is used to fuse image and text features to promote the super-resolution of license plate images. The output is represented as I const-t , C const-t ;

[0080] (3) Then, at each diffusion step t, the fused image feature representation I const-t , use the denoising model to gradually remove the noise and get Z T-1 , gradually get Z 0 ;

[0081] (4) For the fused text feature representation C const-t, the text features are gradually denoised using a transformer-based decoder to obtain c T-1 , gradually get c 0 ;

[0082] (5) Finally, for the image diffusion model, Z 0 Use the image feature decoder V to reconstruct the image and obtain a high-resolution image X HR ; For the text diffusion model, c 0 Perform text decoding to obtain the license plate character text sequence. When the loss calculation result meets the preset conditions, the license plate character recognition is completed. The model loss function includes two parts: image reconstruction loss function and license plate character text recognition loss function. For the image reconstruction loss function, the L1 norm loss function can be used for calculation; for the license plate character text recognition loss function, the CTC loss function can be used.

[0083] The license plate character recognition method provided in this embodiment realizes the restoration of the real style of the license plate image by utilizing a powerful diffusion model. The diffusion model can not only simulate the distribution of real images, but also be used to learn the distribution of text. The prior knowledge of text is of great significance for the restoration and reconstruction of license plate character images. Therefore, the present invention guides the image diffusion model by constructing a text character diffusion model, so that the image diffusion model can reconstruct a more realistic license plate character image. In addition, a multimodal hybrid module is added to promote the cooperation between the text diffusion model and the image diffusion model, so as to reconstruct a more realistic and clear license plate character image and improve the accuracy of license plate character recognition.

[0084] In this embodiment, a device for recognizing license plate characters is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0085] This embodiment provides a license plate character recognition device, such as Figure 2 As shown, including:

[0086] The data acquisition module 201 is used to acquire a video image dataset of vehicles in a preset area, wherein the video image dataset includes an image frame of a target vehicle, and the image frame of the target vehicle includes a license plate character image and a license plate character text.

[0087] The preprocessing module 202 is used to preprocess the video image data set.

[0088] The license plate character recognition module 203 is used to recognize the preprocessed video image data set by constructing a preset license plate image recognition model, and complete the recognition of the license plate characters according to the preset loss function when the loss calculation result meets the preset conditions.

[0089] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0090] The license plate character recognition device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0091] The embodiment of the present invention also provides a computer device having the above Figure 3 The device for recognizing license plate characters is shown.

[0092] See also Figure 3 , Figure 3 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 3 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to an interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 3 A processor 10 is taken as an example.

[0093] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0094] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0095] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0096] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0097] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0098] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0099] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for recognizing license plate characters, characterized in that: The method comprises: Acquire a video image dataset of vehicles in a preset area, wherein the video image dataset includes an image frame of a target vehicle, and the image frame of the target vehicle includes a license plate character image and a license plate character text; Preprocess the video image dataset; By constructing a preset license plate image recognition model, the preprocessed video image data set is recognized. According to the preset loss function, when the loss calculation result meets the preset conditions, the recognition of the license plate characters is completed.

2. The method according to claim 1, characterized in that Preprocessing of video image datasets includes: According to the preset screening resolution, images that meet the preset resolution conditions are screened out from the video image data set; The screened images are subjected to image quality degradation processing to complete the preprocessing of the video image data set.

3. The method according to claim 2, characterized in that The quality degradation processing of the image includes: downsampling the image and performing random mixed noise processing on the image.

4. The method according to claim 1, characterized in that The preset license plate image recognition model includes: a preset image diffusion model, a preset license plate character diffusion model and a preset multimodal hybrid model, wherein: The preset image diffusion model is used to encode the image data and generate the feature representation Z after adding Gaussian white noise. T and low-resolution feature representation Z LR , where Z represents the feature representation encoded into the preset space. For Z, Gaussian noise is gradually added at each diffusion time step t to obtain Z T , t=1,...T-1,T; The license plate character diffusion model is preset to predict the license plate character text and generate the license plate character text prediction result c T ; A preset multimodal hybrid model is used to fuse the image features in the preset image diffusion model and the text information in the preset license plate character diffusion model. According to the preset loss function, when the loss calculation result meets the preset conditions, the recognition of the license plate characters is completed.

5. The method according to claim 4, characterized in that The image features in the preset image diffusion model and the text information in the preset license plate character diffusion model are fused. According to the preset loss function, when the loss calculation result meets the preset conditions, the license plate character recognition is completed, including: The image features in the preset image diffusion model and the text information in the preset license plate character diffusion model are fused to generate text-guided image features I const-t and image-guided text features C const-t ; At each diffusion time step t, the fused image feature representation I const-t , use the preset denoising model to gradually remove the noise and generate Z T-1 , until Z0 is obtained; For the fused text feature representation C const-t , using the transformer-based decoder to gradually denoise the text features and generate c T-1 , until c0 is obtained; Use the preset image feature decoder to reconstruct the image Z0 and generate an image X that meets the preset resolution. HR , using a preset text feature decoder to reconstruct the text of c0 and generate a license plate character text sequence that meets the preset resolution; According to the preset loss function, when the image loss calculation result of the preset resolution and the license plate character text sequence loss calculation result of the preset resolution meet the preset conditions, the recognition of the license plate characters is completed.

6. The method according to claim 5, characterized in that The pre-conditions include: The image loss calculation result of the preset resolution and the license plate character text sequence loss calculation result of the preset resolution are both less than the first preset value, or, The image loss calculation result of the preset resolution is less than the second preset value, or, The calculation result of the license plate character text sequence loss at the preset resolution is less than the third preset value.

7. The method according to claim 6, characterized in that The preset loss function includes: image reconstruction loss function and license plate character text recognition loss function, wherein, Image reconstruction loss function, used for calculation of vehicle images; License plate character text recognition loss function is used to calculate the vehicle's license plate character text sequence.

8. A device for recognizing license plate characters, characterized in that: The device comprises: A data acquisition module, used to acquire a video image data set of vehicles in a preset area, wherein the video image data set includes an image frame of a target vehicle, and the image frame of the target vehicle includes a license plate character image and a license plate character text; A preprocessing module, used for preprocessing the video image data set; The license plate character recognition module is used to recognize the preprocessed video image data set by building a preset license plate image recognition model. According to the preset loss function, when the loss calculation result meets the preset conditions, the license plate character recognition is completed.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the license plate character recognition method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the license plate character recognition method according to any one of claims 1 to 7.

Images