License plate recognition method and device, and license plate recognition model training method and equipment

Abstract

The embodiment of the invention provides a license plate recognition method and device and a license plate recognition model training method and equipment, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring target video data of a target vehicle; performing image frame extraction on the target video data to obtain a target license plate region image set; performing feature extraction on each image in the target license plate region image set through a pre-trained license plate recognition model to obtain a target license plate feature map; performing feature fusion on the target license plate feature map through the license plate recognition model and the attention weight corresponding to the target license plate feature map to obtain target license plate fusion data; and analyzing the target license plate fusion data through the license plate recognition model to obtain a target license plate recognition result of the target vehicle. The embodiment of the invention can improve the accuracy of license plate recognition.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a license plate recognition method and device, and a license plate recognition model training method and equipment. Background Art

[0002] License plate recognition refers to the process of character recognition on license plates.

[0003] In the related art, license plate recognition usually requires that the vehicle is not moving or is moving slowly, and there is brighter ambient light, so that a clearer license plate image can be obtained. However, in actual situations, the vehicle is usually in a state of fast movement and / or the ambient light is more complex. At this time, the accuracy of the license plate recognition method in the related art will be reduced. Based on this, how to provide a license plate recognition method that can accurately recognize the license plate when the vehicle is moving faster and / or the ambient light is more complex has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a license plate recognition method and device, a license plate recognition model training method and equipment, aiming to improve the accuracy of license plate recognition.

[0005] To achieve the above object, a first aspect of an embodiment of the present application provides a license plate recognition method, the method comprising:

[0006] Acquire target video data of a target vehicle;

[0007] Extracting image frames from the target video data to obtain a target license plate area image set;

[0008] Inputting the target license plate area image set into a pre-trained license plate recognition model;

[0009] Extracting features from each image in the target license plate area image set by using the license plate recognition model to obtain a target license plate feature map;

[0010] Performing feature fusion on the target license plate feature map through the license plate recognition model and the attention weight corresponding to the target license plate feature map to obtain target license plate fusion data;

[0011] The target license plate fusion data is parsed by the license plate recognition model to obtain a target license plate recognition result of the target vehicle.

[0012] In some embodiments, the target license plate area image set includes a first license plate area image, a second license plate area image, and a third license plate area image, the second license plate area image is a front frame image of the first license plate area image, the third license plate area image is a rear frame image of the first license plate area image, and the target license plate feature map includes a first feature map corresponding to the first license plate area image, a second feature map corresponding to the second license plate area image, and a third feature map corresponding to the third license plate area image;

[0013] The step of performing feature fusion on the target license plate feature map by using the license plate recognition model and the attention weight corresponding to the target license plate feature map to obtain target license plate fusion data includes:

[0014] Determine, by means of the license plate recognition model, a characteristic license plate region corresponding to a target license plate region in the first license plate region image in the first characteristic image;

[0015] Determine the attention weight corresponding to the second feature map based on the characteristic license plate area, and calculate the first attention feature map according to the second feature map and the attention weight corresponding to the second feature map;

[0016] Determine the attention weight corresponding to the third feature map based on the feature license plate area, and calculate the second attention feature map according to the third feature map and the attention weight corresponding to the third feature map;

[0017] Feature fusion is performed on the first attention feature map and the second attention feature map to obtain the target license plate fusion data.

[0018] In some embodiments, determining the attention weight corresponding to the second feature map based on the feature license plate area, and calculating the first attention feature map according to the second feature map and the attention weight corresponding to the second feature map, includes:

[0019] Determining a plurality of regional feature vectors in the characteristic license plate region;

[0020] Calculate the attention weight corresponding to the second feature map based on each of the regional feature vectors and the corresponding feature vector in the second feature map;

[0021] Performing a weighted sum operation on the second feature map based on the attention weight corresponding to the second feature map and obtaining a sub-attention feature map;

[0022] The sub-attention feature maps are arranged to obtain the first attention feature map.

[0023] In some embodiments, the step of performing feature fusion on the first attention feature map and the second attention feature map to obtain the target license plate fusion data includes:

[0024] Performing feature fusion on the first attention feature map and the second attention feature map to obtain a total attention feature map;

[0025] Superimposing the total attention feature map with the image corresponding to the characteristic license plate area to obtain a target feature map;

[0026] Performing a size transformation operation on the target feature map to obtain a transformed feature map;

[0027] Feature extraction is performed on the transformed feature map to obtain the target license plate fusion data.

[0028] In some embodiments, determining, by the license plate recognition model, a characteristic license plate region corresponding to a target license plate region in the first license plate region image in the first characteristic map includes:

[0029] Determine the target coordinate data of the target license plate area and the image size of the first feature map by using the license plate recognition model;

[0030] Calculating feature coordinate data based on a preset model scaling factor, the image size and the target coordinate data;

[0031] The characteristic license plate area is determined in the first characteristic map based on the characteristic coordinate data.

[0032] To achieve the above purpose, a second aspect of an embodiment of the present application proposes a license plate recognition model training method, the method comprising:

[0033] Acquire sample video data of a sample vehicle and sample license plate data of the sample vehicle;

[0034] Extracting image frames from the sample video data to obtain a sample license plate area image set;

[0035] In each round of iteration, the sample license plate area image set is input into the current license plate recognition model to be trained;

[0036] Extracting features from each image in the sample license plate area image set using the current license plate recognition model to be trained to obtain a sample license plate feature map;

[0037] Performing feature fusion on the sample license plate feature map through the current license plate recognition model to be trained and the attention weight corresponding to the sample license plate feature map to obtain sample license plate fusion data;

[0038] Parsing the sample license plate fusion data through the current license plate recognition model to be trained to obtain predicted license plate data of the sample vehicle;

[0039] According to the predicted license plate data and the sample license plate data, the parameters of the current license plate recognition model to be trained are iteratively adjusted until an iteration stop condition is met.

[0040] In some embodiments, extracting image frames from the sample video data to obtain a sample license plate area image set includes:

[0041] Extracting image frames from the sample video data to obtain a license plate image set;

[0042] Performing license plate region recognition on any image in the license plate image set according to a preset license plate detection model to obtain a sample license plate region;

[0043] Performing region amplification processing on the sample license plate region to obtain an amplified license plate region;

[0044] The images in the license plate image set are cropped according to the amplified license plate area to obtain the sample license plate area image set.

[0045] To achieve the above-mentioned purpose, a third aspect of an embodiment of the present application provides a license plate recognition device, the device comprising:

[0046] A data acquisition module, used to acquire target video data of a target vehicle;

[0047] An image frame extraction module is used to extract image frames from the target video data to obtain a target license plate area image set;

[0048] A feature extraction module is used to extract features from each image in the target license plate area image set using a pre-trained license plate recognition model to obtain a target license plate feature map;

[0049] A feature fusion module, used to perform feature fusion on the target license plate feature map through the license plate recognition model and the attention weight corresponding to the target license plate feature map to obtain target license plate fusion data;

[0050] The parsing module is used to parse the target license plate fusion data through the license plate recognition model to obtain the target license plate recognition result of the target vehicle.

[0051] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the method described in the first aspect above, or implements the method described in the second aspect above.

[0052] To achieve the above-mentioned purpose, the fifth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method described in the first aspect above, or implements the method described in the second aspect above.

[0053] The license plate recognition method and device, license plate recognition model training method and equipment proposed in the present application can obtain a target license plate area image set by extracting image frames from target video data. The target license plate recognition result can be obtained based on the pre-trained license plate recognition model and the target license plate area image set. It can be seen that the embodiment of the present application performs license plate recognition on the target vehicle based on the target license plate area image set. Compared with the method of performing license plate recognition based on only a single frame image in the related art, the license plate recognition method based on the license plate recognition model and the target license plate area image set in the embodiment of the present application can improve the accuracy of license plate recognition, and can also perform license plate recognition on the target vehicle in motion. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of a license plate recognition method provided by an embodiment of the present application;

[0055] Figure 2 is a schematic diagram of a license plate recognition model provided in an embodiment of the present application;

[0056] Figure 3 yes Figure 1 Flow chart of step S104 in FIG.

[0057] Figure 4 is a schematic diagram of a target license plate area provided in an embodiment of the present application;

[0058] Figure 5 yes Figure 3 Flow chart of step S301 in FIG.

[0059] Figure 6 yes Figure 3 Flow chart of step S302 in FIG.

[0060] Figure 7 yes Figure 3 Flow chart of step S304 in FIG.

[0061] Figure 8 is a flow chart of the license plate recognition model training method provided in an embodiment of the present application;

[0062] Fig.9A yes Figure 8 Flowchart of step S802 in FIG.

[0063] Fig. 9Bis a schematic diagram of a sample license plate area provided in an embodiment of the present application;

[0064] Fig.10 is a flow chart of another embodiment of the license plate recognition model training method provided in an embodiment of the present application;

[0065] Fig.11A is a schematic diagram of the structure of a license plate recognition device provided in an embodiment of the present application;

[0066] Fig. 11B It is a structural schematic diagram of a license plate recognition model training device provided in an embodiment of the present application;

[0067] Fig.12 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0069] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0071] First, some nouns involved in this application are analyzed:

[0072] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0073] License plate recognition refers to the process of character recognition on license plates.

[0074] In the related art, when performing license plate recognition, it is usually required that the vehicle is in a stationary state or a slow-moving state, and there is brighter ambient light, so that a clearer license plate image can be obtained. For example, license plate recognition can usually be applied to sentry booths, in which the vehicle is usually in a stationary state or a slow-moving state, and there is brighter ambient light in the scene. For scenes other than sentry booths, vehicles usually move quickly, and the ambient light of the scene is complex. The vehicle image obtained in this scene is usually blurred, which affects the accuracy of license plate recognition.

[0075] In addition, the related art usually performs license plate recognition based on a single frame image. Specifically, the license plate area in the image is first recognized, and then the license plate number is recognized based on the license plate area. Among them, the methods for performing license plate number recognition include the following two:

[0076] The first method uses image processing methods to perform binary segmentation to obtain single characters, and then uses pattern matching methods to recognize single characters;

[0077] The second method is to input the image corresponding to the license plate area into the neural network model for character recognition, and then perform recognition and error correction based on the paradigm of the license plate.

[0078] Among them, the first method requires less computing resources, but has poor robustness, and the recognition effect is not good under poor lighting conditions, license plate stains, etc. The second method has good robustness, but requires more computing resources.

[0079] Based on this, the embodiments of the present application provide a license plate recognition method and device, a license plate recognition model training method and equipment, aiming to improve the accuracy of license plate recognition.

[0080] The license plate recognition method and device, and the license plate recognition model training method and equipment provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the license plate recognition method in the embodiments of the present application is described.

[0081] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0082] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0083] The license plate recognition method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The license plate recognition method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the license plate recognition method, etc., but is not limited to the above forms.

[0084] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0085] Figure 1 is an optional flow chart of the license plate recognition method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105.

[0086] Step S101, obtaining target video data of a target vehicle;

[0087] Step S102, extracting image frames from the target video data to obtain a target license plate area image set;

[0088] Step S103, extracting features from each image in the target license plate area image set using a pre-trained license plate recognition model to obtain a target license plate feature map;

[0089] Step S104, performing feature fusion on the target license plate feature map through the license plate recognition model and the attention weight corresponding to the target license plate feature map to obtain target license plate fusion data;

[0090] Step S105, analyzing the target license plate fusion data through the license plate recognition model to obtain a target license plate recognition result of the target vehicle.

[0091] In steps S101 to S105 shown in the embodiment of the present application, by extracting image frames from the target video data, a target license plate area image set can be obtained. The target license plate recognition result can be obtained based on the pre-trained license plate recognition model and the target license plate area image set. It can be seen that the embodiment of the present application performs license plate recognition on the target vehicle based on the target license plate area image set. Compared with the method of performing license plate recognition based on only a single frame image in the related art, the license plate recognition method based on the license plate recognition model and the target license plate area image set in the embodiment of the present application can improve the accuracy of license plate recognition, and can also perform license plate recognition on the target vehicle in motion.

[0092] In step S101 of some embodiments, the target vehicle may refer to a vehicle that needs to be license plate recognized, and the target vehicle may be in motion or at rest, which is not specifically limited in the embodiments of the present application. The target video data may refer to data obtained by photographing the target vehicle based on a preset camera device, and the duration of the target video data may be adaptively set according to actual needs, which is not specifically limited in the embodiments of the present application. For example, in a sentry booth scene, the camera device may be installed at the sentry booth, and the target video data may be 20 seconds of video data before the target vehicle arrives at the sentry booth and stops moving.

[0093] In step S102 of some embodiments, image frame extraction may refer to image extraction of sample video data based on a frame rate. Based on image frame extraction, multiple images may be obtained, and a target license plate area image set may be obtained by aggregating the multiple images. It is understandable that the number of extracted images may be adaptively set according to actual needs, and the embodiments of the present application do not specifically limit this. For example, when a higher license plate recognition accuracy is required, a larger number of images may be extracted from the target video data. When a faster license plate recognition speed is required, a smaller number of images may be extracted from the target video data. It is understandable that in actual applications, the shooting frame rates of different camera devices may be different. In order to improve the accuracy of license plate recognition, image frame extraction may be performed based on the shooting frame rate of the camera device. For example, image I may be extracted from the target video data. i , and extract image I i The first k images and the last k images of the target license plate area are collected together to obtain the target license plate area image set C = {C i-k ,...,C i-1 ,C i ,C i+1 ,...,C i+k}.

[0094] In step S103 of some embodiments, the license plate recognition model may refer to a pre-trained model with license plate recognition capability. The target license plate area image set is used as input data of the license plate recognition model. Figure 2As shown, the license plate recognition model may include a feature extraction module, a feature fusion module and a license plate recognition module. The feature extraction module may refer to a module built based on a convolutional neural network (CNN), and the feature extraction module may include two feature extraction blocks (such as CNN Block1 and CNN Block2). Each feature extraction block may include structures such as convolution, pooling, activation function, batch normalization and jump connection. The target license plate area image set is used as the input data of the feature extraction module. After being processed by the two feature extraction blocks, multiple target license plate feature maps can be obtained. For example, when k=2, corresponding to the target license plate area image set C={C i-2 ,C i-1 ,C i ,C i+1 ,C i+2} can get F i-2 、F i-1 、F i 、F i+1 and F i+2 There are five target license plate feature images in total. It is understandable that each feature extraction block can reduce the height (H) and width (W) of the input data to 1 / 2 of the original size. Therefore, after two feature extraction blocks, the height and width of the target license plate feature image are reduced to 1 / 4 of the height and width of the image in the target license plate area image set. It is understandable that before the target license plate area image set is input into the feature extraction module, each image in the target license plate area image set can also be grayscale processed.

[0095] In step S104 of some embodiments, the attention weight can be used to indicate the importance of the corresponding target license plate feature map. Feature fusion processing can be performed on multiple target license plate feature maps based on the feature fusion module and the attention weight to fuse the license plate information of the multiple target license plate feature maps to obtain target license plate fusion data.

[0096] Reference Figure 3 In some embodiments, the target license plate area image set includes a first license plate area image, a second license plate area image, and a third license plate area image, the second license plate area image is a front frame image of the first license plate area image, and the third license plate area image is a rear frame image of the first license plate area image. The license plate feature map includes a first feature map corresponding to the first license plate area image, a second feature map corresponding to the second license plate area image, and a third feature map corresponding to the third license plate area image. Step S104 includes but is not limited to steps S301 to S304.

[0097] Step S301, determining a characteristic license plate region corresponding to a target license plate region in a first license plate region image in a first characteristic map through a license plate recognition model;

[0098] Step S302, determining the attention weight corresponding to the second feature map based on the feature license plate area, and calculating the first attention feature map according to the second feature map and the attention weight corresponding to the second feature map;

[0099] Step S303, determining the attention weight corresponding to the third feature map based on the feature license plate area, and calculating the second attention feature map according to the third feature map and the attention weight corresponding to the third feature map;

[0100] Step S304, performing feature fusion on the first attention feature map and the second attention feature map to obtain target license plate fusion data.

[0101] It should be noted that the previous frame image may refer to the previous k images, and the next frame image may refer to the next k images. For example, when k=2, the first license plate area image may be image C i , the second license plate area image may include image C i-2 and image C i-1 , the third license plate area image may include image C i+1 and image C i+2 Correspondingly, the first feature map can be the feature map F i , the second feature map may include a feature map F i-2 and feature map F i-1 , the third feature map may include the feature map F i+1 and feature map F i+2 .

[0102] In step S301 of some embodiments, referring to Figure 4 The target license plate area b2 may refer to a license plate area image (such as the first license plate area image C i ) is the area where the license plate is located. Feature license plate area b0 * It can refer to the area corresponding to the target license plate area b0 in the first feature map. Feature license plate area b0 * It can be obtained by size transformation mapping.

[0103] Reference Figure 5 In some embodiments, step S301 includes but is not limited to steps S501 to S503.

[0104] Step S501, determining target coordinate data of a target license plate area and an image size of a first feature map through a license plate recognition model;

[0105] Step S502, calculating feature coordinate data based on a preset model scaling factor, image size and target coordinate data;

[0106] Step S503: determine the characteristic license plate area in the first characteristic map based on the characteristic coordinate data.

[0107] In step S501 of some embodiments, the target coordinate data may refer to the coordinates of the target license plate area b0 in the first license plate area image, such as [x1, y1, x2, y2]. The image size of the first feature map may include a width size and a height size, such as (W, H).

[0108] In step S502 of some embodiments, the model scaling factor may refer to the scaling factor of the input image by the feature extraction module, such as r=min(W * / W,H * / H). Among them, (W * ,H * ) represents the input size of the feature extraction module. When the first license plate area image is input to the feature extraction module, the first license plate area image needs to be scaled and filled to (W * ,H * ). From the above description, we can see that each feature extraction block can reduce the height (H) and width (W) of the input data to 1 / 2 of the original size. Therefore, the image size of the first feature map is (W * / 4,H * / 4). Based on the model scaling factor, image size and target coordinate data, the feature coordinate data can be calculated as [x1*r / 4, y1*r / 4, x2*r / 4, y2*r / 4].

[0109] In step S503 of some embodiments, a region position is determined in the first feature map based on the feature coordinate data, and the determined region position is used as the feature license plate region b0. * .

[0110] In step S302 of some embodiments, based on the relationship between the characteristic license plate area and the second characteristic map, the importance of the second characteristic map is determined, and the attention weight corresponding to the second characteristic map is obtained. i-2 The attention weight W i-2 , and get the second feature map F i-1 The attention weight W i-1 , so that the first attention feature map V can be calculated i-2 =W i-2 *F i-2 and the first attention feature map V i-1 =W i-1 *Fi-1 .

[0111] Reference Figure 6 In some embodiments, step S302 includes but is not limited to steps S601 to S604.

[0112] Step S601, determining a plurality of regional feature vectors in a characteristic license plate region;

[0113] Step S602, calculating the attention weight corresponding to the second feature map based on the feature vector of each region and the feature vector of the second feature map;

[0114] Step S603, performing a weighted sum operation on the second feature map based on the attention weight corresponding to the second feature map to obtain a sub-attention feature map;

[0115] Step S604, arrange the sub-attention feature maps to obtain a first attention feature map.

[0116] In step S601 of some embodiments, the characteristic license plate area is part of the first characteristic map, and the characteristic map (such as the first characteristic map) usually has the characteristics of three dimensions (H, W, C), so the characteristic license plate area also has the characteristics of these three dimensions. Among them, H represents the height dimension, W represents the width dimension, and C represents the number of channels (such as the number of RGB three color channels). The regional feature vector can refer to a vector with a length of C corresponding to each pixel position in the characteristic license plate area, and there are a total of H*W regional feature vectors.

[0117] In step S602 of some embodiments, the following operation may be performed on each regional feature vector: the regional feature vector is calculated with each feature vector in the second feature map to obtain an attention weight w corresponding to each feature vector in the second feature map. i-2 Among them, the method of calculating the regional feature vector and each feature vector in the second feature map may include the following:

[0118] The first method is to perform dot product calculation on the regional feature vector and each feature vector in the second feature map to determine the similarity between the two vectors, and use the dot product calculation result as the attention weight.

[0119] The second method connects the regional feature vector with each feature vector in the second feature map, applies linear transformation to calculate the connection result, and uses the linear transformation result as the attention weight. The specific calculation formula is as follows:

[0120] Linear transformation (concat(vector1, vector2))

[0121] Among them, vector 1 can represent a feature vector, and vector 2 can represent a feature vector in the second feature map.

[0122] The third method is to calculate the attention weight as follows:

[0123] (vector3·vector4) / sqrt(vector dimension)

[0124] Among them, vector 3 can represent the regional feature vector, and vector 4 can represent the feature vector in the second feature map. The vector dimension can represent the dimension of the regional feature vector. From the calculation method of the third method, it can be seen that the third method can be regarded as an improvement of the first method, and the third method can alleviate the influence of the vector dimension on the attention weight.

[0125] In step S603 of some embodiments, the calculated multiple attention weights w i-2 The weighted summation is performed with the feature vector in the corresponding second feature map to obtain the sub-attention feature map v * i-2 .

[0126] In step S604 of some embodiments, it can be understood that a sub-attention feature map v can be calculated based on each region feature vector and the feature vector in the second feature map. * i-2 . All sub-attention feature maps v * i-2 Arrange and get the first sub-attention feature map V i-2 .

[0127] It is understandable that the attention weight W i-2 There can be multiple attention weights w i-2 A collection of .

[0128] In step S303 of some embodiments, the attention weight corresponding to the third feature map can be calculated similarly to the method described in steps S601 to S604. i+2 The attention weight W i+2 , and get the second feature map F i+1 The attention weight W i+1 , so that the second attention feature map V can be calculated i+2 =W i+2 *F i+2 and the second attention feature map V i+1 =W i+1 *F i+1 .

[0129] In step S304 of some embodiments, the first attention feature map and the second attention feature map are superimposed and fused at corresponding positions to obtain target license plate fusion data. It can be understood that the target license plate fusion data fuses the license plate information contained in the first feature map, the second feature map and the third feature map.

[0130] The advantage of steps S301 to S304 is that the license plate information contained in the first feature map, the second feature map and the third feature map can be fused, and the subsequent license plate recognition based on the fusion result (i.e., the target license plate fusion data) can improve the recognition accuracy and reduce the low accuracy of license plate recognition due to possible information missing in a single image.

[0131] Reference Figure 2 and Figure 7 In some embodiments, step S304 includes but is not limited to steps S701 to S704.

[0132] Step S701, performing feature fusion on the first attention feature map and the second attention feature map to obtain a total attention feature map;

[0133] Step S702, superimposing the total attention feature map with the image corresponding to the characteristic license plate area to obtain a target feature map;

[0134] Step S703, performing a size transformation operation on the target feature map to obtain a transformed feature map;

[0135] Step S704, extracting features from the transformed feature map to obtain target license plate fusion data.

[0136] In step S701 of some embodiments, the first attention feature map and the second attention feature map are added and fused at corresponding positions to obtain a total attention feature map Att=W i-2 *F i-2 +W i-1 *F i-1 +W i+1 *F i+1 +W i+2 *F i+2 .

[0137] In step S702 of some embodiments, the total attention feature map Att and the image corresponding to the characteristic license plate area are superimposed in the channel direction to obtain a target feature map.

[0138] In step S703 of some embodiments, the size transformation operation may refer to transforming the high size of the target feature map, such as using a linear interpolation method to scale the high size of the target feature map to obtain a transformed feature map. It is understandable that the size of the proportional scaling can be adaptively set according to actual conditions, and the embodiments of the present application do not impose specific restrictions on this. For example, the high size of the transformed feature map obtained by proportional scaling can be 8.

[0139] In step S704 of some embodiments, the method of extracting features from the transformed feature graph is similar to the method of extracting features based on the feature extraction module, that is, the feature extraction block can be set to extract features from the transformed feature graph to obtain the target license plate fusion data. Figure 2 As shown, three feature extraction blocks, CNN Block3, CNN Block4 and CNNBlock5, can be set. After the transformed feature map passes through these three feature extraction blocks, it will be transformed into a feature map with a height size of 1 (i.e., the target license plate fusion data).

[0140] In step S105 of some embodiments, the target license plate fusion data may be contextually associated and parsed based on the license plate recognition module to obtain a target license plate recognition result. It is understandable that the target license plate recognition result may include the license plate data of the target vehicle.

[0141] It can be understood from the above description that the embodiments of the present application can perform license plate recognition on the target vehicle based on the target video data, so in some application scenarios, a license plate recognition booth may not be set up. That is, the camera equipment can be set in different places, so that multiple target video data can be obtained. The shooting angles of the target license plate and the influence of ambient light on the multiple target video data are different. In this case, the multiple target video data can be processed by but not limited to the following two methods to improve the accuracy of license plate recognition:

[0142] The first method is to identify multiple target video data separately based on the license plate recognition model to obtain multiple recognition results, and then perform fusion analysis on the multiple recognition results to obtain the final recognition result.

[0143] The second method extracts at least one frame of image from each target video data, groups multiple image frames into an image set, and uses the image set as input data for the license plate recognition model to perform license plate recognition on the target vehicle. In this method, when determining the attention weight corresponding to each image in the image set, the feature fusion module of the license plate recognition model can also adjust the attention weight based on factors such as the shooting angle and ambient light corresponding to each image.

[0144] The license plate recognition method provided in the embodiment of the present application can perform license plate recognition on the target license plate through the license plate recognition model and the target video data, which can reduce the problem in the related art that the recognition accuracy is affected by the vehicle motion state and ambient light. In addition, since the embodiment of the present application performs license plate recognition based on the target video data, there is no specific restriction on the location where the target vehicle performs license plate recognition, such as no license plate recognition booth is required. Therefore, the license plate recognition method provided in the embodiment of the present application can also enrich the scene of the target vehicle performing license plate recognition.

[0145] Reference Figure 8 In some embodiments, the present application also provides a license plate recognition model training method, which can train the license plate recognition model in any of the above embodiments. The license plate recognition model training method includes but is not limited to steps S801 to S807.

[0146] Step S801, obtaining sample video data of a sample vehicle and sample license plate data of a sample vehicle;

[0147] Step S802, extracting image frames from the sample video data to obtain a sample license plate area image set;

[0148] Step S803, in each round of iteration, inputting the sample license plate area image set into the current license plate recognition model to be trained;

[0149] Step S804, extracting features from each image in the sample license plate area image set using the current license plate recognition model to be trained to obtain a sample license plate feature map;

[0150] Step S805, performing feature fusion on the sample license plate feature map through the current license plate recognition model to be trained and the attention weight corresponding to the sample license plate feature map to obtain sample license plate fusion data;

[0151] Step S806, parsing the sample license plate fusion data through the current license plate recognition model to be trained to obtain predicted license plate data of the sample vehicle;

[0152] Step S807, iteratively adjust the parameters of the current license plate recognition model to be trained according to the predicted license plate data and the sample license plate data until the iteration stop condition is met.

[0153] In step S801 of some embodiments, the sample vehicle may refer to a vehicle that needs to be license plate recognized, the sample vehicle may be the same as or different from the target vehicle, and the sample vehicle may be in motion or at rest, which is not specifically limited in the embodiments of the present application. The sample video data may refer to data obtained by photographing the sample vehicle based on a preset camera device, and the duration of the sample video data may be adaptively set according to actual needs, which is not specifically limited in the embodiments of the present application. For example, in a sentry booth scene, the camera device may be installed at the sentry booth, and the sample video data may be 20 seconds of video data before the sample vehicle arrives at the sentry booth and stops moving. The sample license plate data may refer to the license plate number of the sample vehicle, and the sample license plate data may be obtained by manual annotation and the like. Corresponding to the case where there is no license plate, the sample license plate data may be represented as "none".

[0154] In step S802 of some embodiments, image frame extraction may refer to image extraction of sample video data according to a frame rate. Based on image frame extraction, multiple images may be obtained, and a sample license plate area image set may be obtained by aggregating the multiple images. It is understandable that the number of extracted images may be adaptively set according to actual needs, and the embodiments of the present application do not specifically limit this. For example, when a higher license plate recognition accuracy is required, a larger number of images may be extracted from the sample video data. When a faster license plate recognition speed is required, a smaller number of images may be extracted from the sample video data.

[0155] Reference Fig.9A In some embodiments, step S802 includes but is not limited to steps S901 to S904.

[0156] Step S901, extracting image frames from sample video data to obtain a license plate image set;

[0157] Step S902, performing license plate region recognition on any image in the license plate image set according to a preset license plate detection model to obtain a sample license plate region;

[0158] Step S903, performing region amplification processing on the sample license plate region to obtain an amplified license plate region;

[0159] Step S904, cropping the images in the license plate image set according to the enlarged license plate area to obtain a sample license plate area image set.

[0160] In step S901 of some embodiments, multiple frames of images obtained after image frame extraction from sample video data are collected to obtain a license plate image set. i , and extract image I iThe first k images and the last k images of the license plate are combined to obtain the license plate image set I = {I i-k ,...,I i-1 ,I i ,I i+1 ,...,I i+k}.

[0161] In step S902 of some embodiments, the license plate detection model may refer to a model with the ability to detect a license plate frame. The license plate detection model may be a model built based on the YOLO model (the YOLO model is a target detection model, and the target detection task is to find all regions of interest in the image and determine the probability of the location and category of these regions), and trained on a public license plate data set. The training method of the license plate detection model may be a supervised training method or other training methods. Any image in the license plate image set is used as input data of the license plate detection model to obtain multiple detection frames representing the license plate area. The license plate area corresponding to one of the detection frames is selected as the sample license plate area. For example, as shown in 9B, a sample license plate area b0 can be obtained.

[0162] It can be understood that in the license plate image set, the first k images and the last k images are based on image I i So the first k images and the last k images are the same as image I i The difference may be small. At this time, in order to improve the efficiency and accuracy of obtaining the sample license plate area image set, the image I i As input data for the license plate detection model.

[0163] In step S903 of some embodiments, Fig. 9B As shown, the region enlargement process may refer to enlarging the license plate region. The region enlargement process may include equal-proportional enlargement, custom enlargement, etc., which are not specifically limited in the present embodiment. The region enlargement process is performed on the sample license plate region to obtain an enlarged license plate region b1.

[0164] In step S904 of some embodiments, all images in the license plate image set are cropped according to the coordinates of the enlarged license plate area, and the cropped images are collected to obtain a sample license plate area image set Q = {Q i-k ,...,Q i-1 ,Q i ,Q i+1 ,...,Q i+k}.

[0165] The benefit of step S901 to step S904 is that a sample license plate area image set including multiple license plate information can be obtained. Since the sample license plate area image set is obtained by image cropping, noise (such as partial license plate areas of other vehicles except the sample vehicle) can be reduced, thereby improving the accuracy of subsequent license plate recognition based on the license plate area image set.

[0166] Reference Fig.10 In some embodiments, before step S803, the method provided in the embodiment of the present application may further include performing a data enhancement operation on the sample license plate area image set, specifically including but not limited to steps S1001 to S1002.

[0167] Step S1001, transforming the images in the sample license plate area image set according to preset transformation parameters;

[0168] Step S1002, performing data mapping between the sample license plate data and the transformed license plate area image set.

[0169] In step S1001 of some embodiments, the transformation parameters may refer to the parameters used when transforming the image, and the transformation may include adjusting one or more combinations of illumination, contrast, saturation, hue, Gaussian blur, motion blur, and adding noise.

[0170] In step S1002 of some embodiments, data mapping is performed between the sample license plate data and the transformed license plate area image set, that is, the sample license plate data is still used as the license plate data of the transformed license plate area image set, so that the transformed license plate area image set also has a data pair form of "image set-license plate data".

[0171] The benefit of step S1001 to step S1002 is that the data set used to train the license plate recognition model can be expanded on the basis of reducing the cost of data collection and data labeling.

[0172] In step S803 of some embodiments, the license plate recognition model can be a module constructed based on a recurrent neural network (RNN) and a CTC model (the CTC model is usually connected to the last layer of the RNN module, and the CTC module can predict data labels based on the output data of the RNN model), or can be a module constructed based on a sequence to sequence model (Seq2Seq) and an attention mechanism. The license plate recognition model can be trained for multiple rounds of iterations. In each round of iterative training, the sample license plate area image set can be input into the current license plate recognition model to be trained.

[0173] In step S804 of some embodiments, the sample license plate region image set is used as input data of the current license plate recognition model to be trained. Figure 2 As shown, the license plate recognition model may include a feature extraction module, a feature fusion module and a license plate recognition module. The feature extraction module may refer to a module constructed based on a convolutional neural network (CNN), and the feature extraction module may include two feature extraction blocks (such as CNN Block1 and CNN Block2). Each feature extraction block may include structures such as convolution, pooling, activation function, batch normalization and jump connection. The sample license plate area image set is used as the input data of the feature extraction module. After being processed by the two feature extraction blocks, multiple sample license plate feature maps can be obtained. For example, when k=2, corresponding to the sample license plate area image set Q={Q i-2 ,Q i-1 ,Q i ,Q i+1 ,Q i+2} can get L i-2 , L i-1 , L i , L i+1 and L i+2 There are five sample license plate feature images in total. It is understandable that each feature extraction block can reduce the height (H) and width (W) dimensions of the input data to 1 / 2 of the original dimensions. Therefore, after two feature extraction blocks, the height and width dimensions of the sample license plate feature image are reduced to 1 / 4 of the height and width dimensions of the image in the sample license plate area image set. It is understandable that before the sample license plate area image set is input into the feature extraction module, each image in the sample license plate area image set can also be grayscale processed.

[0174] In some implementations of step S805, the attention weight can be used to indicate the importance of the corresponding sample license plate feature map. Feature fusion processing is performed on multiple sample license plate feature maps based on the feature fusion module and the attention weight to fuse the license plate information of the multiple sample license plate feature maps to obtain sample license plate fusion data.

[0175] In step S806 of some embodiments, the sample license plate fusion data is contextually associated and parsed based on the current license plate recognition module to obtain predicted license plate data. It can be understood that the predicted license plate data is the prediction result obtained by the current license plate recognition model to be trained based on the sample license plate area image set to predict the license plate of the sample vehicle.

[0176] In step S807 of some embodiments, the recognition error of the current license plate recognition model to be trained can be determined based on the error between the predicted license plate data and the sample license plate data, and the parameters of the license plate recognition model are adjusted based on the recognition error. Steps S803 to S807 are executed cyclically until the iteration stop condition is met. Among them, the iteration stop condition may include that the number of iterations reaches a preset number of times, the recognition error is less than a preset error, etc., which is not specifically limited in the embodiments of the present application.

[0177] The specific implementation of the license plate recognition model training method is basically the same as the specific implementation example of the above-mentioned license plate recognition method, and will not be repeated here.

[0178] See also Fig.11A The embodiment of the present application also provides a license plate recognition device, which can implement the above license plate recognition method, and the device includes:

[0179] The data acquisition module 1101 is used to acquire target video data of a target vehicle;

[0180] An image frame extraction module 1102 is used to extract image frames from target video data to obtain a target license plate area image set;

[0181] The feature extraction module 1103 is used to extract features from each image in the target license plate area image set by using a pre-trained license plate recognition model to obtain a target license plate feature map;

[0182] The feature fusion module 1104 is used to perform feature fusion on the target license plate feature map through the license plate recognition model and the attention weight corresponding to the target license plate feature map to obtain target license plate fusion data;

[0183] The parsing module 1105 is used to parse the target license plate fusion data through the license plate recognition model to obtain the target license plate recognition result of the target vehicle.

[0184] In some embodiments, the target license plate area image set includes a first license plate area image, a second license plate area image, and a third license plate area image, the second license plate area image is a front frame image of the first license plate area image, the third license plate area image is a rear frame image of the first license plate area image, and the target license plate feature map includes a first feature map corresponding to the first license plate area image, a second feature map corresponding to the second license plate area image, and a third feature map corresponding to the third license plate area image. The feature fusion module 1104 is used to:

[0185] Determine, by means of a license plate recognition model, a feature license plate region corresponding to a target license plate region in the first license plate region image in the first feature map;

[0186] Determine the attention weight corresponding to the second feature map based on the feature license plate area, and calculate the first attention feature map according to the second feature map and the attention weight corresponding to the second feature map;

[0187] Determine the attention weight corresponding to the third feature map based on the feature license plate area, and calculate the second attention feature map according to the third feature map and the attention weight corresponding to the third feature map;

[0188] The first attention feature map and the second attention feature map are subjected to feature fusion to obtain the target license plate fusion data.

[0189] In some embodiments, the feature fusion module 1104 is used to:

[0190] Determine a plurality of regional feature vectors in the characteristic license plate region;

[0191] The attention weight corresponding to the second feature map is calculated based on the feature vector of each region and the feature vector in the second feature map;

[0192] Perform a weighted sum operation on the second feature map based on the attention weight corresponding to the second feature map to obtain a sub-attention feature map;

[0193] Arrange the sub-attention feature maps to obtain the first attention feature map.

[0194] In some embodiments, the feature fusion module 1104 is used to:

[0195] Perform feature fusion on the first attention feature map and the second attention feature map to obtain a total attention feature map;

[0196] Superimpose the total attention feature map with the image corresponding to the characteristic license plate area to obtain the target feature map;

[0197] Performing a size transformation operation on the target feature map to obtain a transformed feature map;

[0198] Feature extraction is performed on the transformed feature map to obtain the target license plate fusion data.

[0199] In some embodiments, the feature fusion module 1104 is used to:

[0200] Determine target coordinate data of the target license plate area and the image size of the first feature map through the license plate recognition model;

[0201] Calculate feature coordinate data based on a preset model scaling factor, image size, and target coordinate data;

[0202] A characteristic license plate area is determined in the first characteristic map based on the characteristic coordinate data.

[0203] The specific implementation of the license plate recognition device is basically the same as the specific implementation of the license plate recognition method described above, and will not be repeated here.

[0204] See also Fig. 11B The embodiment of the present application also provides a license plate recognition model training device, which can implement the above license plate recognition model training method, and the device includes:

[0205] The training data acquisition module 1111 is used to acquire sample video data of a sample vehicle and sample license plate data of a sample vehicle;

[0206] The training image frame extraction module 1112 is used to extract image frames from sample video data to obtain a sample license plate area image set;

[0207] The data input module 1113 is used to input the sample license plate area image set into the current license plate recognition model to be trained in each round of iteration;

[0208] The training feature extraction module 1114 is used to extract features from each image in the sample license plate area image set by using the current license plate recognition model to be trained to obtain a sample license plate feature map;

[0209] The training feature fusion module 1115 is used to perform feature fusion on the sample license plate feature map through the current license plate recognition model to be trained and the attention weight corresponding to the sample license plate feature map to obtain sample license plate fusion data;

[0210] The training and parsing module 1116 is used to parse the sample license plate fusion data through the current license plate recognition model to be trained to obtain the predicted license plate data of the sample vehicle;

[0211] The parameter adjustment module 1117 is used to iteratively adjust the parameters of the current license plate recognition model to be trained according to the predicted license plate data and the sample license plate data until the iteration stop condition is met.

[0212] In some embodiments, the training image frame extraction module 1112 is used to:

[0213] Extract image frames from sample video data to obtain a license plate image set;

[0214] According to the preset license plate detection model, the license plate area of ​​any image in the license plate image set is recognized to obtain a sample license plate area;

[0215] Performing region amplification processing on the sample license plate region to obtain an amplified license plate region;

[0216] The images in the license plate image set are cropped according to the amplified license plate area to obtain a sample license plate area image set.

[0217] The specific implementation of the license plate recognition model training device is basically the same as the specific implementation of the above-mentioned license plate recognition model training method, and will not be repeated here.

[0218] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above license plate recognition method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0219] See also Fig.12 , Fig.12 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0220] The processor 1201 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0221] The memory 1202 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1202 can store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1202, and the processor 1201 calls and executes the license plate recognition method or license plate recognition model training method of the embodiments of this application;

[0222] Input / output interface 1203, used to implement information input and output;

[0223] The communication interface 1204 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0224] A bus 1205 that transmits information between various components of the device (e.g., the processor 1201, the memory 1202, the input / output interface 1203, and the communication interface 1204);

[0225] The processor 1201 , the memory 1202 , the input / output interface 1203 and the communication interface 1204 are connected to each other in communication within the device via the bus 1205 .

[0226] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned license plate recognition method or license plate recognition model training method.

[0227] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0228] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0229] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0230] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0231] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0232] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0233] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0234] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0235] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0236] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0237] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0238] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A license plate recognition method, It is characterized in that The method comprises: Acquire target video data of a target vehicle; Extracting image frames from the target video data to obtain a target license plate area image set; Extracting features from each image in the target license plate area image set using a pre-trained license plate recognition model to obtain a target license plate feature map; Performing feature fusion on the target license plate feature map through the license plate recognition model and the attention weight corresponding to the target license plate feature map to obtain target license plate fusion data; The target license plate fusion data is parsed by the license plate recognition model to obtain a target license plate recognition result of the target vehicle.

2. The method according to claim 1, It is characterized in that The target license plate area image set includes a first license plate area image, a second license plate area image and a third license plate area image, the second license plate area image is a front frame image of the first license plate area image, the third license plate area image is a rear frame image of the first license plate area image, and the target license plate feature map includes a first feature map corresponding to the first license plate area image, a second feature map corresponding to the second license plate area image and a third feature map corresponding to the third license plate area image; The step of performing feature fusion on the target license plate feature map by using the license plate recognition model and the attention weight corresponding to the target license plate feature map to obtain target license plate fusion data includes: Determine, by means of the license plate recognition model, a characteristic license plate region corresponding to a target license plate region in the first license plate region image in the first characteristic image; Determine the attention weight corresponding to the second feature map based on the characteristic license plate area, and calculate the first attention feature map according to the second feature map and the attention weight corresponding to the second feature map; Determine the attention weight corresponding to the third feature map based on the feature license plate area, and calculate the second attention feature map according to the third feature map and the attention weight corresponding to the third feature map; Feature fusion is performed on the first attention feature map and the second attention feature map to obtain the target license plate fusion data.

3. The method according to claim 2, It is characterized in that The determining the attention weight corresponding to the second feature map based on the feature license plate area, and calculating the first attention feature map according to the second feature map and the attention weight corresponding to the second feature map, comprises: Determining a plurality of regional feature vectors in the characteristic license plate region; Obtaining an attention weight corresponding to the second feature map by calculating based on each of the region feature vectors and the feature vector in the second feature map; Performing a weighted sum operation on the second feature map based on the attention weight corresponding to the second feature map and obtaining a sub-attention feature map; The sub-attention feature maps are arranged to obtain the first attention feature map.

4. The method according to claim 2, It is characterized in that The step of fusing the first attention feature map and the second attention feature map to obtain the target license plate fusion data includes: Performing feature fusion on the first attention feature map and the second attention feature map to obtain a total attention feature map; Superimposing the total attention feature map with the image corresponding to the characteristic license plate area to obtain a target feature map; Performing a size transformation operation on the target feature map to obtain a transformed feature map; Feature extraction is performed on the transformed feature map to obtain the target license plate fusion data.

5. The method according to claim 2, It is characterized in that The determining, by the license plate recognition model, a characteristic license plate region corresponding to a target license plate region in the first license plate region image in the first characteristic map includes: Determine the target coordinate data of the target license plate area and the image size of the first feature map by using the license plate recognition model; Calculating feature coordinate data based on a preset model scaling factor, the image size and the target coordinate data; The characteristic license plate area is determined in the first characteristic map based on the characteristic coordinate data.

6. A license plate recognition model training method, It is characterized in that The method comprises: Acquire sample video data of a sample vehicle and sample license plate data of the sample vehicle; Extracting image frames from the sample video data to obtain a sample license plate area image set; In each round of iteration, the sample license plate area image set is input into the current license plate recognition model to be trained; Extracting features from each image in the sample license plate area image set using the current license plate recognition model to be trained to obtain a sample license plate feature map; Performing feature fusion on the sample license plate feature map through the current license plate recognition model to be trained and the attention weight corresponding to the sample license plate feature map to obtain sample license plate fusion data; Parsing the sample license plate fusion data through the current license plate recognition model to be trained to obtain predicted license plate data of the sample vehicle; According to the predicted license plate data and the sample license plate data, the parameters of the current license plate recognition model to be trained are iteratively adjusted until an iteration stop condition is met.

7. The method according to claim 6, It is characterized in that The step of extracting image frames from the sample video data to obtain a sample license plate area image set includes: Extracting image frames from the sample video data to obtain a license plate image set; Performing license plate area recognition on any image in the license plate image set according to a preset license plate detection model to obtain a sample license plate area; Performing region amplification processing on the sample license plate region to obtain an amplified license plate region; The images in the license plate image set are cropped according to the amplified license plate area to obtain the sample license plate area image set.

8. A license plate recognition device, It is characterized in that The device comprises: A data acquisition module, used to acquire target video data of a target vehicle; An image frame extraction module is used to extract image frames from the target video data to obtain a target license plate area image set; A feature extraction module is used to extract features from each image in the target license plate area image set using a pre-trained license plate recognition model to obtain a target license plate feature map; A feature fusion module, used to perform feature fusion on the target license plate feature map through the license plate recognition model and the attention weight corresponding to the target license plate feature map to obtain target license plate fusion data; The parsing module is used to parse the target license plate fusion data through the license plate recognition model to obtain the target license plate recognition result of the target vehicle.

9. An electronic device, It is characterized in that The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method described in any one of claims 1 to 5 or the method described in any one of claims 6 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented, or the method according to any one of claims 6 to 7 is implemented.

Images