A real-time license plate detection and recognition system in unrestricted scenarios

Abstract

The present invention discloses a real-time license plate detection and recognition system in an unrestricted scenario, which is applied to the field of intelligent traffic image detection technology, and includes: a YOLOv5 license plate detection network and a PDLPR license plate recognition network. The license plate detection module uses the YOLOv5 algorithm with relatively high current target detection efficiency, and the license plate recognition module uses the PDLPR license plate recognition algorithm proposed by the present invention. The method includes a feature extractor capable of extracting global feature information. Compared with traditional feature extraction methods, this method can extract rich semantic information. At the same time, it fully utilizes the advantages of multi-head attention. The training data does not need to mark the position of individual characters in the license plate one by one, and the license plate image can be accurately recognized without auxiliary correction means. It shows good performance in natural scenes such as tilted license plates, poor lighting conditions, and blurred characters.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent traffic image detection, and more particularly to a real-time license plate detection and recognition system in non-restricted scenarios. Background Art

[0002] License plates are a crucial identifier for motor vehicles, and their accurate recognition has high application value and has become a recent research hotspot. License plate detection and recognition (LPDR) technology has been widely used in vehicle authentication, intelligent toll collection, vehicle density statistics, access control, intelligent driving, traffic control, and traffic scene understanding.

[0003] License plate detection and recognition methods can be broadly categorized into traditional methods (non-deep learning methods) and deep learning methods. Traditional methods often rely on manually selected features to locate the license plate, then use morphological methods to separate the characters within the license plate and identify them individually. However, this method suffers from low accuracy, especially in complex scenarios. Deep learning methods typically involve two steps, namely, two networks: a detection network that locates the license plate in an image, and a recognition network that identifies the character sequence within the license plate image.

[0004] Despite significant progress in LPDR research, most existing algorithms only perform well in relatively restricted scenarios with relatively fixed lighting, shooting distance, and shooting angle. License plate recognition in natural scenes, in particular, is susceptible to various external constraints. These complexities include: 1) Shooting distance: Different shooting distances result in different widths, heights, and proportions of the license plate in the image; 2) Tilt: Due to the camera's shooting angle, each license plate image has varying degrees of tilt; 3) Light intensity: High ambient light levels can overexpose the license plate, while low ambient light levels can cause a dark image; 4) Weather: Poor weather conditions during license plate image acquisition, such as rain, snow, or fog, increase the noise in the license plate image; 5) Image blur: Lens shake during capture or the vehicle's high speed during image acquisition can cause blurry characters; and 6) Character incompleteness: Partial incomplete characters can occur due to stains or occlusion on the license plate. These limitations make accurate license plate recognition difficult.

[0005] Therefore, it is an urgent problem for those skilled in the art to propose a real-time license plate detection and recognition system in an unrestricted scenario to solve the difficulties existing in the prior art. Summary of the Invention

[0006] In view of this, the present invention provides a real-time license plate detection and recognition system in non-restricted scenarios. License plate images can be accurately recognized without the need for auxiliary correction means, and exhibits good performance in natural scenarios such as tilted license plates, poor lighting conditions, and blurred characters.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A real-time license plate detection and recognition system in unrestricted scenarios, mainly including: a YOLOv5 license plate detection network and a PDLPR license plate recognition network;

[0009] The YOLOv5-based license plate detection network accepts the input of the whole vehicle image, locates the license plate position in the image, and then outputs the license plate image containing only the license plate information; the PDLPR license plate recognition network takes the license plate image as input, obtains the license plate character sequence after feature extraction, encoder encoding and decoder decoding operations.

[0010] Optionally, the PDLPR license plate recognition network includes 3 modules:

[0011] The improved global feature extractor IGFE, encoder Encoder, and parallel decoder ParallelDecoder are connected in sequence.

[0012] Optionally, the specific content of the PDLPR license plate recognition network is as follows:

[0013] The input license plate image is first preprocessed, and then the preprocessed image is input into the improved global feature extractor IGFE for feature extraction. The preprocessed image is converted into a 512*6*18 feature vector; in the Encoder part, the position encoder first position-encodes the feature map obtained from the IGFE part, adds the obtained precise position code to the image feature vector, and then uses the multi-head attention mechanism to encode the image feature map embedded with the position code; in the parallel decoder part, during training, the real license plate sequence is shifted right by one position, and then the corresponding position code is added, and then the multi-head attention mechanism is used to decode the feature vector input by the encoder to predict the character at the next position of each character.

[0014] Optionally, the improved global feature extractor IGFE includes 1 FocusStructure module, 2 ConvDownSampling modules and 4 RESBLOCK modules.

[0015] Optionally, the FocusStructure module is used to:

[0016] In the input image, a value is taken for every pixel, and the image is equally divided into four feature maps. Then these four feature maps are connected along the channel direction. In this way, an image with only 3 channels is turned into a 12-channel feature map with half the width and height. Finally, the obtained feature map is subjected to a convolution operation to complete the double downsampling operation.

[0017] Optionally, the encoder includes 3 units of residual connection, each unit contains 4 submodules: CNN BLOCK1, Multi-Head Attention, CNN BLOCK2 and Add&Norm.

[0018] Optionally, the parallel decoder consists of 3 identical decoding units, each of which contains 4 sub-modules: Multi-Head Attention, Masked Multi-Head Attention, Feed-Forward Network, and Add&Norm.

[0019] It can be seen from the above technical solutions that, compared with the prior art, the present invention provides a real-time license plate detection and recognition system in non-restricted scenarios, which has the following beneficial effects:

[0020] (1) The present invention proposes a YOLOv5-PDLPR real-time license plate detection and recognition algorithm. The YOLOv5 algorithm with relatively high current target detection efficiency is used in the license plate detection module, and the PDLPR license plate recognition algorithm proposed in the present invention is used in the license plate recognition module. The method includes a feature extractor capable of extracting global feature information. Compared with traditional feature extraction methods, this method can extract rich semantic information.

[0021] (2) At the same time, the advantages of multi-head attention are fully utilized. The training data does not need to mark the position of individual characters in the license plate one by one, and the license plate image can be accurately recognized without auxiliary correction means. It shows good performance in natural scenes such as tilted license plates, poor lighting conditions, and blurred characters. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0023] Figure 1 This is the overall framework diagram of the license plate detection and recognition network provided by the present invention;

[0024] Figure 2 The overall framework diagram of the license plate recognition network provided by the present invention;

[0025] Figure 3 The IGFE network structure diagram provided by the present invention;

[0026] Figure 4 Schematic diagram of the FocusStructure slicing process provided by the present invention;

[0027] Figure 5 LeakyRelu and Relu provided by the present invention;

[0028] Figure 6 This is the Encoder network structure diagram provided by the present invention;

[0029] Figure 7 Decoder network structure diagram provided by the present invention

[0030] Figure 8 This is a schematic diagram of some license plate detection and recognition results of the present invention;

[0031] Figure 9 This is the license plate recognition heat map of the present invention. DETAILED DESCRIPTION

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0033] Reference Figure 1 As shown, the present invention discloses a real-time license plate detection and recognition system in an unrestricted scenario, which mainly includes: a license plate detection network based on YOLOv5 and a PDLPR license plate recognition network;

[0034] The YOLOv5-based license plate detection network accepts the input of the whole vehicle image, locates the license plate position in the image, and then outputs the license plate image containing only the license plate information; the PDLPR license plate recognition network takes the license plate image as input, obtains the license plate character sequence after feature extraction, encoder encoding and parallel decoder decoding operations.

[0035] Furthermore, the PDLPR license plate recognition network consists of three modules:

[0036] Improved global feature extractor (IGFE), encoder (Encoder), parallel decoder (ParallelDecoder).

[0037] Furthermore, the specific steps of the PDLPR license plate recognition network are as follows:

[0038] First, the input license plate image is preprocessed and the image size is adjusted to 48×144. Then, the preprocessed image is input into IGFE for feature extraction, and the image is converted into a 512*6*18 feature vector. In the Encoder part, the position encoder first performs position encoding on the feature map obtained from the IGFE part, adds the obtained precise position encoding to the image feature vector, and then uses the multi-head attention mechanism to encode the image feature map embedded with the position encoding. In the parallel decoder part, during training, the real license plate sequence is shifted right by one position, and then the corresponding position encoding is added. Then, the multi-head attention mechanism is used to decode the feature vector input by the encoder to predict the character at the next position of each character.

[0039] Specifically, the parallel decoder is tested by first inputting a special character <start>As a sign to start decoding, and then predict the character at the next position in sequence.

[0040] Furthermore, the improved global feature extractor includes 1 FocusStructure module, 2 ConvDownSampling modules and 4 RESBLOCK modules; the Focus structure is added at the beginning of the network.

[0041] Furthermore, the specific steps in the FocusStructure module are as follows:

[0042] In the input image, a value is taken for every pixel, and the image is equally divided into four feature maps. These four feature maps are then connected along the channel direction, so that an image with only 3 channels becomes a 12-channel feature map with half the width and height of the original. Finally, the obtained feature map is subjected to a convolution operation to complete the double downsampling operation.

[0043] Furthermore, the encoder includes three units of residual connection, each of which contains four submodules: CNNBLOCK1, Multi-Head Attention, CNN BLOCK2 and Add&Norm.

[0044] Furthermore, the parallel decoder consists of 3 identical decoding units, each of which contains 4 sub-modules: Multi-Head Attention, Masked Multi-Head Attention, Feed-Forward Network and Add&Norm.

[0045] Reference Figure 2 As shown in the figure, the model is mainly divided into three modules: Improved Global Feature Extractor (IGFE), Encoder, and Parallel Decoder. The input license plate image is first preprocessed and resized to 48×144. The preprocessed image is then input into the IGFE for feature extraction, and the image is converted into a 512*6*18 feature vector. In the Encoder, the positional encoder first performs positional encoding on the feature map obtained from the IGFE, adds the obtained precise positional encoding to the image feature vector, and then uses a multi-head attention mechanism to encode the image feature map embedded with the positional encoding. In the Parallel Decoder, during training, the real license plate sequence is shifted right by one position, then the corresponding positional encoding is added. The multi-head attention mechanism is then used to decode the encoder input feature vector and predict the next character at each position. When testing, the Parallel Decoder initially inputs a special character. <start>As a sign to start decoding, and then predict the character at the next position in sequence.

[0046] The present invention designs an improved global feature extractor (IGFE), which adds a Focus structure at the beginning of the network to achieve the function of feature map downsampling while ensuring that feature information is not lost; in other parts where downsampling operations are required, convolution operations with a stride of 2 are used to replace pooling operations to ensure the integrity of the features extracted by the network. The complete feature information can make subsequent license plate character recognition more accurate and improve the accuracy of license plate recognition. The structure of the IGFE module is as follows: Figure 3 As shown in the figure, it consists of 1 FocusStructure module, 2 ConvDownSampling modules and 4 RESBLOCK modules.

[0047] A) FocusStructure module

[0048] FocusStructure module structure is as follows Figure 3 As shown in the lower middle part, it is used to slice the image. The operation process is as follows Figure 4 As shown in the figure, every other pixel in the input image is assigned a value, dividing the image into four equal feature maps. These four feature maps are then concatenated along the channel direction, transforming an image with only three channels into a 12-channel feature map with half the width and height. Finally, the resulting feature map is subjected to a convolution operation, completing a two-fold downsampling operation. Compared to other downsampling methods, FocusStructure eliminates feature information loss, ensuring that the semantic information subsequently extracted is more complete and richer.

[0049] B)RESBLOCK module

[0050] Each RESBLOCK module structure is as follows Figure 3 As shown in the middle, it consists of two CNN encoder (CNN BLOCK) modules with residual connections. The residual connection can avoid the gradient disappearance and gradient explosion caused by the network during the forward inference process.

[0051] In the convolutional layer of the CNN encoder module, we use conv2d and set stride = 1 and kernelSize = 3 for feature extraction. The extracted features are then passed through the BatchNormalization layer and the activation function layer in sequence to extract the visual features of the image.

[0052] Activation function usage Figure 5 The leakyRelu shown in (a) instead of Figure 5 In the Relu function shown in (b), when the input is negative, the output is always 0, and its derivative is also 0. This can easily cause "dead neurons" (dead neurons), that is, neurons no longer learn and their parameters no longer change. LeakyRelu gives it a smaller slope value for negative inputs to avoid neuron death.

[0053] C) ConvDownSampling module

[0054] The structure of the downsampling module of the present invention is the same as that of the CNN BLOCK, but the stride is set to 2 in conv2d for downsampling, and the convolution operation is used instead of the pooling operation for downsampling. During the downsampling process, more feature information can be retained, thereby improving the accuracy of license plate recognition.

[0055] Reference Figure 6 As shown in the figure, the encoder structure of the present invention includes three units of residual connection, each unit contains four submodules: CNN BLOCK1, Multi-Head Attention, CNN BLOCK2 and Add&Norm. Here, CNNBLOCK1 and CNN BLOCK2 have the same structure but different parameters, which are introduced below.

[0056] First, use Figure 6 The CNN BLOCK1 at the bottom increases the dimension of the feature vector before calculating the multi-head attention. This allows for the extraction of richer feature information during the multi-head attention calculation process. Here, stride = 1, kernelSize = 1, padding = 1 are set, and the output dimension is 1024.

[0057] Then, multi-head attention is used to encode the input sequence. Here, the attention on each subspace is calculated in parallel, and the calculation results of different spaces are connected in the column direction. Then, a linear transformation is performed to obtain the final encoding result. This can focus on the attention between features in multiple aspects and multiple spaces. The calculation of multi-head attention MHA(Q, K, V) is shown in formula (1):

[0058]

[0059] In formula (1), Q, K, V∈R n×d , n=width*height=108, d=1024, Represents the attention calculation result of the i-th subspace; are the projection matrices projected onto the i-th subspace of Q, K, and V; W O is the matrix for calculating the linear transformation of the head; width and height are the width and height of the feature vector output by CNN BLOCK1 respectively; the value of d is equal to the dimension of the feature vector output by CNN BLOCK1; h is the number of heads in the multi-head attention, which means paying attention to features from h spaces (after experimental comparison, the license plate recognition accuracy is the highest when h = 8, experimental results are shown in Ablation). The calculation of Q, K, and V in formula (1) is shown in formula (2):

[0060]

[0061] In formula (2), X∈R n×d , W Q 、W K 、W V ∈R d×d , x m ∈R 1×d X is the feature vector output by CNNBLOCK1, W Q 、W K 、W V There are three different trainable weights, which are obtained by random initialization at the beginning of training, and then updated by gradient descent during training, and finally a suitable weight is obtained to fit the true value.

[0062] Each of the formula (1) The calculation method is shown in formula (3):

[0063]

[0064] The softmax function is used in formula (3) to obtain the probability distribution of feature positions. To avoid and The dot product of is too large, causing the softmax value to approach 0.

[0065] After calculating the multi-head attention, use Figure 6 CNN BLOCK2 in the middle restores the feature vector dimension to ensure that the input and output dimensions of the encoder unit are consistent. Here, stride = 1, kernelSize = 1, padding = 1, and the output dimension is 512.

[0066] The Add&Norm module connects the feature vectors before and after the calculation of multi-head attention through a residual network, and then performs layer normalization. Layer normalization can prevent overfitting of the model to a certain extent and speed up the convergence of the model.

[0067] The present invention utilizes a parallel decoder in the decoder part. The multi-head attention mechanism used in the parallel decoder and encoder network calculates the correlation between the feature vectors input to the network through multiple attention heads, so that the feature vectors and the license plate characters are matched one by one, achieving the effect of locating the character positions in the license plate image. Therefore, the training data set provided only needs to provide the license plate sequence, and does not need to mark the character positions one by one. The parallel decoder network structure is as follows: Figure 7 As shown in the figure, it consists of 3 identical decoding units, each of which contains 4 sub-modules: Multi-Head Attention, MaskedMulti-Head Attention, Feed-Forward Network and Add&Norm.

[0068] Figure 7 The role of the Masked Multi-Head Attention at the bottom is to prevent the model from paying attention to subsequent sequence information and ensure the parallelism of training. The specific method is to use a matrix whose upper triangular elements are all -∞ to add the input eigenvector, and then perform a softmax operation on the added matrix, so that the original eigenvector matrix becomes a lower triangular eigenvector matrix. Multi-HeadAttention is then calculated for the masked feature matrix. The masking operation can limit the attention area of ​​each time step and ensure that the prediction of each position only depends on the known output before that position. Due to the masked multi-head attention mechanism, the entire training process only requires one forward calculation. When the RNN model is calculating, only when the calculation is completed at time t can the calculation continue at time t+1. Therefore, MaskedMulti-HeadAttention makes the training speed of the model proposed by the present invention much faster than the model using RNN and LSTM.

[0069] Then the output of MaskedMulti-HeadAttention is passed through an Add&Norm module for normalization to prevent overfitting of the model and speed up the convergence of the model.

[0070] Before decoding, the encoder's output feature vector is resized to 512*18 using CNNBLOCK3 and CNNBLOCK4. This reduces the size of the feature vector and, in turn, the decoder's computational effort. The output of the Add&Norm module is then used as Q, and the encoder's output is used as K and V, which are then fed into Multi-HeadAttention. The computation process for Multi-HeadAttention is the same as in 3.2. Here, K and V contain the features of the license plate image, and Q contains the semantic information of the license plate label. Multi-HeadAttention is used to calculate the correlation between each image feature and the label text features. A higher correlation indicates a greater probability that the corresponding location in the image represents a particular character. In this part, CNNBLOCK3 sets the convolutional layer parameters to stride = 3, kernelSize = (2, 1), and padding = 1, resulting in an output dimension of 512. CNNBLOCK4 sets the convolutional layer parameters to stride = 3, kernelSize = 1, and padding = (0, 1), resulting in an output dimension of 512.

[0071] Figure 7 The calculation process for the Multi-Head Attention in the middle section is the same as in 3.2. The output of the Add & Norm module is used as Q, and the output of the Encoder is used as K and V, which are input into the Multi-Head Attention. Here, K and V contain the feature information of the license plate image, and Q contains the semantic information of the license plate label. Multi-Head Attention is used to calculate the correlation between each image feature and the label text feature. A higher correlation indicates a greater probability that the corresponding position in the image contains a certain character.

[0072] The output features of Multi-Head Attention are passed through the Add&Norm module again and input into Figure 7 The upper Feed-Forward Network module consists of two linear transformations. The first linear transformation is followed by the ReLU activation function, and then input to the second linear transformation. The definition of the Feed-Forward Network is shown in formula (4): FFN(x) = max(0,xW1+b1)W2+b2(4)

[0073] Where W1∈R d×d and W2∈R d×d is the weight, b1∈R d and b2∈R d To facilitate the residual connections between layers, all sub-layers in the model have a fixed output dimension of d = 512.

[0074] Finally, the output of the Feed-Forward Network module is passed through the Add&Norm module again to accelerate the convergence of the model. The experimental results of the license plate detection algorithm used in the present invention and other detection algorithms on the CCPD dataset are shown in Table 1.

[0075] Table 1

[0076]

[0077] As can be seen from Table 1, the average accuracy of the YOLOv5 algorithm on the entire CCPD test set is higher than that of all the comparison algorithms. It also improves by 6.1%, 8.9%, 3.5%, 4.2%, and 17.6% on CCPD-DB, CCPD-FN, CCPD-Rotate, CCPD-Tilt, and CCPD-Weather, respectively, compared with the second-highest accuracy algorithm. The detection speed reaches 218.3 FPS, which is 155.3% higher than the fastest RPnet among the comparison algorithms. It can be seen that YOLOv5 has the highest detection efficiency.

[0078] On the CCPD dataset, the comparison results of the framework Yolov5-PDLPR proposed in this paper and other detection and recognition algorithms are shown in Table 2.

[0079] Table 2

[0080]

[0081]

[0082] As shown in Table 2, the average accuracy of the proposed algorithm on the entire CCPD test set is 99.4%, higher than all compared algorithms. The test accuracy on the CCPD test set subsets CCPD-base, CCPD-DB, CCPD-FN, CCPD-Weather, and CCPD-Challenge is 99.9%, 99.5%, 99.5%, 99.4%, and 94.1%, respectively, all exceeding all compared algorithms. The recognition results obtained using the proposed method on the CCPD test set subsets CCPD-Rotate and CCPD-Tilt are 0.1% and 0.3% lower than those obtained by Fan's method trained with synthetic data, because their use of synthetic data allows their model to learn more features. However, in the recognition results of Fan's method that were not trained with synthetic data, the proposed algorithm outperformed theirs. In terms of speed, our proposed algorithm is the fastest, achieving 159.8 FPS, an 87.1% improvement over the next best algorithm. This is because the PDLPR model uses parallel inference in the decoder to improve efficiency and eliminates additional corrections after license plate detection, saving recognition time. Experimental results on this dataset demonstrate that our license plate recognition model is robust and efficient in complex scenarios, making it a real-time recognition framework that meets the requirements of road monitoring.

[0083] Comparison of some license plate detection and recognition results Figure 8 As shown, the method of Zhang et al. does not use the license plate detection network to locate the license plate position, and directly inputs the real license plate image for recognition. The method of Xu et al. and the method proposed in the present invention both use the results of the license plate positioning network for recognition. Figure 6 The first row GT represents the real sequence of license plates, and the Pred below each license plate image represents the predicted license plate sequence. The red font in Pred indicates that the character is not correctly recognized, and the black font indicates that the character is correctly recognized. Figure 8 It can be seen that some characters are misidentified by the methods of Zhang et al. and Xu et al. under different lighting conditions, license plate tilt, and license plate blur, while our method can accurately locate and correctly identify them, which shows that our proposed algorithm performs better in complex scenarios.

[0084] Overlaying the heatmap of some characters with the original image allows us to observe the locations of the characters that the network focuses on. Figure 9 As shown, there are a total of 6 columns, and each column shows the heat map of license plate recognition under different inclination conditions. The first row of each column represents the original images of different license plates, and starting from the second row, the attention character effect diagrams of each character are shown in sequence. From top to bottom in each column, if a certain character in the license plate has a darker color, it indicates that the network pays more attention to the character features at that position, and then the network extracts these features for recognition. For example, in the license plate "皖A0E912", the character "皖" in the second row has a darker color, indicating that the network is more concerned about the features at the position of "皖". The character "A" in the third row has a darker color, indicating that the network pays more attention to the features at the position of the character "A". Similarly, the network can locate the features at all character positions on the license plate, so as to accurately recognize the characters in the license plate.

[0085] In this specification, the various embodiments are described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description in the method section.

[0086] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined in the present invention can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown in the present invention, but will conform to the widest scope consistent with the principles and novel features disclosed in the present invention.< / start> < / start>

Claims

1. A real-time license plate detection and recognition system in an unrestricted scenario, characterized by: Mainly includes: license plate detection network based on YOLOv5 and PDLPR license plate recognition network; The YOLOv5-based license plate detection network accepts the input of a full vehicle image, locates the license plate position in the image, and then outputs a license plate image containing only the license plate information; the PDLPR license plate recognition network takes the license plate image as input, extracts features from the license plate image, encodes it with an encoder, and decodes it with a parallel decoder to obtain a license plate character sequence; The PDLPR license plate recognition network consists of 3 modules: The improved global feature extractor IGFE, encoder Encoder, and parallel decoder ParallelDecoder are connected in sequence; The improved global feature extractor IGFE consists of 1 FocusStructure module, 2 ConvDownSampling modules and 4 RESBLOCK modules; The FocusStructure module is used to: In the input image, a value is taken for every pixel, and the image is equally divided into four feature maps. Then these four feature maps are connected along the channel direction. In this way, an image with only 3 channels is turned into a 12-channel feature map with half the width and height. Finally, the obtained feature map is subjected to a convolution operation to complete the double downsampling operation.

2. The real-time license plate detection and recognition system in non-restricted scenarios according to claim 1, characterized in that: The specific content of the PDLPR license plate recognition network is as follows: The input license plate image is first preprocessed, and then the preprocessed image is input into the improved global feature extractor IGFE for feature extraction. The preprocessed image is converted into a 512*6*18 feature vector; in the Encoder part, the position encoder first position-encodes the feature map obtained from the IGFE part, adds the obtained precise position code to the image feature vector, and then uses the multi-head attention mechanism to encode the image feature map embedded with the position code; in the parallel decoder part, during training, the real license plate sequence is shifted right by one position, and then the corresponding position code is added, and then the multi-head attention mechanism is used to decode the feature vector input by the encoder to predict the character at the next position of each character.

3. The real-time license plate detection and recognition system in non-restricted scenarios according to claim 1, characterized in that: The encoder consists of 3 units with residual connections, each of which contains 4 submodules: CNN BLOCK1, Multi-HeadAttention, CNN BLOCK2 and Add&Norm.

4. The real-time license plate detection and recognition system in non-restricted scenarios according to claim 1, characterized in that: The parallel decoder consists of 3 identical decoding units, each of which contains 4 sub-modules: Multi-HeadAttention, Masked Multi-Head Attention, Feed-Forward Network and Add&Norm.

Images