A license plate recognition method based on positional attention

Abstract

This invention provides a license plate recognition method based on positional attention, belonging to the field of license plate recognition technology. The method includes: inputting a license plate image into a convolutional neural network to extract features T; using adaptive average pooling to transform features T into a first feature sequence T1 of a specific size; reducing the dimensionality of T1 to obtain a second feature sequence z; and calculating and generating an attention map α for each character position. ij , place attention map α ij The final image feature c is obtained by performing a high-dimensional matrix multiplication with the second feature sequence z. i , will c i The license plate is then classified into multiple categories using a neural network to obtain the recognition result. This method can efficiently recognize license plate characters.

Description

Technical Field

[0001] This invention belongs to the field of license plate recognition technology, specifically relating to a license plate recognition method based on positional attention. Background Technology

[0002] License plate detection and recognition systems are widely used in various life scenarios and play a key role in today's intelligent transportation scenarios.

[0003] Existing license plate recognition methods utilize convolutional neural networks (CNNs) to extract and identify license plate image features. CNN-based methods primarily involve classifying and labeling training data, then using a CNN to extract features from the license plate. These features are then used to train the neural network, which is subsequently used to predict the license plate information.

[0004] However, this method uses all license plate features globally when predicting results, which increases the training difficulty of the network, affects the convergence speed, and to some extent impacts the recognition accuracy. Currently, there are methods based on convolutional neural networks that perform character-level annotation of license plates. While this method can converge quickly and improve recognition accuracy, the initial annotation cost is enormous. Summary of the Invention

[0005] To overcome the shortcomings of the existing technology, the present invention provides a license plate recognition method based on positional attention.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A license plate recognition method based on position attention includes:

[0008] The license plate image is input into a convolutional neural network to extract features T. Adaptive average pooling is used to transform feature T into a first feature sequence T1 of a specific size. After dimensionality reduction of T1, a second feature sequence z is obtained.

[0009] The second feature sequence z is processed by three convolutional kernels and three ReLU activation functions to obtain the third feature sequence K, where the feature size of K is (1, M, 512).

[0010] An attention map α is generated for the position of each character in the license plate image. ij Attention map α ij The expression is:

[0011]

[0012] in,

[0013] e ij =q iT k j +b i j = 1...M,

[0014] Where q i For the training vector, b i The training offset is L, where L is the maximum number of characters in the license plate, including a terminator; M is the column number of the third feature sequence K; k j It is the j-th feature vector in the third feature sequence K;

[0015] The attention map α ij The final image feature c is obtained by performing a high-dimensional matrix multiplication with the second feature sequence z. i , will c i The license plate recognition result is obtained by performing multi-class classification through a neural network.

[0016] Furthermore, the feature size of the second feature sequence z is (14, 512).

[0017] Furthermore, the maximum number of characters in the license plate is 9.

[0018] Furthermore, the image feature c i The expression is:

[0019]

[0020] Where zj is the j-th feature vector in the second feature sequence z.

[0021] Furthermore, the expression for the multi-category classification operation is:

[0022]

[0023] Where ρ ij =P(y i =j∨χ), which is the probability that the i-th character on the license plate is the j-th symbol in the character set; These are the trainable parameters of softmax.

[0024] Furthermore, the convolutional neural network uses ResNet18.

[0025] Furthermore, it also includes: normalizing the license plate image, inputting the normalized image into a convolutional neural network to extract features to obtain feature T, wherein the normalization process includes: standardizing the image channel by channel using a normalization method, with the mean and standard deviation of each channel set to 0.5.

[0026] The license plate recognition method based on positional attention provided by this invention has the following beneficial effects:

[0027] This invention proposes a license plate recognition method based on positional attention, which uses a high-dimensional matrix multiplication of the attention map and the second feature sequence z to obtain the final image feature c. i , will c i The license plate recognition result is obtained by performing multi-class classification through a neural network. This method can effectively detect the position of each character and recognize character regions without prior annotation. It solves the problem of huge upfront annotation costs in existing technologies that use convolutional neural networks to annotate license plates at the character level. Attached Figure Description

[0028] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a schematic diagram of a license plate recognition method based on positional attention according to an embodiment of the present invention;

[0030] like Figure 2 The second feature sequence z1, z2, ..., z in this embodiment of the invention M Coverage area diagram;

[0031] like Figure 3 This is an attention diagram of characters at different positions on a license plate in an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0033] Example:

[0034] This invention provides a license plate recognition method based on positional attention, specifically as follows: Figure 1 As shown, the process includes: inputting a license plate image into a convolutional neural network to extract features T; using adaptive average pooling to transform feature T into a first feature sequence T1 of a specific size; reducing the dimensionality of T1 to obtain a second feature sequence z; processing the second feature sequence z through three convolutional kernels and three ReLU activation functions to obtain a third feature sequence K, where the feature size of K is (1, M, 512); and generating an attention map α for each character position. ij Attention map α ij The expression is:

[0035]

[0036] in,

[0037] e ij =q i T k j +b i j = 1...M,

[0038] Where q i b is a trainable vector. i Let L be the trainable offset, L be the maximum number of characters in the license plate, including a terminator; M be the column number of the third feature sequence K; and k be the trainable offset. j It is the j-th feature vector in the third feature sequence K;

[0039] Attention map α ij The final image feature c is obtained by performing a high-dimensional matrix multiplication with the second feature sequence z. i , will c i The license plate recognition result is obtained by performing multi-class classification through a neural network.

[0040] The following are specific embodiments of license plate recognition using the present invention:

[0041] Step 1: Preprocess the license plate image, resize it to (140, 440), and normalize the image channel by channel using a normalization method, setting the mean and standard deviation of each channel to 0.5. The license plate number is encoded according to the pre-defined order of the characters.

[0042] Step 2: Input the license plate image obtained in Step 1 into the network model. First, it passes through a convolutional neural network to extract features (ResNet18 is used in this implementation, but other convolutional networks can also be used) to obtain feature T. Then, adaptive average pooling is used to transform it into a feature sequence T′ of a specific size. Finally, T′ is reduced to (14, 512) to obtain z = (z1, z2, ..., z M The size of M is 14. Each feature vector in this sequence covers a local region of the license plate image, such as... Figure 2 As shown.

[0043] Step 3: A position-aware attention module computes an attention map for each character position and uses this attention map to select local features from the image feature encoding z to identify the character. Since the positions of each character are uncorrelated, the attention module evaluates each attention map independently. The feature sequence obtained in Step 2 is then processed by three convolutional kernels and three ReLU activation functions to obtain K, with a feature size of (1, M, 512). To generate an attention map, we use a trainable query variable q. i and an offset b i To calculate e i :

[0044] e ij =q i T k j +b i ,j=1....M,

[0045] Location Attention Map Finally, it is calculated using the following formula:

[0046]

[0047] Where qi is a trainable vector, b i The trainable offsets are M=14 and L=9, which represent the maximum number of characters in the license plate (including a terminator).

[0048] Step 4: Obtain the positional attention map α ij Then, use a linear decoder for each y. i First, calculate an image feature c. i ∈R 512 The algorithm is as follows:

[0049]

[0050] Step 5: Each character y i probability distribution ρ i c after using Softmax i Obtain

[0051] The expression for multi-category classification is:

[0052]

[0053] Where ρ ij =P(y i =j∨χ), which is the probability that the i-th character on the license plate is the j-th symbol in the character set. θ ij ∈R 512It is sc

[0054] The table below compares this invention with other license plate recognition algorithms:

[0055] Comparison of test parameters between this invention and other license plate recognition algorithms

[0056]

[0057] The above-described embodiments are merely preferred embodiments of the present invention, and the scope of protection of the present invention is not limited thereto. Any simple changes or equivalent substitutions of the technical solutions that can be obviously obtained by those skilled in the art within the scope of the technology disclosed in the present invention shall fall within the scope of protection of the present invention.

Claims

1. A license plate recognition method based on positional attention, characterized in that, include: The license plate image is input into a convolutional neural network to extract features T. Adaptive average pooling is used to transform feature T into a first feature sequence T1 of a specific size. After dimensionality reduction of T1, a second feature sequence z is obtained. The second feature sequence z is processed by convolution operations with three convolution kernels and three ReLU activation functions to obtain the third feature sequence K, with feature size (1, M, 512). An attention map is generated for the position of each character in the license plate image. Attention graph The expression is: in, , in, For the training vectors, The training offset is L, where L is the maximum number of characters in the license plate, including a terminator; M is the column number of the third feature sequence K; k j It is the j-th feature vector in the third feature sequence K; The attention map The final image features are obtained by performing a high-dimensional matrix multiplication with the second feature sequence z. The final image features The license plate recognition result is obtained by performing multi-class classification through a neural network.

2. The license plate recognition method based on positional attention according to claim 1, characterized in that, The feature size of the second feature sequence z is (14, 512).

3. The license plate recognition method based on positional attention according to claim 1, characterized in that, The It equals 9.

4. The license plate recognition method based on positional attention according to claim 1, characterized in that, The image features The expression is: Among them, z j Let be the j-th feature vector in the second feature sequence z.

5. The license plate recognition method based on positional attention according to claim 1, characterized in that, The expression for the multi-category classification operation is: in Let be the probability that the i-th character on the license plate is the j-th symbol in the character set; , These are the parameters used in softmax training.

6. The license plate recognition method based on positional attention according to claim 1, characterized in that, The convolutional neural network used is ResNet18.

7. The license plate recognition method based on positional attention according to claim 1, characterized in that, Also includes: The license plate image is normalized, and the normalized image is input into a convolutional neural network to extract features T. The normalization process includes: standardizing the image channel by channel using a normalization method, with the mean and standard deviation of each channel set to 0.5.

Images