Lightweight license plate recognition method based on perspective transformation

Abstract

The invention relates to a light-weight license plate recognition method based on perspective transformation, and the method comprises the following steps: obtaining an image to be recognized, inputting the image into a license plate recognition model, and outputting a model recognition result, the license plate recognition model comprises a light-weight license plate positioning module LVLP and a perspective transformation light-weight license plate recognition module LVLR, the lightweight license plate positioning module LVLP outputs the coordinates of the position matrix of the license plate, and the license plate recognition model is embedded into a U2-net framework. Compared with the prior art, the method has the advantages of improving the license plate recognition accuracy and the like.

Description

Technical Field

[0001] The invention relates to the technical field of license plate recognition, and in particular to a lightweight license plate recognition method based on perspective transformation. Background Art

[0002] Identifying whether the license plate is on the front or the rear of the vehicle is crucial for vehicle management. Automatic license plate recognition (LPR) plays a key role in many application scenarios. Traditional methods have many limitations, such as only being able to recognize specific types of license plates, limiting vehicle speed, and relying on background conditions. License plates are often affected by stains, light, rust, deformation, etc., resulting in defects or unclear markings after character segmentation, which seriously affects the accuracy of feature description. Standard Chinese license plates have many Chinese marks and strokes, making them difficult to recognize. China's standard license plates are composed of Chinese characters, English characters, and Arabic numerals. The difference between Chinese characters and English letters and numbers in the recognition process is significant, resulting in a decrease in recognition accuracy, which increases the technical challenge.

[0003] Existing neural network recognition often relies on RNN neural networks, which easily leads to a decrease in recognition accuracy. Summary of the invention

[0004] The purpose of the present invention is to provide a lightweight license plate recognition method based on perspective transformation in order to improve the accuracy of license plate recognition.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A lightweight license plate recognition method based on perspective transformation, the method comprising the following steps:

[0007] Acquire an image to be recognized, input the image into a license plate recognition model, and output a model recognition result. The license plate recognition model includes a lightweight license plate positioning module LVLP and a perspective transformation lightweight license plate recognition module LVLR. The lightweight license plate positioning module LVLP outputs the matrix coordinates of the license plate position. The license plate recognition model is embedded in a U2-net framework.

[0008] Among them, the lightweight license plate positioning module LVLP module includes data enhancement, two consecutive Focus+CSP modules, FPN+PAN module, and CIOU_Loss loss function module, and the CIOU_Loss loss function module is used to train the lightweight license plate positioning module LVLP module;

[0009] The training process of the lightweight license plate positioning module LVLP is as follows:

[0010] The license plate image is corrected by perspective transformation technology as a training set, and the corrected image is input into LVLP. The character features are extracted through the backbone network of LVLR. Then CTC and beam search algorithms are used to output the predicted license plate number. The parameters of the lightweight license plate positioning module LVLP are trained based on the predicted license plate number.

[0011] Furthermore, the Focus in the Focus+CSP module is used to transform the data-augmented image from the WH plane to the channel dimension, and then convolution is used to separate different features.

[0012] Furthermore, the loss function in the CIOU_Loss loss function module is

[0013]

[0014] Among them, v represents the unification of the aspect ratio, Dis_b represents the distance between the center points of the prediction box and the reference box, and Dis_c refers to the diagonal length of the minimum rectangle enclosing the prediction box and the reference box. IOU represents the intersection-over-union loss function.

[0015] Furthermore, in the lightweight license plate positioning module LVLP module, after the image is input into the LVLP module, it passes through data enhancement, two consecutive Focus+CSP modules and FPN+PAN modules in sequence, outputs a prediction box, and outputs the matrix coordinates of the license plate location.

[0016] Furthermore, a non-maximum suppression algorithm is used before outputting the prediction box to remove redundant boxes.

[0017] Furthermore, the specific steps of extracting character features through the LVLR backbone network and then using CTC and beam search algorithms to output the predicted license plate number are as follows:

[0018] The prediction box is input into LVLR, and the convolution module of the backbone network based on LVLR is matched with the license plate template set to output a probability sequence, which reflects the possibility of each character. Its length is only related to the width of the input image of LVLR. The top n probability sequences with the highest probability are returned through the beam search algorithm, and the CTC loss function of the top n probability sequences is calculated. The length of the top n probability sequences is optimized based on the loss function, and the predicted license plate number is output.

[0019] Furthermore, the convolution kernel size of the convolution module is 1x13.

[0020] Furthermore, the specific steps of returning the top n probability sequences with the highest probability through the beam search algorithm are:

[0021] When searching for the top n probability sequences with the highest probability, a breadth-first approach is used to construct a search tree, and nodes are sorted according to their probability on each layer of the tree, retaining breadth nodes with high scores, while expanding nodes on the next layer and removing nodes with low scores.

[0022] Furthermore, the CTC loss function is:

[0023]

[0024] Among them, G represents the probability loss or total loss value that needs to be identified, X represents the identified variable value, W represents the weight or probability value of the variable value, P represents the probability value, o represents the scaling factor, Q represents the number of characters that need to be aligned, E represents a fixed loss constant, D represents the total number of fields that need to be aligned, and T represents the possible number of changes for each character.

[0025] Furthermore, the training step of the license plate recognition model is: obtaining the trained lightweight license plate positioning module LVLP and the perspective transformation lightweight license plate recognition module LVLR, integrating the two modules and performing parameter fine-tuning together to obtain the license plate recognition model.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] The present invention replaces the LSTM-based RNN neural network with the context structure of local characters. This method avoids the reliance on RNN, so that the output of the subnetwork can be regarded as a probability sequence, which reflects the possibility of each character, and its length is only related to the width of the input image. The number of characters is controlled in accordance with the target by combining the CTC loss function, and end-to-end license plate recognition is realized. Therefore, the LVLR algorithm module can be created and used in embedded systems. When recognizing relatively complex Chinese license plates, the lightweight LVLR still has good performance. At the same time, the clustering algorithm is used to search for the highest probability sequence to improve the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flow chart of the present invention;

[0029] Figure 2 This is the LVLP structure diagram;

[0030] Figure 3 It is the structure of LVLR backbone network;

[0031] Figure 4 The final recognition result. DETAILED DESCRIPTION

[0032] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0033] The present invention proposes a lightweight license plate recognition method based on perspective transformation, which comprises the following steps:

[0034] Acquire an image to be recognized, input the image into a license plate recognition model, and output a model recognition result. The license plate recognition model includes a lightweight license plate positioning module LVLP and a perspective transformation lightweight license plate recognition module LVLR. The lightweight license plate positioning module LVLP outputs the matrix coordinates of the license plate position. The license plate recognition model is embedded in a U2-net framework.

[0035] Among them, the lightweight license plate positioning module LVLP module includes data enhancement, two consecutive Focus+CSP modules, FPN+PAN module, and CIOU_Loss loss function module, and the CIOU_Loss loss function module is used to train the lightweight license plate positioning module LVLP module;

[0036] The training process of the lightweight license plate positioning module LVLP is as follows:

[0037] The license plate image is corrected by perspective transformation technology as a training set, and the corrected image is input into LVLP. The character features are extracted through the backbone network of LVLR. Then CTC and beam search algorithms are used to output the predicted license plate number. The parameters of the lightweight license plate positioning module LVLP are trained based on the predicted license plate number.

[0038] The perspective transformation lightweight license plate recognition network model of the present invention mainly uses a lightweight license plate positioning module (Lightweight Vehicle Plate Positioning LVLP) and a perspective transformation lightweight license plate recognition module (Lightweight Vehicle Plate Recognition LVLR) to solve relatively complex actual mining production scenarios. Similar to the traditional recognition process, it is divided into two subsystems: one is LVLP for image segmentation and license plate location identification; the other is the LVLR system, whose goal is to correct and identify license plate distortion and eliminate noise. LVLP is a single-stage target recognition algorithm that takes performance and accuracy into consideration. It not only understands the effectiveness of rapid recognition, but also improves the accuracy of small target recognition. The overall workflow of license plate recognition is shown in Figure 1 .

[0039] The present invention combines LVLP and LVLR to solve the problem of vehicle license plate distortion in complex mining environment. In the early stage, data needs to be integrated before training. For LVLR, too many training models are not needed. For a single vehicle model, only a single training is needed. After the training model is well trained, no further training is needed in the later stage. The trained license plate detection system LVLP is combined with the character recognition system LVLR to process the input image, and the detection system is used to generate positioning frames. These positioning frames are then used as inputs of the LVLR network. By binarizing the license plate image, the four corner points of the license plate can be easily identified, and then the license plate image is corrected by perspective transformation technology, and the corrected image is input into LVLR for processing. The character recognition system LVLR first extracts character features from the identified candidate area through its backbone network, and then uses CTC and beam search algorithms to output the final license plate number. In view of the fact that the license plate may be tilted due to factors such as viewing angle in actual applications, and the images captured by most camera devices are distorted, the system prolongs the time of the detection stage to achieve a more accurate recognition effect.

[0040] LVLP consists of four parts: data enhancement, Focus+CSP, FPN+PAN, and CIOU_Loss. Figure 2 shown.

[0041] First, the Focus layer here is similar to the PassThrough layer of YOLOv3, converting information from the WH plane to the channel dimension, and then separating different features through convolution. The Focus layer is used for downsampling. Downsampling in neural networks is mainly used to reduce the number of parameters, reduce the dimension, and increase local sensitivity. Compared with the convolution layer and pooling layer with a step size of 2, the Focus layer can effectively reduce the information loss caused by downsampling and reduce the amount of calculation. The structure of Bottleneck is through 1X1 and 3X3 convolutions, in which the convolution process is to halve and then double the number of channels. Therefore, the number of channels does not change the feature scale before and after passing through the Bottleneck module. In BottleneckCSP, cv2 and cv3 use the system-level convolution layer of YOLO. After the concat connection, it is connected with the BN layer and the activation function; C3 directly uses a custom convolution layer.

[0042] At the input end, y5 has not changed much compared to y4. Both use Mosaic data enhancement and are improved by referring to the data enhancement method of CutMix. The previous two pictures are randomly cut, randomly combined, and randomly assembled into four pictures. In this way, a lot of data containing small targets is obtained, which enriches the data set and improves the detection ability of small targets.

[0043] Y5 also adds an adaptive algorithm to determine the anchor box value. In the previous YOLO series, this step was performed by a separate module. The LVLP module directly incorporates this function into the code. In the YOLO algorithm, it is usually necessary to set the initial anchor box before model training. During the model training process, it is necessary to make a prediction based on the initial anchor box, take the difference between the predicted value and the initial value, and then perform a reverse update and iteration to make the anchor box closer to the actual value. The four modules in the LVLP module code are different from the cfg files in the previous YOLOv3 and YOLOv4. They are all presented in the form of yaml. Moreover, the contents of the four files are basically the same, with only the two parameters depth_multiple and width_multiple at the top being different.

[0044] The most important point for the input end may be the size of the image. If the size of the image is biased, it will be meaningless for subsequent training and prediction. In the YOLO series of algorithms, the input image size is uniform, which is very convenient for training and data processing of pictures of uniform size. The code of the LVLP module optimizes this problem. In real application scenarios, it is observed that the aspect ratios of many images are different, resulting in inconsistent black borders on both sides of the image after scaling and padding. Excessive padding may cause an excess of information, which in turn has an adverse effect on the inference speed. To this end, the Letterbox function is adjusted in the code of the WZ module so that the original image can adaptively add the least black border.

[0045] The prediction module is formed by loss function and non-maximum suppression. In the prediction module, IOU is a training parameter, which is the intersection-union ratio of the prediction box and the target box, and its value is between 0 and 1. The loss function used in the LVLP module is CIOU_Loss. Compared with the traditional loss function, the main difference is that the intersection-union ratio is optimized. The optimized formula is:

[0046]

[0047] Where v represents the uniform aspect ratio, Dis_b represents the distance between the center points of the prediction box and the reference box, and Dis_c refers to the diagonal length of the smallest rectangle that encloses the prediction box and the reference box. In the process of generating many target boxes, this chapter uses the weighted non-maximum suppression (WNMS) method for screening and processing. The non-maximum suppression algorithm is usually used to remove redundant boxes after model prediction, so as to accelerate faster and improve accuracy.

[0048] In the LVLR-based license plate character recognition module, character recognition is involved in the license plate character recognition process. License plate recognition is a complex and important task in urban traffic management, video surveillance, vehicle identification and parking management. The problem of recognizing license plates becomes more complicated because pictures, lighting conditions, weather conditions, image deformation and variable license plates are all reasons that make license plate recognition difficult. A powerful character recognition system should be able to handle switching between different environments without losing accuracy. LVLR does not pre-segment characters and can be said to be an end-to-end license plate recognition algorithm. Convolutional neural networks demonstrate their effectiveness and advantages in computer vision tasks such as image classification, object detection and semantic segmentation. The lightweight LVLR module composed of convolutional neural networks supports end-to-end training. Thanks to the design of the LVLR architecture, this real-time recognition system does not use RNN technology. Therefore, the LVLR algorithm module can be created and used in embedded systems. The lightweight LVLR still has good performance when recognizing relatively complex Chinese license plates. The structure of the LVLR backbone network is as follows Figure 3 As shown. The core network starts with the original RGB image and analyzes the spatial layout of a large number of functions. Wide convolution (1*13 convolution kernel) is used instead of the traditional LSTM-based RNN neural network to exploit the context structure of local characters. This method avoids the reliance on RNN and allows the output of the sub-network to form a probability sequence that reflects the possibility of each character and its length is only related to the width of the input image. Since the length of the decoder output is inconsistent with the target sequence length, CTC loss is introduced to support end-to-end training without segmentation. CTC loss is a common method to solve the problem of mismatch in input and output sequence length.

[0049] The original RGB image is an RGB image with a source network, which is used as the input of the CNN and extracts image attributes. A 1*13 connection kernel associated with the context is used instead of an LSTM-based RNN. The main output of the network is processed into a sequence representing character probabilities, and its sequence length is only related to the width of the input image. Since the length of the network output does not match the length of the actual license plate, this study uses CTC loss to achieve end-to-end training. In addition, CTC converts the probability of each time frame into an output probability.

[0050] Typically, character recognition can be achieved through LSTM, Seq2Seq combined with CTC, or CNN methods, which do not require individual segmentation of characters. First, features are extracted through the core network of LVLR, and then the convolutional layer is used to recognize the character sequence on the license plate. Considering the inconsistency in length between the sequence produced by the LVLR network and the target sequence, CTC loss is chosen to bridge this difference. On the other hand, the use of Softmax-Loss requires establishing a one-to-one correspondence between each column of the output and each character element, that is, marking the specific position of each character in the picture in the training data, which can be very time-consuming. CTC (Connectionist Temporal Classification) technology is specifically used to deal with the inconsistency between labels and network outputs. The loss is calculated using the following formula:

[0051]

[0052]

[0053] LVLR also uses beam search technology, which is also called beam-directed search and is a heuristic search strategy. This algorithm uses a heuristic function to self-evaluate the search efficiency of each node. This algorithm becomes particularly important when the solution space is large. In order to save memory and time resources, this method will eliminate those nodes with low scores when performing a deep search, and only retain the width nodes with high scores. This approach not only reduces the use of space, but also improves the efficiency of the search. Beam search uses a breadth-first method to build a search tree, and sorts the nodes on each layer of the tree according to the probability size, retaining only nodes of a certain width, while expanding the nodes of the next layer and discarding the remaining nodes. Through beam search, LVLR can find the top n sequences with the highest probability and return the first successfully matched license plate template set, which is based on the standard of the motor vehicle license plate. Due to the limitation of beam_size, n represents the number of potential words retained at each decision point.

[0054] The present invention has verified the wide applicability of convolutional neural networks (CNN) for license plate recognition tasks without size restrictions. Combining the U2-net and LVLR models, this study can effectively handle the complex deformation problem of license plate data. Experimental results show that the system using U2-net and LVLR in combination can achieve real-time inference performance across a variety of hardware platforms such as CPU, GPU and FPGA. Undoubtedly, the license plate recognition scheme proposed in this study has demonstrated the ability to achieve real-time processing on low-power embedded devices. By applying modern model pruning and quantization techniques, this chapter is expected to further simplify the computational requirements of the network model. Future work can consider incorporating CNN-based vehicle front-end and back-end detection functions into this algorithm to expand the application scope of LVLR, and merge the detection and recognition processes into a single network model for evaluation, aiming to improve the overall recognition accuracy.

[0055] Let’s do the experiment:

[0056] In order to design a better license plate recognition algorithm, this chapter first built the overall model of LVLP and LVLR, realized end-to-end recognition, entered the neural network from the overall image to realize the black box operation, and improved the license plate positioning accuracy of LVLP by optimizing the loss function in the neural network of LVLP. By comparing with the traditional license plate recognition algorithm, a more obvious gap was found.

[0057] In the special case of mining vehicles, the LVLP and LVLR models were retrained, which not only improved the accuracy of the experimental results, but also prevented the overfitting phenomenon of the recognition results. The experimental data in this chapter are pictures and equipment provided by a mining company and a surveillance camera company. The experimental data includes the appearance, front, rear, and multiple interference sources of the vehicle. The experiment requires that the model can be trained to recognize three types of "front", "rear", and "special vehicles". Special vehicles include private cars, sprinkler trucks, trailers, etc. After the classification is completed, the license plate recognition model is used to obtain the specific digital content of the license plate.

[0058] The experiment first transforms the data set, converts the labelme-annotated xml data into the LVLP recognition txt file format, and then configures the training parameters. The specific parameter table is shown in Table 1. The images input into LVLP must be output in the format so that they can enter the neural network for training correctly. Batch-size is the number of single training images, and epochs is the number of training rounds.

[0059] Table 1 Hyperparameter settings

[0060]

[0061] The first phase of training to locate license plates through LVLP took 5 hours. By strengthening the annotation accuracy of the data set and optimizing the model, the overall recognition accuracy was improved. During the training process, data enhancement and other models were used to further enrich the data on a limited data set. The experiment randomly selected 2,000 pictures from the data set as training objects, 80% as training sets and 20% as test sets. In this experiment, this chapter used Adam as the optimization algorithm. Its main advantage is that as the training process progresses, the learning rate will gradually decrease, allowing the network parameters to be continuously fine-tuned. The key hyperparameter adjustments are mainly based on factors such as weight decay and learning rate. In particular, the learning rate hyperparameter plays a vital role in preventing the occurrence of model overfitting. Adjust the learning rate according to the training situation.

[0062] After training for 300 epchs, the LVLP model has fully reached a convergence state for the license plate recognition of the slag truck in the mine. During the model training process, the accuracy has been steadily improving, which is mainly reflected in the continuous reduction of the loss. The final recognition effect is as follows: Figure 4 As shown in the figure, the location of the license plate has been clearly marked in red. Whether the vehicle is sideways, at night, or when there is a lot of dust, the license plate can be accurately located.

[0063] In the process of building the LVLR model, it was found that the tilt angle of the license plate would seriously affect the model's prediction of the results. After trying some correction algorithms, it was found that the effect was not particularly ideal, so a physical projection transformation was added to make the license plate with a large tilt angle more accurately predicted. Many LPR license plate detection algorithms are designed based on neural networks, and LVLR is no exception. Unlike other license plate detection algorithms, LVLR is designed for small embedded devices. Therefore, it is generally necessary to train a model in a targeted manner and then import it into the embedded device.

[0064] The collection process of the license plate dataset is different from that of the front and rear datasets. The recognition of license plate characters needs to use a principle similar to OCR character recognition. The label of the image corresponds to the characters on the license plate. The license plate needs to be cropped and trimmed to make the inclination angle of the license plate approach zero. The performance of the model is analyzed. Table 2 shows the recognition results of different models on truck license plates.

[0065] Table 2 Recognition rate and frame rate of each model

[0066]

[0067] It can be seen that on the data set given by the mining company, the perspective transformation based lightweight license plate model PTLVLPR proposed in this chapter can reach a recognition frame rate of 58 frames on the GPU, and the recognition efficiency is improved by 5.63%, 9.38%, and 3.12% respectively compared with License_plate, YOLOv4, and YOLOv5. Regarding the recognition speed, on the average performance GeForceGTX1080, the GPU with a video memory size of 4GB, the FPS of the model proposed in this chapter is 58.57 frames per second, which is significantly improved compared with the FPS of the traditional model; Through the above experimental data analysis, it can be seen that the perspective transformation lightweight license plate recognition network model proposed in this chapter has greatly improved the speed of license plate recognition on the average performance GPU or even CPU while maintaining the recognition rate of 95.6%. Therefore, the real-time performance of license plate recognition is improved, which has certain practical application value for the recognition of license plates in future autonomous driving.

[0068] In the increasingly advanced modern traffic system, traffic management, traffic monitoring and traffic optimization require a lot of manpower and material resources. If the system can replace some tedious and repetitive management work, it will save a lot of manpower and material resources. Automatic license plate recognition is an important link and one of the key technologies for implementing intelligent traffic management. In the mining environment of the mine environment, due to the complex mining environment, serious license plate pollution and distortion, license plate recognition is relatively complicated, and the recognition efficiency is not very ideal. It is necessary to train the model according to the specific situation of the vehicle, and avoid overfitting during training. Therefore, this chapter designs a lightweight license plate recognition network PTLVLPR based on perspective transformation. The network is mainly divided into two stage modules, one is the LVLP license plate positioning module, and the other is the LVLR license plate recognition module. The LVLP module draws on the latest YOLO algorithm and uses the one-stage target detection segmentation technology to extract the matrix coordinates of the license plate location and input it to the next stage of the license plate recognition module LVLR. In the license plate recognition module LVLR, the perspective transformation technology is used to transform the distorted, deformed, and side-biased license plate into a standard rectangular shape. After that, the rectangular digital image module is used as input to the convolution module of the LVLR network. Here, (1x13 convolution kernel) is used. This method replaces the LSTM-based RNN neural network with the context structure of the local characters. This method avoids the dependence on RNN, so that the output of the subnetwork can be regarded as a probability sequence, which reflects the possibility of each character and its length is only related to the width of the input image. This is because the length of the decoder output is inconsistent with the length of the target sequence. Therefore, the CTC loss function is introduced here. The CTC loss function is a method commonly used to solve the problem of mismatch between input and output sequences without the need for segmented end-to-end training. The U2-net framework is used to integrate these technologies to form the LVLR license plate character recognition module. The LVLP and LVLR modules are trained separately and then connected as a whole PTLVLPR lightweight license plate recognition network based on perspective transformation. Then the parameters are uniformly trained and fine-tuned. The experimental results show that the algorithm has a good recognition rate in terms of speed and recognition rate, as well as the recognition of lateral distorted license plates. Once vehicle license plate recognition is intelligent and completely unmanned, it will make vehicle management more efficient and convenient. At the same time, it also makes mine management in the mining environment more orderly, standardized and intelligent.

[0069] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A lightweight license plate recognition method based on perspective transformation, characterized in that: The method comprises the following steps: Acquire an image to be recognized, input the image into a license plate recognition model, and output a model recognition result. The license plate recognition model includes a lightweight license plate positioning module LVLP and a perspective transformation lightweight license plate recognition module LVLR. The lightweight license plate positioning module LVLP outputs the matrix coordinates of the license plate position. The license plate recognition model is embedded in a U2-net framework. Among them, the lightweight license plate positioning module LVLP module includes data enhancement, two consecutive Focus+CSP modules, FPN+PAN module, and CIOU_Loss loss function module, and the CIOU_Loss loss function module is used to train the lightweight license plate positioning module LVLP module; The training process of the lightweight license plate positioning module LVLP is as follows: The license plate image is corrected by perspective transformation technology as a training set, and the corrected image is input into LVLP. The character features are extracted through the backbone network of LVLR. Then CTC and beam search algorithms are used to output the predicted license plate number. The parameters of the lightweight license plate positioning module LVLP are trained based on the predicted license plate number.

2. The lightweight license plate recognition method based on perspective transformation according to claim 1 is characterized in that: The Focus in the Focus+CSP module is used to convert the data-augmented image from the WH plane to the channel dimension, and then separate different features through convolution.

3. The lightweight license plate recognition method based on perspective transformation according to claim 2 is characterized in that: The loss function in the CIOU_Loss loss function module is: Among them, v represents the unification of the aspect ratio, Dis_b represents the distance between the center points of the prediction box and the reference box, and Dis_c refers to the diagonal length of the minimum rectangle enclosing the prediction box and the reference box. IOU represents the intersection-over-union loss function.

4. The lightweight license plate recognition method based on perspective transformation according to claim 3 is characterized in that: In the lightweight license plate positioning module LVLP module, after the image is input into the LVLP module, it passes through data enhancement, two consecutive Focus+CSP modules and FPN+PAN modules in sequence, outputs the prediction frame, and outputs the matrix coordinates of the license plate location.

5. The lightweight license plate recognition method based on perspective transformation according to claim 4 is characterized in that: The non-maximum suppression algorithm is used before outputting the prediction box to remove redundant boxes.

6. A lightweight license plate recognition method based on perspective transformation according to claim 5, characterized in that: The specific steps of extracting character features through the LVLR backbone network and then using CTC and beam search algorithms to output the predicted license plate number are as follows: The prediction box is input into LVLR, and the convolution module of the backbone network based on LVLR is matched with the license plate template set to output a probability sequence, which reflects the possibility of each character. Its length is only related to the width of the input image of LVLR. The top n probability sequences with the highest probability are returned through the beam search algorithm, and the CTC loss function of the top n probability sequences is calculated. The length of the top n probability sequences is optimized based on the loss function, and the predicted license plate number is output.

7. A lightweight license plate recognition method based on perspective transformation according to claim 6, characterized in that: The convolution kernel size of the convolution module is 1x13.

8. The lightweight license plate recognition method based on perspective transformation according to claim 7 is characterized in that: The specific steps to return the top n probability sequences with the highest probability through the beam search algorithm are: When searching for the top n probability sequences with the highest probability, a breadth-first approach is used to construct a search tree, and nodes are sorted according to their probability on each layer of the tree, retaining breadth nodes with high scores, while expanding nodes on the next layer and removing nodes with low scores.

9. The lightweight license plate recognition method based on perspective transformation according to claim 8 is characterized in that: The CTC loss function is: Among them, G represents the probability loss or total loss value that needs to be identified, X represents the identified variable value, W represents the weight or probability value of the variable value, P represents the probability value, o represents the scaling factor, Q represents the number of characters that need to be aligned, E represents a fixed loss constant, D represents the total number of fields that need to be aligned, and T represents the possible number of changes for each character.

10. The lightweight license plate recognition method based on perspective transformation according to claim 1, characterized in that: The training steps of the license plate recognition model are: obtaining the trained lightweight license plate positioning module LVLP and the perspective transformation lightweight license plate recognition module LVLR, integrating the two modules and performing parameter fine-tuning together to obtain the license plate recognition model.

Images