License plate recognition method, related equipment and readable storage medium

Abstract

The present application discloses a license plate recognition method, related equipment and readable storage medium. First, a license plate area image including a single-line license plate area image and a double-line license plate area image is used as a training sample, and the license plate color, number of license plate rows and license plate characters annotated by the training license plate area image are used as sample labels to train a license plate recognition model. After determining the license plate area image to be recognized, the license plate area image to be recognized is input into the license plate recognition model, and the license plate recognition model recognizes the license plate area image to be recognized, and the license plate color, number of license plate rows and license plate characters corresponding to the license plate area image to be recognized can be obtained. This solution can be applied to both single-line license plate recognition and double-line license plate recognition.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to a license plate recognition method, related equipment and a readable storage medium. Background Art

[0002] License plate recognition refers to the technology that can automatically extract license plate information (including license plate characters and license plate color) for processing. This technology has been applied to various scenarios such as highway toll collection, parking management, weighing systems, traffic guidance, traffic law enforcement, highway inspection, vehicle dispatching, and vehicle detection. License plates can be divided into single-line license plates and multi-line license plates based on the number of lines of characters in the license plate. The license plates of small cars are generally single-line license plates, and the license plates of large cars, trailers, armed police vehicles, military vehicles, agricultural vehicles, motorcycles, etc. are generally double-line license plates.

[0003] The current license plate recognition scheme can only be used to recognize single-row license plates or double-row license plates, but cannot be used to recognize both single-row license plates and double-row license plates at the same time.

[0004] Therefore, how to provide a license plate recognition method that can be applied to both single-row license plate recognition and double-row license plate recognition has become a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] In view of the above problems, this application proposes a license plate recognition method, related equipment and readable storage medium. The specific scheme is as follows:

[0006] A license plate recognition method, the method comprising:

[0007] Determine the license plate area image to be recognized;

[0008] The license plate area image to be identified is input into a license plate recognition model, and the license plate recognition model identifies the license plate area image to be identified to obtain the license plate color, license plate row number and license plate character corresponding to the license plate area image to be identified; the license plate recognition model is trained using training license plate area images as training samples and the license plate color, license plate row number and license plate character annotated with the training license plate area images as sample labels, and the training license plate area images include single-row license plate area images and double-row license plate area images.

[0009] Optionally, determining the license plate area image to be recognized includes:

[0010] Get the original image provided by the user;

[0011] Detecting the original image to obtain location information of the license plate in the original image;

[0012] Cutting the original image according to the position information of the license plate to obtain at least one original license plate area image;

[0013] Based on the at least one original license plate area image, a license plate area image to be recognized is determined.

[0014] Optionally, detecting the original image to obtain the position information of the license plate in the original image includes:

[0015] The original image is input into a license plate detection model, and the license plate detection model outputs the position information of the license plate in the original image; the license plate detection model is trained using the training image as a training sample and the position information of the license plate annotated in the training image as a sample label.

[0016] Optionally, determining the license plate area image to be recognized based on the at least one original license plate area image includes:

[0017] For each original license plate area image, the original license plate area image is judged to obtain a first judgment result or a second judgment result, wherein the first judgment result is used to indicate that the image is a license plate image; and the second judgment result is used to indicate that the image is not a license plate image;

[0018] Based on the original license plate area image corresponding to the first determination result, a license plate area image to be identified is determined.

[0019] Optionally, the determining the original license plate area image includes:

[0020] The original license plate area image is input into a license plate determination model, and the license plate determination model outputs a determination result corresponding to the original license plate area image; the license plate determination model is trained using training images as training samples and using the first determination result or the second determination result annotated with the training images as sample labels.

[0021] Optionally, determining the license plate area image to be identified based on the original license plate area image corresponding to the first determination result includes:

[0022] Performing key point detection on the original license plate area image corresponding to the first determination result to determine the key points in the original license plate area image corresponding to the first determination result;

[0023] Based on the key points in the original license plate area image corresponding to the first determination result, an affine transformation is performed on the original license plate area image corresponding to the first determination result to obtain a corrected license plate area image, and the corrected license plate area image is the license plate area image to be identified.

[0024] Optionally, performing key point detection on the original license plate area image corresponding to the first determination result to determine the key points in the original license plate area image corresponding to the first determination result includes:

[0025] The original license plate area image corresponding to the first judgment result is input into a key point detection model, and the key point detection model outputs the key points in the original license plate area image corresponding to the first judgment result. The key point detection mode is obtained by training using the training license plate image as a training sample and the key points annotated with the training license plate image as sample labels.

[0026] Optionally, the license plate recognition model includes an image feature extraction module, a license plate color and license plate row number recognition module, an image feature optimization module and a license plate character recognition module;

[0027] The license plate recognition model recognizes the license plate area image to be recognized, and obtains the license plate color, license plate row number and license plate characters corresponding to the license plate area image to be recognized, including:

[0028] The image feature extraction module extracts features from the image of the license plate area to be identified, and obtains an initial feature vector of the image of the license plate area to be identified;

[0029] The license plate color and license plate row number recognition module determines the license plate color and the license plate row number based on the initial feature vector;

[0030] The image feature optimization module optimizes the initial feature vector to obtain a final feature vector of the image of the license plate area to be identified;

[0031] The license plate character recognition module determines the license plate character based on the final feature vector.

[0032] Optionally, the image feature optimization module optimizes the initial feature vector to obtain a final feature vector of the image of the license plate area to be identified, including:

[0033] Converting the initial feature vector into a first feature vector, wherein the length of the first feature vector is half of the length of the initial feature vector and the width is the same as the width of the initial feature vector;

[0034] The first eigenvector is equally divided in width to obtain a second eigenvector and a third eigenvector; the width of the second eigenvector and the third eigenvector are both half of the width of the initial eigenvector;

[0035] The second eigenvector and the third eigenvector are concatenated in length to obtain a fourth eigenvector, wherein the length of the fourth eigenvector is the same as that of the initial eigenvector, and the width is half of the width of the initial eigenvector;

[0036] Converting the initial eigenvector into a fifth eigenvector, wherein the length of the fifth eigenvector is half of the length of the initial eigenvector and the width is half of the width of the initial eigenvector;

[0037] The fourth eigenvector and the fifth eigenvector are concatenated in length to obtain a final eigenvector of the image of the license plate area to be identified.

[0038] A license plate recognition device, comprising:

[0039] A determination unit, used to determine the license plate area image to be identified;

[0040] A license plate recognition unit is used to input the license plate area image to be recognized into a license plate recognition model, and the license plate recognition model recognizes the license plate area image to be recognized to obtain the license plate color, license plate row number and license plate character corresponding to the license plate area image to be recognized; the license plate recognition model is trained using training license plate area images as training samples and the license plate color, license plate row number and license plate character annotated with the training license plate area images as sample labels, and the training license plate area images include single-row license plate area images and double-row license plate area images.

[0041] Optionally, the determining unit includes:

[0042] An original image acquisition unit, used to acquire an original image provided by a user;

[0043] A detection unit, used to detect the original image to obtain the position information of the license plate in the original image;

[0044] A cropping unit, used for cropping the original image according to the position information of the license plate to obtain at least one original license plate area image;

[0045] The first determining unit is used to determine the license plate area image to be recognized based on the at least one original license plate area image.

[0046] Optionally, the detection unit is specifically used to:

[0047] The original image is input into a license plate detection model, and the license plate detection model outputs the position information of the license plate in the original image; the license plate detection model is trained using the training image as a training sample and the position information of the license plate annotated in the training image as a sample label.

[0048] Optionally, the first determining unit includes:

[0049] A determination unit, for determining each original license plate area image, to obtain a first determination result or a second determination result, wherein the first determination result is used to indicate that the image is a license plate image; and the second determination result is used to indicate that the image is not a license plate image;

[0050] The second determining unit is used to determine the license plate area image to be identified based on the original license plate area image corresponding to the first determination result.

[0051] Optionally, the determining unit is specifically configured to:

[0052] The original license plate area image is input into a license plate determination model, and the license plate determination model outputs a determination result corresponding to the original license plate area image; the license plate determination model is trained using training images as training samples and using the first determination result or the second determination result annotated with the training images as sample labels.

[0053] Optionally, the second determining unit includes:

[0054] a detection unit, configured to perform key point detection on the original license plate area image corresponding to the first determination result, and determine the key points in the original license plate area image corresponding to the first determination result;

[0055] The third determination unit is used to perform an affine transformation on the original license plate area image corresponding to the first determination result based on the key points in the original license plate area image corresponding to the first determination result to obtain a corrected license plate area image, and the corrected license plate area image is the license plate area image to be identified.

[0056] Optionally, the detection unit is specifically used to:

[0057] The original license plate area image corresponding to the first judgment result is input into a key point detection model, and the key point detection model outputs the key points in the original license plate area image corresponding to the first judgment result. The key point detection mode is obtained by training using the training license plate image as a training sample and the key points annotated with the training license plate image as sample labels.

[0058] Optionally, the license plate recognition model includes an image feature extraction module, a license plate color and license plate row number recognition module, an image feature optimization module and a license plate character recognition module;

[0059] The license plate recognition model recognizes the license plate area image to be recognized, and obtains the license plate color, license plate row number and license plate characters corresponding to the license plate area image to be recognized, including:

[0060] The image feature extraction module extracts features from the image of the license plate area to be identified, and obtains an initial feature vector of the image of the license plate area to be identified;

[0061] The license plate color and license plate row number recognition module determines the license plate color and the license plate row number based on the initial feature vector;

[0062] The image feature optimization module optimizes the initial feature vector to obtain a final feature vector of the image of the license plate area to be identified;

[0063] The license plate character recognition module determines the license plate character based on the final feature vector.

[0064] Optionally, the image feature optimization module optimizes the initial feature vector to obtain a final feature vector of the image of the license plate area to be identified, including:

[0065] Converting the initial feature vector into a first feature vector, wherein the length of the first feature vector is half of the length of the initial feature vector and the width is the same as the width of the initial feature vector;

[0066] The first eigenvector is equally divided in width to obtain a second eigenvector and a third eigenvector; the width of the second eigenvector and the third eigenvector are both half of the width of the initial eigenvector;

[0067] The second eigenvector and the third eigenvector are concatenated in length to obtain a fourth eigenvector, wherein the length of the fourth eigenvector is the same as that of the initial eigenvector, and the width is half of the width of the initial eigenvector;

[0068] Converting the initial eigenvector into a fifth eigenvector, wherein the length of the fifth eigenvector is half of the length of the initial eigenvector and the width is half of the width of the initial eigenvector;

[0069] The fourth eigenvector and the fifth eigenvector are concatenated in length to obtain a final eigenvector of the image of the license plate area to be identified.

[0070] A license plate recognition device, comprising a memory and a processor;

[0071] The memory is used to store programs;

[0072] The processor is used to execute the program to implement the various steps of the license plate recognition method described above.

[0073] A readable storage medium stores a computer program, which, when executed by a processor, implements the various steps of the license plate recognition method described above.

[0074] By means of the above technical scheme, the present application discloses a license plate recognition method, related equipment and readable storage medium. First, a training license plate area image including a single-line license plate area image and a double-line license plate area image is used as a training sample, and the license plate color, number of license plate rows and license plate characters annotated by the training license plate area image are used as sample labels for training to obtain a license plate recognition model. After determining the license plate area image to be recognized, the license plate area image to be recognized is input into the license plate recognition model, and the license plate recognition model recognizes the license plate area image to be recognized, and the license plate color, number of license plate rows and license plate characters corresponding to the license plate area image to be recognized can be obtained. This scheme can be applied to both single-line license plate recognition and double-line license plate recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0076] Figure 1 A flowchart of a license plate recognition method disclosed in an embodiment of the present application;

[0077] Figure 2 A schematic diagram of the structure of the license plate recognition model disclosed in the embodiment of the present application;

[0078] Figure 3 A schematic diagram of optimizing the initial feature vector disclosed in the embodiment of the present application to obtain the final feature vector;

[0079] Figure 4 A schematic diagram of the structure of a license plate recognition device disclosed in an embodiment of the present application;

[0080] Figure 5 This is a hardware structure block diagram of a license plate recognition device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0081] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0082] Next, the license plate recognition method provided by the present application is introduced through the following embodiments.

[0083] Reference Figure 1 , Figure 1 The following is a flow chart of a license plate recognition method disclosed in an embodiment of the present application. The method may include:

[0084] Step S101: Determine the license plate area image to be recognized.

[0085] In the present application, the license plate area image to be identified may be a single-row license plate area image or a double-row license plate area image, and the present application does not impose any limitation on this.

[0086] The license plate area image to be identified can be the original license plate area image provided by the user, or can be obtained by processing the original license plate area image provided by the user. This application does not impose any limitation on this. The specific method of determining the license plate area image to be identified will be described in detail through the following embodiments and will not be described in detail here.

[0087] Step S102: Input the license plate area image to be identified into a license plate recognition model, and the license plate recognition model identifies the license plate area image to be identified to obtain the license plate color, license plate row number and license plate character corresponding to the license plate area image to be identified; the license plate recognition model is trained using the training license plate area image as a training sample, and the license plate color, license plate row number and license plate character annotated with the training license plate area image as sample labels, and the training license plate area image includes a single-row license plate area image and a double-row license plate area image.

[0088] In the present application, the license plate area image for training of the license plate recognition model can be the original image or the image after preprocessing (scaling, cropping, subtracting the mean, etc.). The license plate color, number of license plate rows and license plate characters annotated on the training license plate area image can be in different label forms, and the present application does not impose any restrictions on this.

[0089] The structure of the license plate recognition model can adopt a variety of neural network structures, such as residual neural network (resnet18), convolutional neural network, etc., and this application does not impose any restrictions on this. The specific structure of the license plate recognition model will be described in detail through the following embodiments and will not be described here.

[0090] During the training process of the license plate recognition model, the training is performed with the joint loss of the three labels of license plate color, number of license plate rows and license plate characters. As an implementable method, the loss of the two labels of license plate color and number of license plate rows can use Softmax Loss, and the loss of the license plate character label can use CTC Loss. The loss of the license plate recognition model is a combination of the three losses. The losses of the two labels of license plate color and number of license plate rows each account for 0.1 of the total loss weight, and the loss of the license plate character label accounts for 0.8 of the total loss weight.

[0091] This embodiment discloses a license plate recognition method. First, a training license plate area image including a single-line license plate area image and a double-line license plate area image is used as a training sample, and the license plate color, number of license plate rows and license plate characters annotated in the training license plate area image are used as sample labels for training to obtain a license plate recognition model. After determining the license plate area image to be recognized, the license plate area image to be recognized is input into the license plate recognition model, and the license plate recognition model recognizes the license plate area image to be recognized, and the license plate color, number of license plate rows and license plate characters corresponding to the license plate area image to be recognized can be obtained. This solution can be applied to both single-line license plate recognition and double-line license plate recognition.

[0092] In another embodiment of the present application, a specific implementation method of determining the license plate area image to be recognized in step S101 is described, and the method may include the following steps:

[0093] Step S201: Acquire an original image provided by a user.

[0094] In the present application, the original image provided by the user may be an image taken by a camera device without any processing or after only simple pre-processing (such as scaling, cropping, mean subtraction, etc.).

[0095] Step S202: Detect the original image to obtain the position information of the license plate in the original image.

[0096] In the present application, a license plate detection model can be obtained by pre-training with a training image as a training sample and the position information of the license plate annotated in the training image as a sample label. Then, the original image is detected to obtain the position information of the license plate in the original image, which may include: inputting the original image into the license plate detection model, and the license plate detection model outputting the position information of the license plate in the original image.

[0097] It should be noted that the training images of the license plate detection model come from a large number of vehicle images collected from video surveillance. The types of vehicles mainly include: cars, trucks, passenger cars, vans, etc.; the license plates cover keywords of various provinces and cities across the country, and include embassy special vehicles, military license plates, civil aviation vehicles, etc. The license plate colors cover all colors seen on the market: blue, yellow, green, white, black, yellow-green, etc.; the number of license plate rows includes single row and double row. The position information of the license plate annotated in the training image can be the coordinates of any two diagonal vertices or all vertices in the upper left vertex, lower left vertex, upper right vertex and lower right vertex of the license plate area.

[0098] It should be noted that the license plate detection model can adopt a variety of neural network structures, such as the SSD (Single Shot MultiBox Detector) network.

[0099] Step S203: cropping the original image according to the position information of the license plate to obtain at least one original license plate area image.

[0100] Step S204: Determine a license plate area image to be recognized based on the at least one original license plate area image.

[0101] As an implementable method, the original license plate area image can be used as the license plate area image to be identified. However, the original license plate area image may be detected by mistake and does not contain the license plate. In this case, if subsequent identification is performed, the license plate recognition result may be incorrect.

[0102] In order to avoid this problem, in the present application, as another possible implementation method, based on the at least one original license plate area image, determining the license plate area image to be identified may include:

[0103] Step S301: for each original license plate area image, the original license plate area image is judged to obtain a first judgment result or a second judgment result.

[0104] The first determination result is used to indicate that the image is a license plate image; the second determination result is used to indicate that the image is not a license plate image.

[0105] In the present application, a license plate determination model can be obtained by pre-training with training images as training samples and using the first determination result or the second determination result annotated with the training images as sample labels. Then, determining the original license plate area image can include: inputting the original license plate area image into the license plate determination model, and the license plate determination model outputting the determination result corresponding to the original license plate area image.

[0106] It should be noted that the training images of the license plate recognition model come from a large number of vehicle images collected from video surveillance. Positive samples are various types of license plate data, and negative samples are a large number of scene areas, vehicle parts, and character plates in the image. The first judgment result or the second judgment result annotated with the training image can be in different label forms. For example, the label of the first judgment result is 1, and the label of the second judgment result can be 0. The training images can be preprocessed before training (such as scaling, cropping, subtracting the mean, etc.).

[0107] It should be noted that the license plate recognition model can adopt a variety of neural network structures, such as resnet14 network.

[0108] Step S302: determining a license plate area image to be identified based on the original license plate area image corresponding to the first determination result.

[0109] As an implementation method, the original license plate area image corresponding to the first determination result can be used to determine the license plate area image to be recognized. However, although an image is determined to be a license plate image, the license plate in the image may be tilted due to the shooting angle, which may cause the license plate recognition result to be incorrect.

[0110] In order to avoid this problem, in the present application, as another possible implementation method, the determining of the license plate area image to be identified based on the original license plate area image corresponding to the first determination result may include:

[0111] Step S401: performing key point detection on the original license plate area image corresponding to the first determination result to determine the key points in the original license plate area image corresponding to the first determination result.

[0112] In the present application, a key point detection model can be trained in advance using a training license plate image as a training sample and the key points annotated in the training license plate image as sample labels. Then, key point detection is performed on the original license plate area image corresponding to the first judgment result to determine the key points in the original license plate area image corresponding to the first judgment result, including: inputting the original license plate area image corresponding to the first judgment result into the key point detection model, and the key point detection model outputs the key points in the original license plate area image corresponding to the first judgment result.

[0113] It should be noted that the license plate images used for training the key point detection model come from a large number of vehicle images collected from video surveillance. The key points annotated in the training license plate images can be the four vertices of the license plate. The training images can be preprocessed before training (e.g., scaling, cropping, mean subtraction, etc.).

[0114] It should be noted that the key point detection model can adopt a variety of neural network structures, such as resnet14 network.

[0115] Step S402: Based on the key points in the original license plate area image corresponding to the first determination result, an affine transformation is performed on the original license plate area image corresponding to the first determination result to obtain a corrected license plate area image, and the corrected license plate area image is the license plate area image to be identified.

[0116] In another embodiment of the present application, the specific structure of the license plate recognition model is described. Figure 2 , Figure 2 The structure diagram of the license plate recognition model disclosed in the embodiment of the present application, the license plate recognition model includes an image feature extraction module, a license plate color and license plate row number recognition module, an image feature optimization module and a license plate character recognition module;

[0117] The license plate recognition model recognizes the license plate area image to be recognized, and obtains the license plate color, license plate row number and license plate characters corresponding to the license plate area image to be recognized, including:

[0118] The image feature extraction module extracts features from the image of the license plate area to be identified, and obtains an initial feature vector of the image of the license plate area to be identified;

[0119] The license plate color and license plate row number recognition module determines the license plate color and the license plate row number based on the initial feature vector;

[0120] The image feature optimization module optimizes the initial feature vector to obtain a final feature vector of the image of the license plate area to be identified;

[0121] The license plate character recognition module determines the license plate character based on the final feature vector.

[0122] In this embodiment, since the recognition complexity of license plate color and license plate row number is lower than that of license plate characters, the recognition of license plate color and license plate row number, as well as the recognition of license plate characters are divided into two branches. The recognition results of license plate color and license plate row number are obtained in advance at the front part of the license plate recognition network, and the back part of the license plate recognition network recognizes the license plate characters, which can improve the recognition efficiency.

[0123] In another embodiment of the present application, a specific implementation method of the image feature optimization module optimizing the initial feature vector to obtain the final feature vector of the image of the license plate area to be identified is described, and the method may include:

[0124] Converting the initial feature vector into a first feature vector, wherein the length of the first feature vector is half of the length of the initial feature vector and the width is the same as the width of the initial feature vector;

[0125] The first eigenvector is equally divided in width to obtain a second eigenvector and a third eigenvector; the width of the second eigenvector and the third eigenvector are both half of the width of the initial eigenvector;

[0126] The second eigenvector and the third eigenvector are concatenated in length to obtain a fourth eigenvector, wherein the length of the fourth eigenvector is the same as that of the initial eigenvector, and the width is half of the width of the initial eigenvector;

[0127] Converting the initial eigenvector into a fifth eigenvector, wherein the length of the fifth eigenvector is half of the length of the initial eigenvector and the width is half of the width of the initial eigenvector;

[0128] The fourth eigenvector and the fifth eigenvector are concatenated in length to obtain a final eigenvector of the image of the license plate area to be identified.

[0129] It should be noted that when segmenting the feature vector, the convolution layer can be used to avoid the problem of unsupported model quantization.

[0130] For ease of understanding, refer to Figure 3 , Figure 3 Schematic diagram of optimizing the initial feature vector disclosed in the embodiment of this application to obtain the final feature vector, such as Figure 3 As shown, the initial feature vector is N*C*H*W, where N is the number of batches, C is the number of channels, H is the feature vector width, W is the feature vector length, the first feature vector is N*C*H*(W / 2), the second and third feature vectors are N*C*(H / 2)*(W / 2), the fourth feature vector is N*C*(H / 2)*W, the fifth feature vector is N*C*(H / 2)*(W / 2), and the final feature vector is N*C*(H / 2)*(W / 2*3).

[0131] The license plate recognition device disclosed in the embodiment of the present application is described below. The license plate recognition device described below and the license plate recognition method described above can be referenced to each other.

[0132] Reference Figure 4 , Figure 4 Schematic diagram of the structure of a license plate recognition device disclosed in the embodiment of this application. Figure 4 As shown, the license plate recognition device may include:

[0133] A determination unit 11, used to determine a license plate area image to be identified;

[0134] The license plate recognition unit 12 is used to input the license plate area image to be recognized into a license plate recognition model, and the license plate recognition model recognizes the license plate area image to be recognized to obtain the license plate color, license plate row number and license plate character corresponding to the license plate area image to be recognized; the license plate recognition model is trained with the training license plate area image as a training sample, and the license plate color, license plate row number and license plate character annotated with the training license plate area image as sample labels, and the training license plate area image includes a single-row license plate area image and a double-row license plate area image.

[0135] As an implementable embodiment, the determining unit includes:

[0136] An original image acquisition unit, used to acquire an original image provided by a user;

[0137] A detection unit, used to detect the original image to obtain the position information of the license plate in the original image;

[0138] A cropping unit, used for cropping the original image according to the position information of the license plate to obtain at least one original license plate area image;

[0139] The first determining unit is used to determine the license plate area image to be recognized based on the at least one original license plate area image.

[0140] As an implementable embodiment, the detection unit is specifically used for:

[0141] The original image is input into a license plate detection model, and the license plate detection model outputs the position information of the license plate in the original image; the license plate detection model is trained using the training image as a training sample and the position information of the license plate annotated in the training image as a sample label.

[0142] As an implementable embodiment, the first determining unit includes:

[0143] A determination unit, for determining each original license plate area image, to obtain a first determination result or a second determination result, wherein the first determination result is used to indicate that the image is a license plate image; and the second determination result is used to indicate that the image is not a license plate image;

[0144] The second determining unit is used to determine the license plate area image to be identified based on the original license plate area image corresponding to the first determination result.

[0145] As an implementable embodiment, the determination unit is specifically used for:

[0146] The original license plate area image is input into a license plate determination model, and the license plate determination model outputs a determination result corresponding to the original license plate area image; the license plate determination model is trained using training images as training samples and using the first determination result or the second determination result annotated with the training images as sample labels.

[0147] As an implementable embodiment, the second determining unit includes:

[0148] a detection unit, configured to perform key point detection on the original license plate area image corresponding to the first determination result, and determine the key points in the original license plate area image corresponding to the first determination result;

[0149] The third determination unit is used to perform an affine transformation on the original license plate area image corresponding to the first determination result based on the key points in the original license plate area image corresponding to the first determination result to obtain a corrected license plate area image, and the corrected license plate area image is the license plate area image to be identified.

[0150] As an implementable embodiment, the detection unit is specifically used for:

[0151] The original license plate area image corresponding to the first judgment result is input into a key point detection model, and the key point detection model outputs the key points in the original license plate area image corresponding to the first judgment result. The key point detection mode is obtained by training using the training license plate image as a training sample and the key points annotated with the training license plate image as sample labels.

[0152] As an implementable embodiment, the license plate recognition model includes an image feature extraction module, a license plate color and license plate row number recognition module, an image feature optimization module and a license plate character recognition module;

[0153] The license plate recognition model recognizes the license plate area image to be recognized, and obtains the license plate color, license plate row number and license plate characters corresponding to the license plate area image to be recognized, including:

[0154] The image feature extraction module extracts features from the image of the license plate area to be identified, and obtains an initial feature vector of the image of the license plate area to be identified;

[0155] The license plate color and license plate row number recognition module determines the license plate color and the license plate row number based on the initial feature vector;

[0156] The image feature optimization module optimizes the initial feature vector to obtain a final feature vector of the image of the license plate area to be identified;

[0157] The license plate character recognition module determines the license plate character based on the final feature vector.

[0158] As an implementable method, the image feature optimization module optimizes the initial feature vector to obtain the final feature vector of the image of the license plate area to be identified, including:

[0159] Converting the initial feature vector into a first feature vector, wherein the length of the first feature vector is half of the length of the initial feature vector and the width is the same as the width of the initial feature vector;

[0160] The first eigenvector is equally divided in width to obtain a second eigenvector and a third eigenvector; the width of the second eigenvector and the third eigenvector are both half of the width of the initial eigenvector;

[0161] The second eigenvector and the third eigenvector are concatenated in length to obtain a fourth eigenvector, wherein the length of the fourth eigenvector is the same as that of the initial eigenvector, and the width is half of the width of the initial eigenvector;

[0162] Converting the initial eigenvector into a fifth eigenvector, wherein the length of the fifth eigenvector is half of the length of the initial eigenvector and the width is half of the width of the initial eigenvector;

[0163] The fourth eigenvector and the fifth eigenvector are concatenated in length to obtain a final eigenvector of the image of the license plate area to be identified.

[0164] Reference Figure 5 , Figure 5 The hardware structure diagram of the license plate recognition device provided in the embodiment of the present application is shown in FIG. Figure 5 ,The hardware structure of the license plate recognition device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;

[0165] In the embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4;

[0166] The processor 1 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0167] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0168] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0169] Determine the license plate area image to be recognized;

[0170] The license plate area image to be identified is input into a license plate recognition model, and the license plate recognition model identifies the license plate area image to be identified to obtain the license plate color, license plate row number and license plate character corresponding to the license plate area image to be identified; the license plate recognition model is trained using training license plate area images as training samples and the license plate color, license plate row number and license plate character annotated with the training license plate area images as sample labels, and the training license plate area images include single-row license plate area images and double-row license plate area images.

[0171] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0172] The embodiment of the present application further provides a readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0173] Determine the license plate area image to be recognized;

[0174] The license plate area image to be identified is input into a license plate recognition model, and the license plate recognition model identifies the license plate area image to be identified to obtain the license plate color, license plate row number and license plate character corresponding to the license plate area image to be identified; the license plate recognition model is trained using training license plate area images as training samples and the license plate color, license plate row number and license plate character annotated with the training license plate area images as sample labels, and the training license plate area images include single-row license plate area images and double-row license plate area images.

[0175] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0176] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0177] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0178] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A license plate recognition method, characterized in that: The method comprises: Determine the license plate area image to be recognized; The license plate area image to be identified is input into a license plate recognition model, and the license plate recognition model identifies the license plate area image to be identified to obtain the license plate color, license plate row number and license plate character corresponding to the license plate area image to be identified; the license plate recognition model is trained by using the training license plate area image as a training sample and the license plate color, license plate row number and license plate character annotated by the training license plate area image as sample labels, and the training license plate area image includes a single-row license plate area image and a double-row license plate area image; The license plate recognition model includes an image feature extraction module, a license plate color and license plate row number recognition module, an image feature optimization module and a license plate character recognition module; The license plate recognition model recognizes the license plate area image to be recognized, and obtains the license plate color, license plate row number and license plate characters corresponding to the license plate area image to be recognized, including: The image feature extraction module extracts features from the image of the license plate area to be identified, and obtains an initial feature vector of the image of the license plate area to be identified; The license plate color and license plate row number recognition module determines the license plate color and the license plate row number based on the initial feature vector; The image feature optimization module optimizes the initial feature vector to obtain a final feature vector of the image of the license plate area to be identified; The license plate character recognition module determines the license plate character based on the final feature vector; The image feature optimization module optimizes the initial feature vector to obtain the final feature vector of the image of the license plate area to be identified, including: Converting the initial feature vector into a first feature vector, wherein the length of the first feature vector is half of the length of the initial feature vector and the width is the same as the width of the initial feature vector; The first eigenvector is equally divided in width to obtain a second eigenvector and a third eigenvector; the width of the second eigenvector and the third eigenvector are both half of the width of the initial eigenvector; The second eigenvector and the third eigenvector are concatenated in length to obtain a fourth eigenvector, wherein the length of the fourth eigenvector is the same as that of the initial eigenvector, and the width is half of the width of the initial eigenvector; Converting the initial eigenvector into a fifth eigenvector, wherein the length of the fifth eigenvector is half of the length of the initial eigenvector and the width is half of the width of the initial eigenvector; The fourth eigenvector and the fifth eigenvector are concatenated in length to obtain a final eigenvector of the image of the license plate area to be identified.

2. The method according to claim 1, characterized in that The step of determining the license plate area image to be identified includes: Get the original image provided by the user; Detecting the original image to obtain location information of the license plate in the original image; Cutting the original image according to the position information of the license plate to obtain at least one original license plate area image; Based on the at least one original license plate area image, a license plate area image to be recognized is determined.

3. The method according to claim 2, characterized in that The detecting the original image to obtain the position information of the license plate in the original image includes: The original image is input into a license plate detection model, and the license plate detection model outputs the position information of the license plate in the original image; the license plate detection model is trained using the training image as a training sample and the position information of the license plate annotated in the training image as a sample label.

4. The method according to claim 2, characterized in that: The determining of the license plate area image to be identified based on the at least one original license plate area image comprises: For each original license plate area image, the original license plate area image is judged to obtain a first judgment result or a second judgment result, wherein the first judgment result is used to indicate that the image is a license plate image; and the second judgment result is used to indicate that the image is not a license plate image; Based on the original license plate area image corresponding to the first determination result, a license plate area image to be identified is determined.

5. The method according to claim 4, characterized in that The determining of the original license plate area image includes: The original license plate area image is input into a license plate determination model, and the license plate determination model outputs a determination result corresponding to the original license plate area image; the license plate determination model is trained using training images as training samples and using the first determination result or the second determination result annotated with the training images as sample labels.

6. The method according to claim 4, characterized in that The determining of the license plate area image to be identified based on the original license plate area image corresponding to the first determination result includes: Performing key point detection on the original license plate area image corresponding to the first determination result to determine the key points in the original license plate area image corresponding to the first determination result; Based on the key points in the original license plate area image corresponding to the first determination result, an affine transformation is performed on the original license plate area image corresponding to the first determination result to obtain a corrected license plate area image, and the corrected license plate area image is the license plate area image to be identified.

7. The method according to claim 6, characterized in that The performing key point detection on the original license plate area image corresponding to the first determination result to determine the key points in the original license plate area image corresponding to the first determination result includes: The original license plate area image corresponding to the first judgment result is input into a key point detection model, and the key point detection model outputs the key points in the original license plate area image corresponding to the first judgment result. The key point detection mode is obtained by training using the training license plate image as a training sample and the key points annotated with the training license plate image as sample labels.

8. A license plate recognition device, characterized in that: The device comprises: A determination unit, used to determine the license plate area image to be identified; A license plate recognition unit, used for inputting the license plate area image to be recognized into a license plate recognition model, wherein the license plate recognition model recognizes the license plate area image to be recognized and obtains the license plate color, number of license plate rows and license plate characters corresponding to the license plate area image to be recognized; the license plate recognition model is trained by using the training license plate area image as a training sample and the license plate color, number of license plate rows and license plate characters annotated by the training license plate area image as sample labels, and the training license plate area image includes a single-row license plate area image and a double-row license plate area image; The license plate recognition model includes an image feature extraction module, a license plate color and license plate row number recognition module, an image feature optimization module and a license plate character recognition module; The license plate recognition model recognizes the license plate area image to be recognized, and obtains the license plate color, license plate row number and license plate characters corresponding to the license plate area image to be recognized, including: The image feature extraction module extracts features from the image of the license plate area to be identified, and obtains an initial feature vector of the image of the license plate area to be identified; The license plate color and license plate row number recognition module determines the license plate color and the license plate row number based on the initial feature vector; The image feature optimization module optimizes the initial feature vector to obtain a final feature vector of the image of the license plate area to be identified; The license plate character recognition module determines the license plate character based on the final feature vector; The image feature optimization module optimizes the initial feature vector to obtain the final feature vector of the image of the license plate area to be identified, including: Converting the initial feature vector into a first feature vector, wherein the length of the first feature vector is half of the length of the initial feature vector and the width is the same as the width of the initial feature vector; The first eigenvector is equally divided in width to obtain a second eigenvector and a third eigenvector; the width of the second eigenvector and the third eigenvector are both half of the width of the initial eigenvector; The second eigenvector and the third eigenvector are concatenated in length to obtain a fourth eigenvector, wherein the length of the fourth eigenvector is the same as that of the initial eigenvector, and the width is half of the width of the initial eigenvector; Converting the initial eigenvector into a fifth eigenvector, wherein the length of the fifth eigenvector is half of the length of the initial eigenvector and the width is half of the width of the initial eigenvector; The fourth eigenvector and the fifth eigenvector are concatenated in length to obtain a final eigenvector of the image of the license plate area to be identified.

9. A license plate recognition device, characterized in that: including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the license plate recognition method as described in any one of claims 1 to 7.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the license plate recognition method according to any one of claims 1 to 7 is implemented.

Images