License plate number recognition method, device, equipment and readable storage medium

Abstract

The present invention provides a license plate number recognition method, device, equipment and readable storage medium. The method comprises: preprocessing a license plate image to be recognized to obtain a first image; performing affine transformation on the first image to obtain a second image; extracting features from the second image to obtain a first image feature map; performing dimensionality reduction processing on the first image feature map to obtain a second image feature map; inputting the second image feature map into multiple classifiers respectively, and obtaining the license plate number according to the output results of the multiple classifiers, wherein different classifiers are used to predict characters in different character areas in the second image feature map. Through the present invention, the accuracy of license plate number recognition is improved; and different types of license plate numbers can be recognized through the same process.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a license plate number recognition method, device, equipment and readable storage medium. Background Art

[0002] License plate number recognition technology is a technology that automatically captures license plate images from complex backgrounds and identifies license plate numbers. This technology has been widely used in parking lot import and export management systems, highway vehicle management systems and other systems.

[0003] The common license plate number recognition technology is to train a neural network model through a sample license plate image, and then recognize the license plate number in the license plate image based on the trained neural network model. This method requires training different neural network models for different types of license plates (such as license plates of variable length, single-row license plates, and double-row license plates) to recognize different types of license plates respectively, and the process is cumbersome and complicated. At the same time, when shooting the license plate image, the license plate image obtained has an angle offset due to the influence of the shooting angle, which affects the final recognition effect. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a license plate number recognition method, device, equipment and readable storage medium.

[0005] In a first aspect, the present invention provides a method for recognizing a license plate number, the method comprising:

[0006] Preprocessing the license plate image to be recognized to obtain a first image;

[0007] Performing an affine transformation on the first image to obtain a second image;

[0008] Performing feature extraction on the second image to obtain a feature map of the first image;

[0009] Performing dimensionality reduction processing on the first image feature map to obtain a second image feature map;

[0010] The second image feature map is input into a plurality of classifiers respectively, and the license plate number is obtained according to the output results of the plurality of classifiers, wherein different classifiers are used to predict characters in different character areas in the second image feature map.

[0011] Optionally, the step of preprocessing the license plate image to be recognized to obtain the first image includes:

[0012] The size of the license plate image to be identified is scaled to a preset size, and the pixel value of the license plate image to be identified is normalized to a preset interval to obtain a first image.

[0013] Optionally, the step of performing affine transformation on the first image to obtain the second image includes:

[0014] Inputting the first image into the image space transformation network to obtain an affine transformation matrix output by the image space transformation network;

[0015] Performing affine transformation on the first image according to the affine transformation matrix to obtain a second image.

[0016] Optionally, the step of extracting features from the second image to obtain a feature map of the first image includes:

[0017] The second image is input into the feature extraction network to obtain a feature map of the first image output by the feature extraction network.

[0018] Optionally, the step of inputting the second image feature map into a plurality of classifiers respectively and obtaining the license plate number according to the output results of the plurality of classifiers comprises:

[0019] Input the second image feature map into the nth classifier to obtain the nth vector output by the nth classifier for the nth character of the license plate number in the license plate image to be recognized, where the initial value of n is 1;

[0020] Determine the prediction result of the nth character according to the nth vector;

[0021] Detecting whether the character prediction result is a placeholder character;

[0022] If it is a placeholder character, concatenate the prediction results of each character that is not a placeholder character to get the license plate number;

[0023] If it is not a placeholder character, determine whether n is equal to N, where N is the number of classifiers;

[0024] If n is not equal to N, n+1 is used as the new n, and the process returns to input the second image feature map into the nth classifier.

[0025] Optionally, the step of determining the prediction result of the nth character according to the nth vector includes:

[0026] Find the maximum element value in the nth vector;

[0027] The character corresponding to the maximum element value is used as the nth character prediction result.

[0028] Optionally, before the step of preprocessing the license plate image to be recognized to obtain the first image, the step further includes:

[0029] Read training images from the training dataset;

[0030] Preprocessing the training image to obtain a first training image;

[0031] Inputting the first training picture into the image space conversion network, and obtaining an affine transformation matrix corresponding to the first training picture output by the image space conversion network;

[0032] Performing an affine transformation on the first training picture according to the affine transformation matrix corresponding to the first training picture to obtain a second training picture;

[0033] Inputting the second training image into the feature extraction network to obtain a feature map of the first training image output by the feature extraction network;

[0034] Performing dimensionality reduction processing on the first training image feature map to obtain a second training image feature map;

[0035] Inputting the second training image feature map into multiple classifiers respectively, and obtaining the predicted license plate number according to the output results of the multiple classifiers;

[0036] Calculate the loss value based on the predicted license plate number and the actual license plate number corresponding to the training image;

[0037] Update the parameters of the image space conversion network, feature extraction network and multiple classifiers according to the loss value;

[0038] Return to the step of reading training images from the training dataset until the stopping condition is met.

[0039] In a second aspect, the present invention further provides a vehicle license plate number recognition device, the vehicle license plate number recognition device comprising:

[0040] A preprocessing module, used for preprocessing the license plate image to be recognized to obtain a first image;

[0041] An affine transformation module, used for performing affine transformation on the first image to obtain a second image;

[0042] A feature extraction module, used to extract features from the second image to obtain a feature map of the first image;

[0043] A dimensionality reduction processing module, used for performing dimensionality reduction processing on the first image feature map to obtain a second image feature map;

[0044] The recognition module is used to input the second image feature map into multiple classifiers respectively, and obtain the license plate number according to the output results of the multiple classifiers, wherein different classifiers are used to predict characters in different character areas in the second image feature map.

[0045] In a third aspect, the present invention also provides a license plate number recognition device, comprising a processor, a memory, and a license plate number recognition program stored in the memory and executable by the processor, wherein when the license plate number recognition program is executed by the processor, the steps of the license plate number recognition method as described above are implemented.

[0046] In a fourth aspect, the present invention further provides a readable storage medium, on which a license plate number recognition program is stored, wherein when the license plate number recognition program is executed by a processor, the steps of the license plate number recognition method as described above are implemented.

[0047] In the present invention, the license plate image to be identified is preprocessed to obtain a first image; the first image is affine transformed to obtain a second image; the second image is feature extracted to obtain a first image feature map; the first image feature map is dimensionality reduced to obtain a second image feature map; the second image feature map is input into a plurality of classifiers respectively, and the license plate number is obtained according to the output results of the plurality of classifiers, wherein different classifiers are used to predict the characters in different character areas in the second image feature map. Through the present invention, the preprocessed license plate image to be identified is affine transformed before the license plate number is identified, so that the angle of the image is corrected, and the accuracy of the license plate number recognition is improved; and the characters in different character areas in the second image feature map are predicted respectively by different classifiers. On this basis, as long as a reasonable number of classifiers are pre-set, different types of license plate numbers can be identified through the same process. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of the hardware structure of a license plate number recognition device involved in an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of a process flow of a method for recognizing a license plate number according to an embodiment of the present invention;

[0050] Figure 3 It is a structural diagram of an image space conversion network in an embodiment of a license plate number recognition method of the present invention;

[0051] Figure 4 A schematic diagram of a scene in which affine transformation is performed in an embodiment of a method for recognizing a license plate number according to the present invention;

[0052] Figure 5 It is a structural schematic diagram of a feature extraction network in an embodiment of a license plate number recognition method of the present invention;

[0053] Figure 6 It is a schematic diagram of the structure of the Bottleneck layer in an embodiment of the vehicle license plate number recognition method of the present invention;

[0054] Figure 7 for Figure 2 A detailed flow chart of step S50;

[0055] Figure 8A schematic diagram of a process of obtaining a license plate number based on a classifier in an embodiment of a license plate number recognition method of the present invention;

[0056] Fig. 9 Schematic diagram of functional modules of an embodiment of a vehicle license plate number recognition device of the present invention.

[0057] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0059] In a first aspect, an embodiment of the present invention provides a license plate number recognition device, which may be a device having a data processing function, such as a personal computer (PC), a notebook computer, or a server.

[0060] Reference Figure 1 , Figure 1 The hardware structure diagram of the license plate number recognition device involved in the embodiment of the present invention. In the embodiment of the present invention, the license plate number recognition device may include a processor 1001 (e.g., a central processing unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components; the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard); the network interface 1004 may optionally include a standard wired interface, a wireless interface (such as a wireless fidelity WIreless-FIdelity, WI-FI interface); the memory 1005 may be a high-speed random access memory (random access memory, RAM), or a stable memory (non-volatile memory), such as a disk storage, and the memory 1005 may optionally be a storage device independent of the aforementioned processor 1001. Those skilled in the art will understand that Figure 1 The hardware structure shown in the figure does not constitute a limitation of the present invention, and may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0061] Continue to refer to Figure 1 , Figure 1The memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a license plate number recognition program. The processor 1001 may call the license plate number recognition program stored in the memory 1005 and execute the license plate number recognition method provided in the embodiment of the present invention.

[0062] In a second aspect, an embodiment of the present invention provides a method for recognizing a license plate number.

[0063] In one embodiment, referring to Figure 2 , Figure 2 FIG. 1 is a flow chart of an embodiment of a method for recognizing a license plate number according to the present invention. Figure 2 As shown, the license plate number recognition method includes:

[0064] Step S10, preprocessing the license plate image to be recognized to obtain a first image;

[0065] In this embodiment, the license plate image to be recognized is preprocessed to obtain a first image. The preprocessing may include image scaling and pixel value normalization.

[0066] Furthermore, in one embodiment, step S10 includes:

[0067] The size of the license plate image to be identified is scaled to a preset size, and the pixel value of the license plate image to be identified is normalized to a preset interval to obtain a first image.

[0068] In this embodiment, the size of the license plate image to be identified is scaled to a preset size, such as 48*96, where 48 represents the height of the image and 96 represents the width of the image; and the pixel value of the license plate image to be identified is normalized to a preset interval, such as (-1,1). It should be noted that the values ​​here are only for illustrative purposes and do not constitute a limitation on the preset size and preset interval. The specific values ​​of the preset size and preset interval can be set according to actual needs.

[0069] Step S20, performing affine transformation on the first image to obtain a second image;

[0070] In this embodiment, in order to avoid the situation where the image of the license plate to be identified is offset in angle due to the shooting angle, thereby affecting the final recognition effect, it is necessary to perform an affine transformation on the first image to correct the angle of the first image and obtain the second image.

[0071] Furthermore, in one embodiment, step S20 includes:

[0072] Step S201, inputting the first image into an image space transformation network to obtain an affine transformation matrix output by the image space transformation network;

[0073] In this embodiment, the first image is input into the image space transformation network, thereby obtaining an affine transformation matrix corresponding to the first image output by the first image input image space transformation network. Figure 3 , Figure 3 This is a schematic diagram of the structure of the image space conversion network in an embodiment of the license plate number recognition method of the present invention. Figure 3 As shown in the figure, the image space conversion network contains two convolutional layers (conv), two maximum pooling layers (MaxPool), two fully connected layers (FC), and several activation layers (ReLU). Figure 3 "3*48*96" means that the first input image is 3-channel data of size 48*96; "5*5conv,32" means the convolution kernel size is 5*5 and there are 32 channels of the convolution layer; "2*2MaxPool" means the maximum value pooling layer with a pooling kernel of 2*2; "ReLU" means the activation layer with the activation function of ReLU (Rectified LinearUnit); "FC,32" means the fully connected layer with 32 output nodes; "FC,6" means the fully connected layer with 6 output nodes.

[0074] Finally, the 6 values ​​output by the fully connected layer with 6 output nodes are rearranged into a 2*3 matrix to obtain the affine transformation matrix M of the first image.

[0075] Step S202: Perform affine transformation on the first image according to the affine transformation matrix to obtain a second image.

[0076] In this embodiment, based on the affine transformation matrix obtained in step S201, the first image is affine transformed by the following formula:

[0077]

[0078] Where (x, y) represents the coordinates on the first image, (u, v) represents the coordinates of (x, y) after affine transformation, That is, based on the affine transformation matrix M obtained in step S201.

[0079] Reference Figure 4 , Figure 4 FIG. 1 is a schematic diagram of a scene in which an affine transformation is performed in an embodiment of a method for recognizing a license plate number according to the present invention. Figure 4 As shown, the first image is input into STNet (image space transformation network) to obtain the affine transformation matrix of the first image output by STNet; then the first image is affine transformed by the affine transformation matrix to obtain the second image, thereby realizing the angle correction of the first image.

[0080] Step S30, extracting features from the second image to obtain a feature map of the first image;

[0081] In this embodiment, the second image may be subjected to feature extraction through a feature extraction network to obtain a feature map of the first image, wherein the feature extraction network may be designed according to actual needs.

[0082] Furthermore, in one embodiment, step S30 includes:

[0083] The second image is input into the feature extraction network to obtain a feature map of the first image output by the feature extraction network.

[0084] In this embodiment, the feature extraction network is designed with reference to the MobileFaceNet network structure. Figure 5 , Figure 5 Schematic diagram of the structure of the feature extraction network in the first embodiment of the license plate number recognition method of the present invention. Figure 5 As shown, the feature extraction network includes:

[0085] Conv1 layer: This layer is the first neural network combination layer of the feature extraction network. The input is the RGB image (second image) data of size 3*48*96. "3*3" represents the size of the convolution kernel, "64" represents the number of channels in the output layer, "s=2" represents the convolution step size of 2, and "p=1" represents the padding factor of 1 pixel around the input in the convolution operation. The convolution layer is followed by the BatchNorm layer ( Figure 5 "bn" in the middle) and Relu layer ( Figure 5 in Chinese).

[0086] dw_conv1 layer: takes the output of the above "conv1" layer as input, and contains a depthwise separable convolution layer with a kernel size of 3*3, an output layer with the number of channels of "64", a convolution step size of s=1, a padding factor of p=1, and is also followed by a BatchNorm layer ( Figure 5 "bn" in the middle) and Relu layer ( Figure 5 in Chinese).

[0087] Block module: After the above dw_conv1 layer, there are several block modules. The block module is composed of multiple Bottleneck layers. The parameters that control the block module are t, c, n, and s. Among them, t represents the channel expansion coefficient, c represents the number of output channels of the module, n represents the number of Bottleneck layers, and s represents the convolution step size. By controlling these four parameters, different block modules can be obtained. Figure 5As shown, the feature extraction network contains a total of 5 block modules, namely Figure 5 block1, block2, block3, block4, block5 in it.

[0088] After the above five block modules, the next layer is the conv2 layer. The conv2 layer is first a convolution layer with a convolution kernel size of 1*1, an output channel number of 512, a convolution step size of s=2, and a padding factor of p=0. The convolution layer is followed by the BatchNorm layer ( Figure 5 "bn" in the middle) and Relu layer ( Figure 5 The final output is a feature map of size 512*6*12 (the first image feature map).

[0089] Reference Figure 6 , Figure 6 This is a schematic diagram of the structure of the Bottleneck layer in an embodiment of the license plate number recognition method of the present invention. Figure 6 As shown in the figure, the Bottleneck layer is the basic building block of the block module. Different block modules are formed by stacking a corresponding number of Bottleneck layers according to the block parameter n. The parameters of the Bottleneck layer are controlled by the corresponding block module. Depending on the convolution step size parameter s, the structure of the corresponding Bottleneck layer is also different.

[0090] When s = 2, the structure of the Bottleneck layer is as follows Figure 6 As shown on the left, the input data passes through the conv1 layer, which has a convolution layer. "1*1" indicates the size of the convolution kernel, and "oup*t" indicates the number of output channels of the convolution layer, where "oup" and "t" are determined by the corresponding block module. Oup is equal to c in the block module parameters, and t is equal to the expansion coefficient parameter t in the block model. "s=1" indicates that the convolution step size of the convolution layer is 1, and "p=0" indicates that the filling factor of the convolution is 0. The convolution layer is followed by a BatchNorm layer ("bn") and a Relu layer ("relu"). The conv1 layer is followed by a dw_conv1 layer, which contains a depthwise separable convolution layer with a convolution kernel size of 3*3, an output channel number of oup*t, a convolution step size of s=2, and a filling factor of p=1. It is also followed by a BatchNorm layer ("bn") and a Relu layer ("relu"). The dw_conv1 layer is followed by the conv2 layer, which contains a convolutional layer with a kernel size of 1*1, the number of output channels is oup, the convolution step size is s=1, the padding factor is p=0, and is followed by a BatchNorm layer ("bn").

[0091] When s=1, the structure of the Bottleneck layer is as follows Figure 6 As shown on the right, the difference between it and the left figure is that in the dw_conv1 layer, the convolution step size s = 1, and an Add layer is added. The Add layer adds the corresponding elements of the two input feature maps (FeatureMap), such as Figure 6 As shown, the initial input and the corresponding elements of the output of the conv2 layer are added as the final output.

[0092] It should be noted that the above is only a schematic description of the feature extraction network and does not constitute a limitation on the feature extraction network. For example, the feature extraction network can also be designed with reference to the network structures of ShuffleNet, ResNet, VGG, etc.

[0093] Step S40, performing dimensionality reduction processing on the first image feature map to obtain a second image feature map;

[0094] In this embodiment, the first image feature map can be subjected to dimensionality reduction processing through a global average pooling layer. For example, the size of the first image feature map is 512*6*12, and after the dimensionality reduction processing, a second image feature map of a size of 512*1*1 is obtained.

[0095] Step S50, inputting the second image feature map into a plurality of classifiers respectively, and obtaining the license plate number according to the output results of the plurality of classifiers, wherein different classifiers are used to predict characters in different character areas in the second image feature map.

[0096] In this embodiment, there are multiple classifiers, and the specific number is set according to actual needs. For example, if a general license plate number includes 7 to 8 characters, 8 or more classifiers can be set. The second image feature map is input into multiple classifiers respectively to obtain the output result of each classifier. Different classifiers are used to predict characters in different character areas in the second image feature map, that is, the output results of different classifiers are the prediction results for different characters. Combining the output results of all classifiers, the license plate number corresponding to the license plate image to be identified can be obtained.

[0097] Further, in one embodiment, referring to Figure 7 , Figure 7 for Figure 2 Detailed flow chart of step S50 in FIG. Figure 7 As shown, step S50 includes:

[0098] Step S501, inputting the second image feature map into the nth classifier, obtaining the nth vector output by the nth classifier for the nth character of the license plate number in the license plate image to be recognized, where the initial value of n is 1;

[0099] Step S502, determining the prediction result of the nth character according to the nth vector;

[0100] Step S503, detecting whether the character prediction result is a placeholder character;

[0101] Step S504, if it is a placeholder character, then concatenate the prediction results of each character that is not a placeholder character to obtain the license plate number;

[0102] Step S505, if it is not a placeholder character, determine whether n is equal to N, where N is the number of classifiers;

[0103] Step S506, if n is not equal to N, n+1 is used as the new n, and the process returns to input the second image feature map into the nth classifier.

[0104] In this embodiment, the second image feature map is first input into the first classifier to obtain the first vector output by the first classifier for the first character of the license plate number in the license plate image to be identified; the first character prediction result is determined according to the first vector, and if the first character prediction result is not a placeholder character, it is determined whether 1 is equal to N, where N is the number of classifiers; if not, the second image feature map is input into the second classifier, and the above steps are repeated until the nth character prediction result is a placeholder character, indicating that all characters contained in the license plate number have been identified. For example, if the 8th character prediction result is detected as a placeholder character, the 1st to 7th character prediction results are spliced ​​in the order of the 1st character prediction result, the 2nd character prediction result ... the 7th character prediction result to obtain the license plate number.

[0105] Furthermore, if n is equal to N, it means that the number of characters in the license plate number corresponding to the license plate image to be identified is greater than or equal to the number of classifiers. At this time, it is impossible to determine whether the license plate number corresponding to the 1st to nth character prediction results is the license plate number corresponding to the license plate image to be identified, so the process ends directly. Of course, a reminder of recognition error can also be output.

[0106] Furthermore, in one embodiment, step S502 includes:

[0107] Find the maximum element value in the nth vector;

[0108] The character corresponding to the maximum element value is used as the nth character prediction result.

[0109] In this embodiment, each classifier includes a convolution layer, and the output dimension is a vector of 75*1*1, where 75 represents a total of 75 different character categories, including Chinese characters, English letters, numbers and placeholders in license plate numbers. The size of each element in the vector output by the classifier represents the probability of the corresponding character. Therefore, the maximum element value in the nth vector is found, and the character corresponding to the maximum element value is used as the prediction result of the nth character. It should be noted that "75" is only for schematic illustration and is not used as a limitation on the output dimension of the classifier.

[0110] Reference Figure 8 , Figure 8 FIG. 1 is a flow chart of obtaining a license plate number based on a classifier in an embodiment of a license plate number recognition method of the present invention. Figure 8 As shown, the first image feature map of size 512*6*12 is subjected to dimensionality reduction processing through the global average pooling layer to obtain a second image feature map of size 512*1*1; then the second image feature map is input into each classifier respectively, each classifier includes a convolution layer (such as classifier 1 includes conv1, classifier 2 includes conv2, and so on), and the output dimension is 75*1*1 vector; finally, the output of each classifier is input into the post-processing decoding module to obtain the license plate number corresponding to the license plate image to be identified. Specifically, the post-processing decoding module determines the character prediction result according to the maximum element value in each input vector, and then sequentially splices the character prediction results of each non-placeholder character to obtain the license plate number corresponding to the license plate image to be identified.

[0111] In this embodiment, the license plate image to be identified is preprocessed to obtain a first image; the first image is affine transformed to obtain a second image; the second image is feature extracted to obtain a first image feature map; the first image feature map is dimensionality reduced to obtain a second image feature map; the second image feature map is input into multiple classifiers respectively, and the license plate number is obtained according to the output results of the multiple classifiers, wherein different classifiers are used to predict the characters in different character areas in the second image feature map. Through this embodiment, the preprocessed license plate image to be identified is affine transformed before the license plate number is recognized, so that the angle of the image is corrected, and the accuracy of license plate number recognition is improved; and different classifiers are used to predict the characters in different character areas in the second image feature map respectively. On this basis, as long as a reasonable number of classifiers are set in advance, different types of license plate numbers can be recognized through the same process.

[0112] Furthermore, in one embodiment, before step S10, the method further includes:

[0113] Step S01, reading training images from a training data set;

[0114] Step S02, preprocessing the training image to obtain a first training image;

[0115] Step S03, inputting the first training picture into the image space conversion network, and obtaining an affine transformation matrix corresponding to the first training picture output by the image space conversion network;

[0116] Step S04, performing an affine transformation on the first training picture according to the affine transformation matrix corresponding to the first training picture to obtain a second training picture;

[0117] Step S05, inputting the second training image into the feature extraction network to obtain a first training image feature map output by the feature extraction network;

[0118] Step S06, performing dimensionality reduction processing on the first training image feature map to obtain a second training image feature map;

[0119] Step S07, inputting the second training image feature map into a plurality of classifiers respectively, and obtaining a predicted license plate number according to the output results of the plurality of classifiers;

[0120] In this embodiment, the training data set contains various types of license plate images, such as single-layer / double-layer license plates and license plates of different colors. Several license plate images are randomly read from the training data set as training images. Thus, the image space conversion network, feature extraction network and multiple classifiers finally trained can be applied to recognize various types of license plates.

[0121] The specific embodiments of steps S02 to S07 refer to the embodiments of steps S10 to S50 above, which will not be described in detail here.

[0122] Step S08, calculating the loss value according to the predicted license plate number and the actual license plate number corresponding to the training image;

[0123] Step S09, updating the parameters of the image space conversion network, the feature extraction network and the multiple classifiers according to the loss value;

[0124] In this embodiment, the loss function selects the cross entropy loss function, and its calculation formula is as follows:

[0125]

[0126] Among them, L represents the loss value, M represents the number of training images, and y m represents the output value of the classifier for the mth character in the training image, x mrepresents the true label of the mth character, where different classifiers correspond to different characters, for example, the first classifier corresponds to the first character in the training image, the second classifier corresponds to the second character in the training image, and so on. In this way, the loss value corresponding to each classifier can be calculated, and then the loss value corresponding to each classifier is accumulated, and then the parameters of the image space conversion network, feature extraction network and multiple classifiers are updated based on the accumulated value.

[0127] Return to step S01 until the stop condition is met.

[0128] In this embodiment, if the stopping condition is not met, steps S01 to S09 are repeated until the stopping condition is met. The stopping condition may be that the number of iterations reaches a preset number or the loss function converges.

[0129] In a third aspect, an embodiment of the present invention further provides a license plate number recognition device.

[0130] In one embodiment, referring to Fig. 9 , Fig. 9 Schematic diagram of the functional modules of an embodiment of the vehicle license plate number recognition device of the present invention. Fig. 9 As shown, the license plate number recognition device includes:

[0131] A preprocessing module 10 is used to preprocess the license plate image to be recognized to obtain a first image;

[0132] An affine transformation module 20, configured to perform an affine transformation on the first image to obtain a second image;

[0133] A feature extraction module 30 is used to extract features from the second image to obtain a feature map of the first image;

[0134] A dimensionality reduction processing module 40 is used to perform dimensionality reduction processing on the first image feature map to obtain a second image feature map;

[0135] The recognition module 50 is used to input the second image feature map into multiple classifiers respectively, and obtain the license plate number according to the output results of the multiple classifiers, wherein different classifiers are used to predict the characters in different character areas in the second image feature map.

[0136] Furthermore, in one embodiment, the preprocessing module 10 is used to:

[0137] The size of the license plate image to be identified is scaled to a preset size, and the pixel value of the license plate image to be identified is normalized to a preset interval to obtain a first image.

[0138] Furthermore, in one embodiment, the affine transformation module 20 is used to:

[0139] Inputting the first image into the image space transformation network to obtain an affine transformation matrix output by the image space transformation network;

[0140] Performing affine transformation on the first image according to the affine transformation matrix to obtain a second image.

[0141] Furthermore, in one embodiment, the feature extraction module 30 is used to:

[0142] The second image is input into the feature extraction network to obtain a feature map of the first image output by the feature extraction network.

[0143] Furthermore, in one embodiment, the identification module 50 is used to:

[0144] Input the second image feature map into the nth classifier to obtain the nth vector output by the nth classifier for the nth character of the license plate number in the license plate image to be recognized, where the initial value of n is 1;

[0145] Determine the prediction result of the nth character according to the nth vector;

[0146] Detecting whether the character prediction result is a placeholder character;

[0147] If it is a placeholder character, concatenate the prediction results of each character that is not a placeholder character to get the license plate number;

[0148] If it is not a placeholder character, determine whether n is equal to N, where N is the number of classifiers;

[0149] If n is not equal to N, n+1 is used as the new n, and the process returns to input the second image feature map into the nth classifier.

[0150] Furthermore, in one embodiment, the identification module 50 is used to:

[0151] Find the maximum element value in the nth vector;

[0152] The character corresponding to the maximum element value is used as the nth character prediction result.

[0153] Furthermore, in one embodiment, the license plate number recognition device further includes a training module for:

[0154] Read training images from the training dataset;

[0155] Preprocessing the training image to obtain a first training image;

[0156] Inputting the first training picture into the image space conversion network, and obtaining an affine transformation matrix corresponding to the first training picture output by the image space conversion network;

[0157] Performing an affine transformation on the first training picture according to the affine transformation matrix corresponding to the first training picture to obtain a second training picture;

[0158] Inputting the second training image into the feature extraction network to obtain a feature map of the first training image output by the feature extraction network;

[0159] Performing dimensionality reduction processing on the first training image feature map to obtain a second training image feature map;

[0160] Inputting the second training image feature map into multiple classifiers respectively, and obtaining the predicted license plate number according to the output results of the multiple classifiers;

[0161] Calculate the loss value based on the predicted license plate number and the actual license plate number corresponding to the training image;

[0162] Update the parameters of the image space conversion network, feature extraction network and multiple classifiers according to the loss value;

[0163] Return to the step of reading training images from the training dataset until the stopping condition is met.

[0164] Among them, the functional implementation of each module in the above-mentioned license plate number recognition device corresponds to the various steps in the above-mentioned license plate number recognition method embodiment, and its functions and implementation processes will not be repeated here one by one.

[0165] In a fourth aspect, an embodiment of the present invention further provides a readable storage medium.

[0166] The readable storage medium of the present invention stores a license plate number recognition program, wherein when the license plate number recognition program is executed by a processor, the steps of the license plate number recognition method as described above are implemented.

[0167] Among them, the method implemented when the license plate number recognition program is executed can refer to the various embodiments of the license plate number recognition method of the present invention, and will not be repeated here.

[0168] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0169] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0170] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present invention.

[0171] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for recognizing a license plate number, characterized in that: The license plate number recognition method comprises: Preprocessing the license plate image to be recognized to obtain a first image; Performing an affine transformation on the first image to obtain a second image; Performing feature extraction on the second image to obtain a feature map of the first image; Performing dimensionality reduction processing on the first image feature map to obtain a second image feature map; Inputting the second image feature map into a plurality of classifiers respectively, and obtaining the license plate number according to the output results of the plurality of classifiers, wherein different classifiers are used to predict characters in different character areas in the second image feature map; The step of inputting the second image feature map into a plurality of classifiers respectively and obtaining the license plate number according to the output results of the plurality of classifiers comprises: Input the second image feature map into the nth classifier to obtain the nth vector output by the nth classifier for the nth character of the license plate number in the license plate image to be recognized, where the initial value of n is 1; Determine the prediction result of the nth character according to the nth vector; Detecting whether the character prediction result is a placeholder character; If it is a placeholder character, concatenate the prediction results of each character that is not a placeholder character to get the license plate number; If it is not a placeholder character, determine whether n is equal to N, where N is the number of classifiers; If n is not equal to N, n+1 is used as the new n, and the process returns to input the second image feature map into the nth classifier.

2. The vehicle license plate number recognition method according to claim 1, characterized in that: The step of preprocessing the license plate image to be recognized to obtain the first image comprises: The size of the license plate image to be identified is scaled to a preset size, and the pixel value of the license plate image to be identified is normalized to a preset interval to obtain a first image.

3. The license plate number recognition method according to claim 1, characterized in that: The step of performing affine transformation on the first image to obtain the second image comprises: Inputting the first image into the image space transformation network to obtain an affine transformation matrix output by the image space transformation network; Performing affine transformation on the first image according to the affine transformation matrix to obtain a second image.

4. The license plate number recognition method according to claim 1, characterized in that: The step of extracting features from the second image to obtain a feature map of the first image comprises: The second image is input into the feature extraction network to obtain a feature map of the first image output by the feature extraction network.

5. The vehicle license plate number recognition method according to claim 1, characterized in that: The step of determining the prediction result of the nth character according to the nth vector comprises: Find the maximum element value in the nth vector; The character corresponding to the maximum element value is used as the nth character prediction result.

6. The vehicle license plate number recognition method according to any one of claims 1 to 5, characterized in that: Before the step of preprocessing the license plate image to be recognized to obtain the first image, the method further includes: Read training images from the training dataset; Preprocessing the training image to obtain a first training image; Inputting the first training picture into the image space conversion network, and obtaining an affine transformation matrix corresponding to the first training picture output by the image space conversion network; Performing an affine transformation on the first training picture according to the affine transformation matrix corresponding to the first training picture to obtain a second training picture; Inputting the second training image into the feature extraction network to obtain a feature map of the first training image output by the feature extraction network; Performing dimensionality reduction processing on the first training image feature map to obtain a second training image feature map; Inputting the second training image feature map into multiple classifiers respectively, and obtaining the predicted license plate number according to the output results of the multiple classifiers; Calculate the loss value based on the predicted license plate number and the actual license plate number corresponding to the training image; Update the parameters of the image space conversion network, feature extraction network and multiple classifiers according to the loss value; Return to the step of reading training images from the training dataset until the stopping condition is met.

7. A vehicle license plate number recognition device, characterized in that: The vehicle license plate number recognition device comprises: A preprocessing module, used for preprocessing the license plate image to be recognized to obtain a first image; An affine transformation module, used for performing affine transformation on the first image to obtain a second image; A feature extraction module, used to extract features from the second image to obtain a feature map of the first image; A dimensionality reduction processing module, used for performing dimensionality reduction processing on the first image feature map to obtain a second image feature map; A recognition module, used for inputting the second image feature map into the nth classifier to obtain the nth vector output by the nth classifier for the nth character of the license plate number in the license plate image to be recognized, where the initial value of n is 1; Determine the prediction result of the nth character according to the nth vector; Detecting whether the character prediction result is a placeholder character; If it is a placeholder character, concatenate the prediction results of each character that is not a placeholder character to get the license plate number; If it is not a placeholder character, determine whether n is equal to N, where N is the number of classifiers; If n is not equal to N, n+1 is used as the new n, and the process returns to input the second image feature map into the nth classifier, wherein different classifiers are used to predict characters in different character areas in the second image feature map.

8. A vehicle license plate number recognition device, characterized in that: The license plate number recognition device includes a processor, a memory, and a license plate number recognition program stored in the memory and executable by the processor, wherein when the license plate number recognition program is executed by the processor, the steps of the license plate number recognition method as described in any one of claims 1 to 6 are implemented.

9. A readable storage medium, characterized in that: The readable storage medium stores a license plate number recognition program, wherein when the license plate number recognition program is executed by a processor, the steps of the license plate number recognition method according to any one of claims 1 to 6 are implemented.

Images