A character verification code recognition method based on improved LeNet-5 network

Abstract

A character verification code recognition method based on an improved LeNet‑5 network. It mainly solves the problems of complex network models and large amount of computation in the current deep learning method in the field of character verification code recognition. First, the input layer is set to 160*60 pixels to adapt to the size of the input character verification code image; secondly, the hollow convolution is used in the convolution layer instead of the standard convolution to obtain a larger receptive field; then, we introduce the BN layer to receive the input of each convolution layer, and use the BN layer to normalize the received parameters to improve the network performance; in addition, the Softmax cross entropy loss function is introduced to further optimize the network. The recognition method designed by the present invention has good effect, high test recognition accuracy and short training time.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a character verification code recognition method based on an improved LeNet-5 network. Background Art

[0002] There are many types of verification codes, the most common of which is the character verification code. The character verification code is an image composed of a group of mixed letters, numbers, and symbols. Users need to enter the verification code in the input box to prove that they are real users, not computer programs. Because the character verification code is simple to make and easy to maintain, it has become the most commonly used verification code among many types of verification codes.

[0003] There are two main types of character verification code recognition methods: segmentation-based character verification code recognition methods and non-segmentation-based character verification code recognition methods. In the early stage of character verification code recognition research, verification code recognition methods were generally based on the algorithm idea of ​​segmentation first and then recognition. The characteristics of the character verification code image at this stage are that there are fewer noise points in the image, almost no background interference, and the characters are relatively simple, lacking character adhesion and character distortion. Therefore, in 2007, Jeff Yan cracked this character verification code by calculating the pixels in the character verification code image, and nearly 100% of the characters were cracked. After that, the designers of the verification code redesigned the verification code image and added interference information to the verification code image. However, in the following year, Jeff Yan used the projection segmentation algorithm to segment this type of character verification code, and the segmentation accuracy reached 90%, and the final recognition accuracy reached 60%. After that, in order to enhance the security of the character verification code, the verification code designer performed many complex transformations on the characters in the character verification code image. However, for such complex verification codes, researchers tried new cracking ideas. Chandavale et al. segmented the verification code according to the character features in the verification code image, and took the verification code of EZ-Gimpy as an example for testing, and their recognition accuracy rate reached 80%.

[0004] With the development of character verification code recognition technology, more and more character verification code recognition algorithms are applied to the recognition of character verification codes. Gao's team used a segmentation-based character verification code recognition algorithm to crack 11 types of online verification codes. In addition, Dazhen Lin et al. used a non-segmentation-based character verification code recognition algorithm and used a convolutional neural network to recognize Chinese character verification codes. By improving the network structure, they learned the Chinese characters in the character verification code, broadening the research field of character verification code recognition. Yu Peng et al. introduced the AlexNet network to address the character adhesion in the character verification code, performed multi-task learning on the character verification code, and achieved good results. Afterwards, Yang Zi et al. designed an end-to-end character verification code recognition algorithm to recognize the verification code as a whole, reducing the preprocessing steps, and still achieved good results.

[0005] In 2020, Cao Tingrong and others used the knowledge of neural networks to further study the verification code recognition method and proposed a new verification code recognition algorithm. In the preprocessing stage of the character image, the algorithm uses the trained PixPix network to process, and then uses a multi-channel convolutional neural network to recognize the character verification code. After that, YaoWang and others were inspired by the verification code recognition method proposed by Cao Tingrong and designed a method based on a generative adversarial network. This method can train the model with a small number of samples, and has a fast processing speed and strong generalization ability. It solves the shortcomings of the current character verification code recognition method with low recognition accuracy and the need for a large number of training samples. In the same year, Li Hao and others designed a lightweight neural network to address the model complexity problem caused by too many neural network layers. The network has a good recognition effect on character verification code images with more background noise interference. Chunhui Li et al. designed an end-to-end recognition method for character CAPTCHAs based on a cycle-consistency generative adversarial network (Cycle-GAN). The recognition method uses Cycle-GAN to generate fake samples of CAPTCHA images, then uses fake samples to train the CAPTCHA recognizer, and finally uses active transfer learning to optimize the CAPTCHA recognizer using a small number of labeled real verification samples. The algorithm analyzes the most popular anti-recognition mechanisms and effectively cracks the CAPTCHAs of 10 mainstream websites.

[0006] In short, the research process of character verification code is a process of continuous development and evolution. With the continuous advancement of technology and the continuous game between the attacker and the defender, the security of character verification code is also constantly improving. In the future, with the continuous advancement of deep learning, the recognition research of character verification code will be further advanced and the application scenarios will be more extensive. Summary of the invention

[0007] In order to overcome the shortcomings of the background technology, the present invention provides a character verification code recognition method based on an improved LeNet-5 network, which mainly solves the problems of complex network model and large amount of calculation existing in the field of character verification code recognition for the current deep learning method.

[0008] The technical solution adopted by the present invention is:

[0009] A character verification code recognition method based on an improved LeNet-5 network comprises the following steps:

[0010] Step 1: Generate a data set using the Python third-party library Captcha. The character content of the character verification code image includes pure numbers, pure letters, and mixed numbers and English letters. It is divided into training set, test set, and validation set in a ratio of 8:1:1;

[0011] Step 2: Build a model based on the LeNet-5 network deep learning network, with an input size of 160*60 pixels in the input layer; insert dilated convolutions with dilation rates of 1, 2, and 5 in the three convolution layers C1, C2, and C3; add BN (Batch Normalization) layers after the C1 and C3 layers respectively;

[0012] Step 4: Model training: start model training based on the organized training set, validation set and constructed model;

[0013] Step 5: Finally, use the trained model for recognition.

[0014] In the step three, the character verification code is also identified by using a Softmax cross entropy loss function; the formula of the Softmax cross entropy loss function is as follows:

[0015]

[0016] Among them, y is the true category label, f(x) is the predicted value of the model, and C is the number of categories.

[0017] Before using the Softmax cross entropy loss function to recognize the character verification code, the verification code characters are one-hot encoded.

[0018] The size of the hollow convolution kernel in the three convolutional layers is 3×3, the filter in the pooling layer is 2×2, and the maximum pooling with a stride of 2. There is a BN layer connected between each convolutional layer and each pooling layer.

[0019] The algorithm of the BN layer is as follows;

[0020] Input: a batch of data x1... m, and learnable parameters γ, β;

[0021] 1. Calculate the mean of the input data

[0022] 2. Calculate the variance of the input data

[0023] 3. Sample data standardization

[0024] 4. Training parameters

[0025] 5. Output: y i , where γ is the weight and β is the bias.

[0026] The beneficial effects of the present invention are: the recognition method designed by the present invention has good effect, high test recognition accuracy and short training time. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of dilated convolution with dilation rates of 1, 2, and 5.

[0028] Figure 2 Schematic diagram of the calculation of the BN layer of two pictures.

[0029] Figure 3 This is the model structure diagram after adding the BN layer.

[0030] Figure 4 This is a network model diagram.

[0031] Figure 5 The following is a graph showing the accuracy of the model training and testing on dataset A.

[0032] Figure 6 The following is a graph showing the accuracy of the model training and testing on dataset A.

[0033] Figure 7 The following is a graph of the training and test loss results of the model on dataset B.

[0034] Figure 8 This is the training and testing accuracy result diagram of the model on dataset B.

[0035] Fig. 9 The following is a graph of the training and testing loss results of the model on dataset C.

[0036] Fig.10 is the training and testing accuracy of the model on dataset C. DETAILED DESCRIPTION

[0037] The present invention is further described below in conjunction with the accompanying drawings: As shown in the figure, a character verification code recognition method based on an improved LeNet-5 network comprises the following steps:

[0038] Step 1: Generate a data set using the Python third-party library Captcha. The character content of the character verification code image includes pure numbers, pure letters, and mixed numbers and English letters. It is divided into training set, test set, and validation set in a ratio of 8:1:1;

[0039] Step 2: Build a model based on the LeNet-5 network deep learning network, with an input size of 160*60 pixels in the input layer; insert dilated convolutions with dilation rates of 1, 2, and 5 in the three convolution layers C1, C2, and C3; add BN (Batch Normalization) layers after the C1 and C3 layers respectively;

[0040] Step 4: Model training: start model training based on the organized training set, validation set and constructed model;

[0041] Step 5: Finally, use the trained model for recognition.

[0042] In the step three, the character verification code is also identified by using a Softmax cross entropy loss function; the formula of the Softmax cross entropy loss function is as follows:

[0043]

[0044] Among them, y is the true category label, f(x) is the predicted value of the model, and C is the number of categories.

[0045] Before using the Softmax cross entropy loss function to recognize the character verification code, the verification code characters are one-hot encoded.

[0046] The size of the hollow convolution kernel in the three convolutional layers is 3×3, the filter in the pooling layer is 2×2, and the maximum pooling with a stride of 2. There is a BN layer connected between each convolutional layer and each pooling layer.

[0047] The algorithm of the BN layer is as follows;

[0048] Input: a batch of data x1... m , and learnable parameters γ, β;

[0049] 1. Calculate the mean of the input data

[0050] 2. Calculate the variance of the input data

[0051] 3. Sample data standardization

[0052] 4. Training parameters

[0053] 5. Output: y i , where γ is the weight and β is the bias.

[0054] The basic content of this application is to improve the LeNet-5 network. First, we set the input layer to 160*60 pixels to adapt to the size of the input character verification code image; secondly, we use hollow convolution instead of standard convolution in the convolution layer to obtain a larger receptive field; then, we introduce the BN layer to receive the input of each convolution layer, and use the BN layer to normalize the received parameters to improve network performance; in addition, we introduce the Softmax cross entropy loss function to further optimize the network.

[0055] The details are as follows:

[0056] (1) Improvements to the input layer and convolutional layer

[0057] Since the classic LeNet-5 network only has a good effect in recognizing a single character data set, we need to optimize and improve the LeNet-5 network. First, we improve the input layer and convolution layer. The improvement strategy is as follows:

[0058] 1) Change the input layer input size to 160*60 pixels

[0059] Change the input layer input size to 160*60 pixels. Since the verification code image contains multiple characters, the 32*32 pixel size of the input layer in the LeNet-5 network cannot effectively recognize the verification code character image. Therefore, we change the input layer input size to 160*60 pixels, which can better meet the aspect ratio of the character verification code image and avoid excessive information loss when normalized to 32*32 size.

[0060] 2) Introduction of dilated convolution

[0061] In this invention, we use dilated convolution to replace the standard convolution layer in the LeNet-5 network because in the traditional convolution operation, each element of the convolution kernel will be convolved with each element of the input feature map to generate the output feature map. In the dilated convolution, due to the existence of the dilation rate, we can have a certain interval between the elements of the convolution kernel, so that each element is convolved with a certain number of elements in the input feature map. In this way, we can increase the receptive field of the convolution layer without increasing the number of model parameters, thereby improving the performance of the model.

[0062] In the network of the present invention, we insert the following into the three convolutional layers C1, C2, and C3: Figure 1 The dilation rates shown are 1, 2, and 5 respectively. This preserves the complete continuous area and ensures the coherence of the receptive field, avoiding the discontinuity problem of the convolution kernel when the same dilation rate is used.

[0063] The introduction of dilated convolution in the present invention can make our model have the following main advantages. First, dilated convolution introduces some additional holes in the convolution kernel, so that the receptive field of the convolution kernel is expanded. In this way, the receptive field of the network can be increased without increasing the size of the convolution kernel and the number of parameters, thereby improving the recognition accuracy of the model designed by the present invention. Secondly, dilated convolution can perform convolution operations at different hole scales, thereby realizing multi-scale feature extraction. In this way, the network's recognition ability for character verification code images of different scales can be improved. Finally, dilated convolution can increase the receptive field size without losing spatial information. In this way, while maintaining the resolution of the feature map, the network's perception of global information can be improved, thereby improving the performance of the network.

[0064] (2) Introduction of BN layer

[0065] After improving the input layer and convolution layer of the LeNet-5 network, we introduced the BN layer and added the Batch Normalization (BN) layer after the C1 and C3 layers respectively. In the neural network, except for the output layer, the other layers will cause changes in the distribution of input data of subsequent layers during the training process because the parameters of the lower-level network are updated. In order to solve this problem, the present invention inserts the BN layer after the convolution layer of the network when designing the network model. The main principle of the BN layer is to standardize the input of each layer in the network to reduce the internal covariate shift problem in the network. The internal covariate shift problem refers to the distribution of the input of each layer in the network constantly changing with the training of the network, causing the output distribution of the network's activation function to constantly change, thereby making the training of the network difficult. The algorithm implementation process of the BN layer is as follows.

[0066] Input: a batch of data x1...m , and learnable parameters γ, β;

[0067] 1. Calculate the mean of the input data

[0068] 2. Calculate the variance of the input data

[0069] 3. Sample data standardization

[0070] 4. Training parameters

[0071] 5. Output: y i , where γ is the weight and β is the bias.

[0072] In the above, the sample mean and sample variance are first calculated, then the sample data is normalized, and finally the parameters γ and β are trained, where γ is the weight and β is the bias. The output is transformed linearly, and finally the data samples are normalized. Figure 2 The calculation process of the BN layer of two pictures is given.

[0073] exist Figure 2 In the figure, we show the BN layer calculation process of two pictures. Assume that Feature1 and Feature2 are the feature maps of the two pictures after convolution, and the number of image channels is 1. Then, the mean and variance of the data in all feature maps are calculated, and finally the eigenvalue of each channel is calculated according to the standard deviation formula.

[0074] In our neural network training process, it will take a long time to improve the training speed of the model. The introduction of the BN layer can make the input distribution of each layer more stable, thereby accelerating the training speed of the neural network. In addition, the introduction of the BN layer can also reduce the network model's dependence on hyperparameters, because the BN layer can reduce the sensitivity to hyperparameters such as learning rate and weight initialization, making it easier to adjust parameters and train neural networks. Therefore, when we improved the LeNet-5 network model, we added a BN layer after each convolutional layer to make the network model more efficient. The model structure diagram with the BN layer is shown in Figure 3.

[0075] (3) Construction of loss function

[0076] In the present invention, we use the Softmax cross entropy loss function to identify the character verification code. The meaning of the Softmax cross entropy loss function is to transform the predicted value through the Softmax function to obtain the probability distribution of each category, and then use the cross entropy loss function to measure the difference between the predicted label and the true label, so as to construct a single-label multi-classification form, that is, each sample can only belong to one class. The formula of the Softmax cross entropy loss function is as follows.

[0077]

[0078] Among them, y is the true category label, f(x) is the predicted value of the model, and C is the number of categories.

[0079] In the present invention's research on the recognition of verification codes, before using the loss function calculation, we first need to one-hot encode the verification code characters, because in the process of recognizing character verification codes by convolutional neural networks, one-hot encoding is a commonly used label encoding method, which is used to convert character labels into numerical forms that can be processed by neural networks. In the output of the network, one-hot encoding is also required to represent the predicted results of each character, so that during the training process, the loss function is used to calculate the gap between the predicted results and the true labels to adjust the weights and parameters of the network and improve the recognition accuracy.

[0080] In short, one-hot encoding means mapping each character label to a fixed-length vector, where the length of the vector is equal to the size of the character set. In the vector, only the element at the position corresponding to the character label is 1, and the elements at the other positions are all 0. Since the characters designed in this chapter are four-character verification codes, the corresponding one-hot in this chapter is 4 rows and 36 columns, where the first 10 positions of the one-hot encoding correspond to the numbers 0-9, and the next 26 positions correspond to the English letters a (A)-z (Z). The one-hot encoding of the characters in some character verification codes in this chapter is shown in Table 1.

[0081] Table 1 One-hot encoding examples of character verification codes

[0082]

[0083] Taking the number "2" in pure digital characters and the one-hot encoding length of 10 as an example, the one-hot encoding of the number "2" is [0, 0, 1, 0, 0, 0, 0, 0, 0], then after Softmax correctly predicts, the probability is [0.1, 0.05, 0.6, 0.0, 0.05, 0.1, 0.0, 0.1, 0.0, 0.0], it can be seen that the label probability corresponding to the character "2" is at most 0.6, then the calculated loss is 0.51, when the prediction is wrong, the probability of Softmax output is [0.1, 0.05, 0.1, 0.0, 0.05, 0.1, 0.0, 0.6, 0.0, 0.01], the label probability corresponding to the character "2" is 0.1, that is, the prediction is wrong, and the calculated loss is 2.3, therefore, when the model predicts wrong, the loss function value will be significantly larger. The calculation process of the Softmax loss cross entropy loss function in the present invention is as follows.

[0084]

[0085] In the learning process of the Softmax cross entropy loss function, the predicted value in the neural network is first sent to the Softmax function, and then the probability pi of each category is obtained, and then the probability of each category is calculated through the cross entropy loss function to calculate the model loss.

[0086] In the present invention, the input image size of the input layer is 160*60, the expansion rates of the hollow convolution kernels in the three convolution layers are 1, 2, and 5, respectively, and the size is 3×3. The filter in the pooling layer is 2×2, with a maximum pooling of 2 steps. There is a BN layer connection between each convolution layer and each pooling layer. In addition, the activation function is the Leaky ReLU activation function, the loss function is the Softmax loss cross entropy loss function, the learning rate is 0.01, and the network model is as follows: Figure 4 shown.

[0087] Data collection and screening:

[0088] Character verification codes are still the most commonly used verification codes on websites today. Research on the recognition of character verification codes can measure the quality of the generation algorithm of character verification codes and help improve the verification code generation algorithm to achieve the purpose of strengthening the security of network systems. In addition, character verification codes have more noise and strong interference. In addition to the common noise points, there are also many interference lines and background interference, which increases the difficulty of character recognition in character verification code images, so it is conducive to the emergence and application of other character recognition algorithms.

[0089] The data set used in the present invention is a character verification code with more noise. However, since there is no open source character verification code data set on the Internet, and the number of data sets required for neural network training and testing is large, the data sets used in the present invention are all generated from the Python third-party library Captcha. In the character verification code data set collected by the present invention, the character content of the character verification code image includes pure numbers, pure letters, and character verification code images mixed with numbers and English letters. Finally, the verification code data set is divided into three categories: A, B, and C. The image situation of each type of character verification code is shown in Table 2.

[0090] Table 2 Verification code for each data set

[0091]

[0092] The present invention measures the recognition effect of the model on the character verification code image by calculating the loss of the model and the recognition accuracy of the model. The calculation formula of the recognition accuracy is as follows.

[0093]

[0094] Among them, M is the number of samples in all test sets, Y i Represents the test sample n i ∈{n1,n2,...,n k}, G i Represents n i The set of true value labels.

[0095] Afterwards, we used the method designed by the present invention to conduct a recognition experiment on the character verification code, and divided each type of data set into a training set, a test set, and a verification set in a ratio of 8:1:1. First, we used these four network models to train and test data set A, and calculated the accuracy and loss of training and testing respectively. Finally, the training and testing losses and accuracy of data set A were calculated as follows: Figure 5 and 6 shown.

[0096] from Figure 6 It can be seen that the character verification code image content of dataset A is relatively simple, with only 0-9 numeric characters. Therefore, the result of testing dataset A is that the loss of the four models can be reduced to below 0.01 after 30 epochs. This also suggests that the four network models may be able to better match the actual output, and the predicted results may be more accurate. This can be seen from Figure 6 The accuracy of the model is well verified. Figure 6From the recognition test of dataset A, it can be found that although the recognition accuracy of the four network models can reach more than 99%, and they all have good recognition effects on dataset A, the recognition accuracy of the network model designed by the present invention reaches the highest 99.7% among the four models, which is 0.15, 0.06, and 0.05 percentage points higher than the LeNet-5, ResNet50, and DenseNet121 network models, respectively. In order to further test the advantages and disadvantages of the model designed by the present invention, after the experiment on dataset A, we use these four models to continue training and testing on dataset B, and calculate their losses and recognition accuracy. Finally, the training and testing losses and accuracy of the model designed by the present invention and the other three models for dataset B are as follows: Figure 7 and 8 shown.

[0097] from Figure 8 It can be seen that since the characters in data set B have more uppercase and lowercase English characters than those in data set A, that is, the characters in data set B are more diverse and more complex, the network model losses of LeNet-5 and ResNet50 dropped to nearly 0.01 after 30 epochs in the test of data set B, while the network model losses of DenseNet121 and the network model of the present invention dropped below 0.01, which indicates that the recognition effect of DenseNet121 and the network model of the present invention in data set B may be better than that of LeNet-5 and ResNet50. Afterwards, we calculated the recognition accuracy of data set B, from Fig. 9 It can be seen that for dataset B, the accuracy of the LeNet-5 network model is the worst. This may be because the LeNet-5 network was born too early. As the number of recognized characters increases, the recognition effect becomes worse and worse. The accuracy of the ResNet50 and DenseNet121 models is relatively close, reaching more than 98%, but the model in this invention has the best effect, and the test result accuracy reaches 99.1%. Finally, we use these four network models to train and test dataset C. The training and test losses and accuracy of dataset B are as follows: Fig. 9 and 10 shown.

[0098] from Fig. 9 It can be seen that for dataset C, after 30 epochs, the loss of LeNet-5, ResNet50, and DenseNet121 models dropped to nearly 0.01, while the loss of the model designed by the present invention dropped below 0.01, which shows that the model designed by the present invention has a better matching effect than other models. Fig.10The recognition accuracy of each model shows that for data set C, the accuracy of the LeNet-5 model is the worst, only 96.7, which means that when there are many types of character recognition, the simple LeNet-5 network can no longer achieve a good recognition effect. The accuracy of ResNet50, DenseNet121 and the model designed by the present invention can all reach more than 98%, among which the model designed by the present invention has the best recognition effect. In order to better see the advantages of the model, we compare the test recognition accuracy and training time of the four models as shown in Table 3.

[0099] Table 3 Comparison of model recognition accuracy and training time

[0100]

[0101]

[0102] The embodiments described with reference to the accompanying drawings are exemplary and intended to be used to explain the present invention, but should not be understood as limiting the present invention. The embodiments should not be regarded as limiting the present invention, but any improvements made based on the spirit of the present invention should be within the scope of protection of the present invention.

Claims

1. A character verification code recognition method based on an improved LeNet-5 network, characterized in that: The following steps are included: Step 1: Generate a data set using the Python third-party library Captcha. The character content of the character verification code image includes pure numbers, pure letters, and mixed numbers and English letters. It is divided into training set, test set, and validation set in a ratio of 8:1:1; Step 2: Build a model based on the LeNet-5 network deep learning network, with an input size of 160*60 pixels in the input layer; insert dilated convolutions with dilation rates of 1, 2, and 5 in the three convolution layers C1, C2, and C3; add BN (Batch Normalization) layers after the C1 and C3 layers respectively; Step 3: Model training: start model training based on the organized training set, validation set and constructed model; Step 4: Finally, use the trained model for recognition; In the step 2, the Softmax cross entropy loss function is also used to identify the character verification code; the formula of the Softmax cross entropy loss function is as follows: Among them, y is the true category label, f(x) is the predicted value of the model, and C is the number of categories; Before using the Softmax cross entropy loss function to recognize the character verification code, the verification code characters are one-hot encoded; The size of the hollow convolution kernel in the three convolutional layers is 3×3, the filter in the pooling layer is 2×2, and the maximum pooling with a stride of 2. There is a BN layer connected between each convolutional layer and each pooling layer.

2. A character verification code recognition method based on an improved LeNet-5 network according to claim 1, characterized in that: The algorithm of the BN layer is as follows; Input: a batch of data x1... m , and learnable parameters γ, β; 1. Calculate the mean of the input data 2. Calculate the variance of the input data 3. Sample data standardization 4. Training parameters 5. Output: y i , where γ is the weight and β is the bias.

Images