Captcha code recognition method and system based on improved YOLO9000 algorithm

Abstract

The application discloses a Captcha recognition method and system based on an improved YOLO9000 algorithm, belongs to the technical field of Captcha recognition, and aims to solve the technical problem of how to quickly and accurately realize Captcha recognition. The method comprises the following steps: collecting Captcha pictures, wherein Captchas in the Captcha pictures are composed of Chinese characters, English and numbers; labeling Chinese characters in the Captchas and performing image transformation processing on the Captcha pictures; for a YOLO9000 model, the convolution of a bottleneck structure is reduced, the convolution layer is improved, a cross-entropy loss function and an average difference similarity function are used as a loss function, and a Captcha recognition model is constructed based on the improved YOLO9000 model; a Captcha recognition model is constructed based on a DBN model; the YOLO model and the DBN model are trained based on the preprocessed training samples; and the classification results are fused by adopting a linear credibility accumulation mode.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of verification code recognition, in particular to a verification code recognition method and system based on an improved YOLO9000 algorithm. BACKGROUND

[0002] Verification code recognition is a hot issue in the field of optical character recognition, and research has been ongoing for more than half a century. Because the number of categories of verification codes with Chinese characters is large (there are 6763 categories of commonly used Chinese characters according to the GB2312-80 standard), the similarity between Chinese characters is large, and the accuracy of current verification code recognition is still small. In verification code recognition, the characters that appear are relatively fixed in the library, the range is relatively narrow, and it is relatively simple compared to general Chinese character recognition, and the number of Chinese character categories is small.

[0003] Because verification codes span English characters, numbers, and Chinese characters, there are many similar characters due to the large number of different font structures. The offline verification code recognition involved in the present application is for recognizing verification codes with Chinese characters in static two-dimensional images by extracting text features from verification code images, so the recognition is more complex and difficult, and the accuracy is difficult to improve.

[0004] There are mainly two kinds of schemes for offline recognition of verification codes at present: the first kind is a method based on traditional machine learning, mainly applying multi-feature verification code recognition technology of support vector machine (SVM), linear discriminant model (LDA), and modified quadratic discriminant function (MQDF), etc. On the basis of extracting grid features, the extraction of various main features of the verification code is increased. This kind of method needs data preprocessing and complex feature engineering construction before modeling, and because the extraction of features is complex, including the processing of Chinese character centroid and stroke features, it is difficult to extract accurate features comprehensively. The other kind is based on deep learning to recognize verification code Chinese characters, and there are various models of convolutional neural networks. At present, the main models applied are VGGNet, ResNet, etc. to build a deep and complex neural network for verification code recognition. This kind of method has problems such as many parameters to be tuned, slow network convergence, and large storage model space.

[0005] Based on the above analysis, how to quickly and accurately realize verification code recognition is a technical problem to be solved. SUMMARY

[0006] The technical task of the present application is to provide a verification code recognition method and system based on an improved YOLO9000 algorithm to solve the technical problem of how to quickly and accurately realize verification code recognition in view of the above shortcomings.

[0007] In a first aspect, the verification code recognition method based on the improved YOLO9000 algorithm of the present application comprises the following steps:

[0008] The training sample is constructed by collecting a verification code picture, and the verification code in the verification code picture is composed of Chinese characters, English and numbers.

[0009] For the training sample, the Chinese characters in the verification code are labeled, and the verification code picture is processed by image transformation to obtain a preprocessed training sample.

[0010] For the YOLO9000 model, the convolution layer is improved by reducing the convolution of the bottleneck structure, and the cross-entropy loss function and the average difference similarity function are used as the loss function to obtain an improved YOLO9000 model, the improved YOLO9000 model retains the pre-training and multi-classification functions, and a verification code recognition model is constructed based on the improved YOLO9000 model and is denoted as a YOLO model.

[0011] A verification code recognition model is constructed based on the DBN model and is denoted as a DBN model.

[0012] The YOLO model and the DBN model are trained based on the preprocessed training sample to obtain a trained YOLO model and a trained DBN model.

[0013] For the verification code picture to be recognized, the Chinese characters in the verification code are labeled, and the verification code picture is processed by image transformation to obtain a preprocessed verification code picture.

[0014] The preprocessed verification code picture is recognized by the trained YOLO model to obtain a first classification result, and the preprocessed verification code picture is recognized by the trained DBN model to obtain a second classification result, and the first classification result and the second classification result are fused by using linear credibility accumulation to obtain a final classification result.

[0015] As preferred, the image transformation processing of the verification code picture includes rotation transformation, translation transformation, shadow processing and picture enhancement processing.

[0016] As preferred, for the training sample and the verification code picture to be recognized, data cleaning is performed to remove invalid and incomplete Chinese characters, English and numbers.

[0017] As preferred, the convolution layer is improved by reducing the convolution of the bottleneck structure, two 1x1 convolutions are deleted, the detection head is reduced from nine convolution layers to six, a dropout layer is connected after each max-pooling layer to prevent overfitting, and the improved YOLO9000 model has fifteen convolution layers.

[0018] As preferred, the DBN model comprises a restricted Boltzmann machine (RBM) and a top layer of back propagation network (BP), the RBM layer has three layers, namely a visible layer, a hidden layer and a full connection layer, the visible layer is used for input data, the hidden layer is used for feature detection, the visible layer and the hidden layer are connected through the full connection, the RBM layer is used for unsupervised machine learning training, the lower RBM is taken as the output of the upper layer, the output result is trained through the BP neural network, the error between the actual output and the expected output is back propagated layer by layer, the weights of the network are adjusted, and finally a three-layer DBN model suitable for CAPTCHA recognition is obtained.

[0019] As preferred, p is defined by the following formula i is the recognition ability of the CAPTCHA recognition model for the i-th character, c i represents the number of times that the i-th character is correctly recognized, n i represents the total number of times that the i-th character appears in the sample set, and N is the total number of character types.

[0020]

[0021] The recognition ability vector of the first CAPTCHA recognition model for N character types is represented as follows:

[0022]

[0023] wherein represents the recognition ability of the YOLO model for the N-th character.

[0024] The recognition ability vector of the second CAPTCHA recognition model for N character types is represented as follows:

[0025]

[0026] wherein represents the recognition ability of the YOLO model for the N-th character.

[0027] As preferred, the first classification result and the second classification result are fused by adopting a linear confidence accumulation method, comprising the following steps:

[0028] The classification recognition of the CAPTCHA picture to be recognized is performed by the trained YOLO model, and a score X YOLO is obtained.

[0029]

[0030] The recognition score of the YOLO model is multiplied point by point with the recognition ability vector p YOLO of the YOLO model, and the result is taken as the final recognition score O YOLO of the model.

[0031]

[0032] The score X obtained by classifying and recognizing the to-be-recognized verification code picture through the trained DBN model DBN is represented as:

[0033]

[0034] The recognition score of the DBN model is multiplied by the capability vector p of the DBN model DBN The result of the dot multiplication is taken as the final recognition score O of the model DBN is represented as:

[0035]

[0036] The components of the score of the trained YOLO model are sorted from large to small, the two largest components are selected and recorded,

[0037] The components of the score of the DBN model are sorted from large to small, the two largest components are selected and recorded,

[0038] The results obtained by fusing the YOLO model and the DBN model are outputted to obtain the final classification and recognition result class, linear confidence accumulation (LCA) is adopted, alpha and beta are introduced as weighting factors, the recognition scores of the YOLO model and the DBN model are fused, wherein the sum of alpha and beta is equal to 1, the values of alpha and beta are adjusted to balance the proportion between the two models, and the calculation formula of the recognition score is as follows,

[0039] O = alpha X YOLO + beta X DBN = (o1, o2, …, o N )

[0040] The component with the largest probability in O is finally obtained and recorded as class and outputted:

[0041] class = argmax (o1, o2, …, o N ).

[0042] In the second aspect, the verification code recognition system based on the improved YOLO9000 algorithm, the verification code is recognized through the verification code recognition method based on the improved YOLO9000 algorithm in any one of the first aspect, and the system comprises:

[0043] A data acquisition module, the data acquisition module is used for acquiring verification code pictures to construct training samples, and the verification code in the verification code picture is composed of Chinese characters, English and numbers;

[0044] A data preprocessing module, configured to label Chinese characters in the verification code, clean data of the verification code image, remove invalid and incomplete Chinese characters, English and numbers, and perform image transformation on the verification code image to obtain preprocessed training samples;

[0045] A YOLO model construction module, configured to, for a YOLO9000 model, improve a convolutional layer by reducing the construction of a convolution of a bottleneck structure, and use a cross-entropy loss function plus an average difference similarity function as a loss function to obtain an improved YOLO9000 model, the improved YOLO9000 model retaining pre-training and multi-classification functions, and based on the improved YOLO9000 model, construct a verification code recognition model, denoted as a YOLO model; and based on the preprocessed training samples, train the YOLO model to obtain a trained YOLO model;

[0046] A DBN model construction module, configured to, based on a DBN model, construct a verification code recognition model, denoted as a DBN model; and based on the preprocessed training samples, train the DBN model to obtain a trained DBN model;

[0047] An identification preprocessing module, configured to, for a verification code image to be identified, call the data preprocessing module, label Chinese characters in the verification code by the data preprocessing module, and perform image transformation on the verification code image to obtain a preprocessed verification code image;

[0048] An identification classification module, configured to, by the trained YOLO model, perform identification classification on the preprocessed verification code image to obtain a first classification result, and by the trained DBN model, perform identification classification on the preprocessed verification code image to obtain a second classification result, and by using a linear confidence accumulation manner, fuse the first classification result and the second classification result to obtain a final classification result.

[0049] Preferably, the YOLO model construction module is configured to improve the convolutional layer by reducing the construction of the convolution of the bottleneck structure, delete two 1x1 convolutions, reduce the detection head from nine convolutional layers to six, and connect a dropout layer after each max-pooling layer to prevent overfitting, and the improved YOLO9000 model has fifteen convolutional layers.

[0050] The DBN model consists of a Restricted Boltzmann Machine (RBM) and a top-level Backpropagation Network (BP). The RBM layer has three layers: a visible layer, a hidden layer, and a fully connected layer. The visible layer is used for input data, and the hidden layer is used for feature detection. The visible layer and the hidden layer are connected by a fully connected layer. Unsupervised machine learning training is performed through the RBM layer, with the lower RBM layer serving as the output of the upper layer. The BP neural network is used to train the output results, and the error between the actual output and the expected output is backpropagated layer by layer to adjust the weights of the network, ultimately resulting in a three-layer DBN model suitable for CAPTCHA recognition.

[0051] Preferably, the identification and classification module defines p by the following formula. i To determine the recognition capability of the CAPTCHA recognition model for the i-th character, c i n represents the number of times the i-th character is correctly recognized. i This represents the total number of times the i-th character appears in the sample set, where N is the total number of character types.

[0052]

[0053] The first CAPTCHA recognition model's ability to recognize N types of characters can be represented by the following vector:

[0054]

[0055] in This represents the YOLO model's ability to recognize the Nth character.

[0056] The second CAPTCHA recognition model's ability to recognize N types of characters is represented by the following vector:

[0057]

[0058] in This represents the YOLO model's ability to recognize the Nth character.

[0059] The identification and classification module is used to fuse the first classification result and the second classification result by employing a linear confidence accumulation method, including the following steps:

[0060] The trained YOLO model is used to classify and recognize the CAPTCHA image to be identified, and the resulting score X is obtained. YOLO Represented as:

[0061]

[0062] The recognition score of the YOLO model is compared with the recognition ability vector p of the YOLO model. YOLO The result of the dot product is used as the model's final recognition score O. YOLO Represented as:

[0063]

[0064] Through the trained DBN model, the score X of the to-be-recognized verification code picture is obtained by classification recognition DBN is expressed as:

[0065]

[0066] The recognition score of the DBN model is multiplied with the capability vector p of the DBN model DBN The result is taken as the final recognition score O of the model DBN is expressed as:

[0067]

[0068] The components of the score of the trained YOLO model are sorted from large to small, the two largest components are selected and recorded,

[0069] The components of the score of the DBN model are sorted from large to small, the two largest components are selected and recorded,

[0070] The results obtained by fusing the YOLO model and the DBN model are outputted to obtain the final classification recognition result class, linear confidence accumulation (LCA) is adopted, alpha and beta are introduced as weighting factors, the recognition scores of the YOLO model and the DBN model are fused, wherein the sum of alpha and beta is equal to 1, the values of alpha and beta are adjusted to balance the proportion between the two models, and the calculation formula of the recognition score is as follows,

[0071] O = alpha X YOLO + beta X DBN = (o1, o2, …, o N )

[0072] Finally, the component with the maximum probability in O is obtained and recorded as class and outputted:

[0073] class = argmax (o1, o2, …, o N ).

[0074] The verification code recognition method and system based on the improved YOLO9000 algorithm have the following advantages:

[0075] 1. Compared with the existing verification code recognition method using a traditional machine learning model, the improved YOLO-9000 fuses a DBN model by improving a loss function and reconstructing a backbone network, so that the recognition rate is improved while the recognition time is ensured.

[0076] 2、Compared with the deep learning model, the fusion model is stronger than the single model in recognition rate by adjusting the weighting factor, and the generalization performance of the model is better than that of the single deep learning model to prevent overfitting. Due to the modification of the convolution layer of the YOLO-9000 backbone network, the operation recognition speed is stronger than that of the general deep learning model. BRIEF DESCRIPTION OF DRAWINGS

[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0078] The present application will be further described below in conjunction with the drawings.

[0079] Figure 1 The flowchart of the verification code recognition method based on the improved YOLO9000 algorithm of embodiment 1 is shown in the figure.

[0080] Figure 2 The structure diagram of the improved YOLO-9000 in the verification code recognition method based on the improved YOLO9000 algorithm of embodiment 1 is shown in the figure.

[0081] Figure 3 The structure diagram of the DBN model in the verification code recognition method based on the improved YOLO9000 algorithm of embodiment 1 is shown in the figure.

[0082] Figure 4 The structure diagram of the fusion model of the improved YOLO-9000 and DBN in the verification code recognition method based on the improved YOLO9000 algorithm of embodiment 1 is shown in the figure. DETAILED DESCRIPTION

[0083] The present application will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present application and implement it. However, the embodiments are not limiting to the present application, and the technical features in the embodiments and the embodiments can be combined with each other without conflict.

[0084] The present application provides a verification code recognition method and system based on an improved YOLO9000 algorithm, which is used to solve the technical problem of how to quickly and accurately realize verification code recognition.

[0085] Embodiment 1:

[0086] The verification code recognition method based on the improved YOLO9000 algorithm of the present application comprises the following steps:

[0087] S100, collect verification code pictures to construct training samples, the verification code in the verification code picture is composed of Chinese characters, English and numbers;

[0088] S200, for the training samples, label the Chinese characters in the verification code, and perform image transformation processing on the verification code picture to obtain preprocessed training samples;

[0089] S300, for the YOLO9000 model, the convolution of the bottleneck structure is reduced, the convolution layer is improved, and the cross-entropy loss function and the average difference similarity function are used as the loss function to obtain the improved YOLO9000 model, the improved YOLO9000 model retains the pre-training and multi-classification functions, and a verification code recognition model is constructed based on the improved YOLO9000 model and is recorded as a YOLO model;

[0090] S400, a verification code recognition model is constructed based on the DBN model and is recorded as a DBN model;

[0091] S500, the YOLO model and the DBN model are trained based on the preprocessed training samples to obtain a trained YOLO model and a trained DBN model;

[0092] S600, for the verification code picture to be recognized, label the Chinese characters in the verification code, and perform image transformation processing on the verification code picture to obtain a preprocessed verification code picture;

[0093] S700, the preprocessed verification code picture is recognized and classified by the trained YOLO model to obtain a first classification result, and the preprocessed verification code picture is recognized and classified by the trained DBN model to obtain a second classification result, and the first classification result and the second classification result are fused by using a linear confidence accumulation method to obtain a final classification result.

[0094] In this embodiment, verification codes with Chinese characters, English and numbers are collected, and the collected verification code pictures are collected as a data set as a training sample set.

[0095] Step S200 is a data processing and image enhancement process. The Chinese characters in the verification code are labeled separately, and the collected pictures are subjected to image enhancement processing. Since some verification code characters appear frequently, data augmentation is performed on the data set to prevent overfitting when fitting the model. The verification code picture is subjected to image transformation processing, including rotation transformation, translation transformation, shadow processing and picture enhancement processing. At the same time, data cleaning is performed to remove invalid and incomplete Chinese characters, English and numbers.

[0096] YOLO-9000 refers to the ability to recognize 9000 kinds of objects. In a common verification code, each character can be regarded as an object, and the commonly used Chinese characters in the verification code recognition are about 500-1000, so the improved YOLO-9000 model can reduce the classification to 1000 categories. The convolutional neural network in the application proposal has 15 convolutional layers, which is based on YOLOv2 and combines the construction idea of GoogleNet. The improved YOLO9000 retains the ability to pre-train and perform multi-classification, and is pre-trained by taking advantage of the large amount of labeled classification data set through data collection in step S100 and data labeling in step S200. The improved convolutional layer is constructed by reducing the convolution of the bottleneck structure, deleting two 1x1 convolutional layers, and reducing the original detection head from 9 convolutional layers to 6, making the body more lightweight and suitable for the needs of Chinese character recognition in verification code. In the selection of the loss function, the application proposal modifies the previous YOLO model and uses the cross-entropy loss function plus the average difference similarity function as the loss function. This approach can increase the inter-class variation and reduce the intra-class variation, thereby obtaining better classification performance.

[0097] The improved YOLO model uses the maximum entropy regularization term plus the average variance similarity function as the loss function, which can increase the inter-class variation and reduce the intra-class variation, thereby obtaining better classification performance. The combination of the two loss functions is much better than the feature classification effect based on cross-entropy as the loss function alone.

[0098] Generally speaking, the predicted intra-class variance in the verification code recognition is also large, that is, the entropy is large, and we hope to regularize the output entropy to make the model more general and reduce overfitting. The expression is as follows, p i is the recognition ability of the model for the ith character, and H(p) is the entropy of the model output,

[0099]

[0100] Entropy is a heat vector that reaches a minimum value when it is uniformly distributed at p. The former is automatically realized by the ordinary cross-entropy loss, while the latter is expected to promote regularization. Therefore, we use negative entropy as the maximum entropy regularization term, which directly acts on the general cross-entropy loss function, as follows, the loss function of regression is represented as L REG is the negative entropy, that is, the reciprocal of the entropy λL MER , and L CE is accumulated

[0101] L MER = -H(P)

[0102] L REG = L CE + λLMER

[0103] Here, λ is a hyperparameter that determines the impact of MER. Intuitively, MER reduces the extreme confidence values ​​caused by cross-entropy loss. Considering the derivative of the regularization loss with respect to the output score, which is directly related to the model, its derivative with respect to the probability distribution is given by [equation missing]. Since the model probability distribution only relates to L... MER If it is related, then the derivative becomes:

[0104]

[0105] After differentiation, according to the chain rule, we can see that the gradient is not always positive or negative. Therefore, with a wider distribution of scores, the probability will not decrease to 0 or increase to 1. This completes the construction of the loss function.

[0106] Based on YOLOv2, an improved YOLO-9000 model was built by incorporating the construction ideas of GoogleNet. The improved YOLO9000 retains the ability to perform multi-class classification through pre-training. Pre-training is conducted using the large amount of labeled classification datasets obtained in step 1 (data acquisition) and step 2 (data annotation). The improved model consists of 15 convolutional layers. By reducing the bottleneck structure of convolutions, two 1×1 convolutional layers and the bottleneck convolution were removed, reducing the number of detection heads from 9 to 6. A dropout layer is added after each max-pooling layer to prevent overfitting, making the model more lightweight and suitable for CAPTCHA recognition requirements.

[0107] DBN is a hierarchical probabilistic generative model that trains the weights between neurons. The DBN model proposed in this application consists of a series of Restricted Boltzmann Machines (RBMs) and a top-level Backpropagation Network (BP). The RBM layer has three layers: the visible layer takes the data as input, the hidden layer performs feature detection, and all layers are fully connected. Unsupervised machine learning training is performed through the RBM layers, using the lower RBM as the output of the upper layer. The output is then trained using a BP neural network, backpropagating the error between the actual and expected output layer by layer, adjusting the network weights, and finally obtaining a three-layer DBN model suitable for CAPTCHA recognition. During the training phase with labeled samples, fine-grained classification adjusts the weights of the BP network, backpropagating the error between the actual and expected data layer by layer. Unsupervised machine learning training is performed through the RBM layers, using the lower RBM as the output of the upper layer. The output is then trained using a BP neural network, backpropagating the error between the actual and expected output layer by layer, adjusting the network weights, and finally obtaining the DBN model.

[0108] Both the two models YOLO model and DBN model can recognize the verification code, and apply respective network to feature extraction. Since the modeling essence of YOLO and DBN model is different, the means of extracting features are advantageous, and the fusion model of YOLO and DBN fusion will exceed the recognition ability of single model in effect. The overall process of YOLO-DBN is divided into two parts of training process and application process. That is, the training and application process are separately carried out. The application process adopts linear confidence accumulation, introduces weighting factors a and b, and obtains the final classification result by adjusting the weighting factors.

[0109] In the embodiment, when recognizing the verification code, the improved YOLO9000 and the DBN model constitute the fusion model for recognition. First, the YOLO model and the DBN model are trained separately, and the trained recognition model is obtained; then the recognition ability of the two models for different characters is counted on the sample set, and p i is defined by the following formula, wherein p i represents the recognition ability of the model for the i-th character; c i represents the number of times that the i-th character is recognized correctly, n YOLO represents the total number of times that the i-th character appears in the sample set. N is the total number of character types.

[0110]

[0111] The recognition ability vector of YOLO for N characters is represented as follows, wherein represents the recognition ability of the YOLO model for the N-th character:

[0112]

[0113] The recognition ability vector of DBN for N characters is represented as follows, wherein represents the recognition ability of the YOLO model for the N-th character:

[0114]

[0115] The main steps of the fusion model algorithm involved are as follows:

[0116] 1) The classification score of the character picture calculated by the YOLO model is X YOLO :

[0117]

[0118] The recognition score of the YOLO model is multiplied with the ability vector p YOLO of the model, and the result is taken as the final recognition score O YOLO of the model:

[0119]

[0120] 2) The score of the classification obtained by the character picture through the DBN model calculation is X DBN

[0121]

[0122] The recognition score of the DBN model is multiplied with the ability vector p of the model DBN The result of the dot product is taken as the final recognition score O of the model DBN :

[0123]

[0124] 3) The components of the YOLO model score are sorted from large to small, the largest two components are selected and recorded,

[0125] 4) The components of the DBN model score are sorted from large to small, the largest two components are selected and recorded,

[0126] 5) The results of the YOLO and DBN two models are fused, and the final classification recognition result class is output, linear confidence accumulation (LCA) is adopted, and α and β are introduced as weighting factors to fuse the recognition scores of the two models, where the sum of α and β is equal to 1, the proportion between the two models is balanced by adjusting the values of α and β, and the calculation formula of the recognition score is as follows,

[0127] O = αX YOLO + βX DBN = (o1o2, …, o N )

[0128] The component with the largest probability in O is finally obtained, denoted as class and output:

[0129] class = argmax (o1o2, …, o N )

[0130] The verification code recognition of the embodiment, in the YOLO model, the maximum entropy regularization is applied to regularize the training process, the maximum entropy regularization adds a negative entropy term in the cross entropy loss, using the maximum entropy regularization and the average variance similarity function as the loss function, this method can increase the class variation and reduce the class variation, so as to obtain better classification performance; at the same time, in the YOLO model, the convolutional neural network has 15 convolutional layers, by improving the convolutional layer, reducing the construction of the convolutional layer of the bottleneck structure, reducing the network magnitude, deleting two 1x1 convolutional layers, the original detection head reduces from 9 convolutional layers to 6, and the dropout layer is applied after each max pooling layer to prevent overfitting; by increasing the weighting factor to adjust the proportion of each model result, finally the maximum probability classification class of the sample is given. The method of fusing machine learning improves the generalization ability of the model, and the scheme of fusing the model further solves the problems of low recognition efficiency and generalization difficulty of heavy network.

[0131] Embodiment 2:

[0132] The application is based on a verification code recognition system improved from YOLO9000 algorithm, comprising a data acquisition module, a data preprocessing module, a YOLO model construction module, a DBN model construction module, a recognition preprocessing module and a recognition classification module. The data acquisition module is used for acquiring verification code pictures to construct training samples. The verification codes in the verification code pictures are composed of Chinese characters, English and numbers. The data preprocessing module is used for labeling Chinese characters in the verification codes, data cleaning of the verification code pictures, removing invalid and incomplete Chinese characters, English and numbers, and image transformation processing of the verification code pictures to obtain preprocessed training samples. The YOLO model construction module is used for improving the convolution layer of the YOLO9000 model by reducing the convolution structure of the bottleneck structure, using a cross-entropy loss function plus an average difference similarity function as a loss function to obtain an improved YOLO9000 model. The improved YOLO9000 model retains the pre-training and multi-classification functions. A verification code recognition model is constructed based on the improved YOLO9000 model and is recorded as a YOLO model. The YOLO model is trained based on the preprocessed training samples to obtain a trained YOLO model. The DBN model construction module is used for constructing a verification code recognition model based on a DBN model and is recorded as a DBN model. The DBN model is trained based on the preprocessed training samples to obtain a trained DBN model. The recognition preprocessing module is used for calling the data preprocessing module, labeling Chinese characters in the verification code of the verification code picture to be recognized by the data preprocessing module, and performing image transformation processing on the verification code picture to obtain a preprocessed verification code picture. The recognition classification module is used for classifying the preprocessed verification code picture by the trained YOLO model to obtain a first classification result, classifying the preprocessed verification code picture by the trained DBN model to obtain a second classification result, and fusing the first classification result and the second classification result by using a linear confidence accumulation method to obtain a final classification result.

[0133] In the system of the embodiment, the YOLO model construction module is used for improving the convolution layer by reducing the convolution structure of the bottleneck structure, deleting two 1x1 convolution layers, reducing the detection head from 9 convolution layers to 6 convolution layers, and connecting a dropout layer after each max-pooling layer to prevent overfitting. The improved YOLO9000 model has fifteen convolution layers.

[0134] Generally, the intra-class variance of the prediction in the verification code recognition is also large, that is, the entropy is large. We hope to regularize the output entropy to make the model more general and reduce overfitting. The expression is as follows: p i is the recognition ability of the model for the i-th character, and H(p) is the entropy of the model output,

[0135]

[0136] Entropy is minimized when the heat vector is uniform, and maximized when the heat vector is uniform. The former is automatically achieved by the ordinary cross-entropy loss, while the latter is expected to promote regularization. Therefore, we take the negative entropy, which is the inverse of entropy, as the maximum entropy regularization term, which directly acts on the general cross-entropy loss function, as follows, the loss function of regression is represented as L REG , which is composed of the negative entropy, i.e. the inverse of entropy, λL MER , and L CE .

[0137] L MER = -H(P)

[0138] L REG = L CE + λL MER

[0139] Where λ is a hyperparameter that determines the impact of MER. Intuitively, MER reduces the extreme confidence values caused by cross-entropy loss. Consider the derivative of the regularization loss with respect to the output score, which is directly related to the model, and the derivative with respect to the probability distribution is, at this time, since the model probability distribution is only related to L MER , the derivative becomes:

[0140]

[0141] After derivation, according to the chain rule, we have, where the gradient is not always positive or negative, so the probability will not decrease to 0 or increase to 1 under more distribution scores. Thus, the construction of the loss function is completed.

[0142] The DBN model includes a restricted Boltzmann machine (RBM) and a top layer of back propagation network (BP). The RBM layer has three layers, namely the visual layer, the hidden layer and the fully connected layer. The visual layer is used for input data, the hidden layer is used for feature detection, and the visual layer is connected to the hidden layer through full connection. Unsupervised machine learning training is performed through the RBM layer. The lower RBM is taken as the output of the upper layer, and the output result is trained through the BP neural network. The error between the actual output and the expected output is propagated back layer by layer, and the weights of the network are adjusted. Finally, a three-layer DBN model suitable for CAPTCHA recognition is obtained.

[0143] The recognition classification module defines p i as the recognition ability of the CAPTCHA recognition model for the i-th character, c i represents the number of times the i-th character is correctly recognized, n i represents the total number of times the i-th character appears in the sample set, and N is the total number of character types.

[0144]

[0145] The recognition ability vector of the first verification code recognition model for N characters is represented as follows:

[0146]

[0147] wherein represents the recognition ability of the YOLO model for the Nth character;

[0148] The recognition ability vector of the second verification code recognition model for N characters is represented as follows:

[0149]

[0150] wherein represents the recognition ability of the YOLO model for the Nth character;

[0151] The recognition classification module is configured to fuse the first classification result and the second classification result by adopting linear credibility accumulation, including the following steps:

[0152] The score X obtained by classifying and recognizing the to-be-recognized verification code picture through the trained YOLO model is represented as: YOLO

[0153]

[0154] The dot product result of the recognition score of the YOLO model and the recognition ability vector p of the YOLO model is taken as the final recognition score O of the model. YOLO YOLO

[0155]

[0156] The score X obtained by classifying and recognizing the to-be-recognized verification code picture through the trained DBN model is represented as: DBN

[0157]

[0158] The dot product result of the recognition score of the DBN model and the ability vector p of the DBN model is taken as the final recognition score O of the model. DBN DBN

[0159]

[0160] The components of the score of the trained YOLO model are sorted from large to small, the largest two components are selected and recorded,

[0161] ​​​​​​The components of the DBN model score are sorted from large to small, the largest two components are selected and recorded,

[0162] The results obtained by fusing the YOLO model and the DBN model are output as the final classification recognition result class, linear confidence accumulation (LCA) is adopted, and alpha and beta are introduced as weighting factors to fuse the recognition scores of the YOLO model and the DBN model, wherein the sum of alpha and beta is equal to 1, the proportion between the two models is balanced by adjusting the values of alpha and beta, and the calculation formula of the recognition score is as follows,

[0163] O = alpha X YOLO + beta X DBN = (o1, o2, …, o N )

[0164] Finally, the component with the largest probability in O is obtained and recorded as class and output:

[0165] class = argmax (o1, o2, …, o N ).

[0166] The system of the embodiment can execute the verification code recognition method and system based on the improved YOLO9000 algorithm disclosed in embodiment 1.

[0167] The application has been described in detail above through the drawings and preferred embodiments, but the application is not limited to these disclosed embodiments, and those skilled in the art can know that the code review means in the above different embodiments can be combined to obtain more embodiments of the application, and these embodiments are also within the protection scope of the application.

Claims

1. A CAPTCHA recognition method based on an improved YOLO9000 algorithm, characterized in that... Includes the following steps: Training samples are constructed by collecting CAPTCHA images, wherein the CAPTCHA in the images consists of Chinese characters, English letters, and numbers; For the training samples, the Chinese characters in the CAPTCHA are labeled, and the CAPTCHA images are processed by image transformation to obtain the preprocessed training samples. For the YOLO9000 model, the convolutional layers are improved by reducing the construction of bottleneck structures, and the cross-entropy loss function plus the average difference similarity function is used as the loss function to obtain the improved YOLO9000 model. The improved YOLO9000 model retains the pre-training and multi-classification functions. A CAPTCHA recognition model is built based on the improved YOLO9000 model and denoted as the YOLO model. A CAPTCHA recognition model is constructed based on the DBN model and denoted as the DBN model. The YOLO model and DBN model are trained based on the preprocessed training samples to obtain the trained YOLO model and the trained DBN model. For the CAPTCHA image to be recognized, the Chinese characters in the CAPTCHA are annotated, and the CAPTCHA image is transformed to obtain the preprocessed CAPTCHA image. The preprocessed CAPTCHA image is identified and classified using the trained YOLO model to obtain a first classification result, and the preprocessed CAPTCHA image is identified and classified using the trained DBN model to obtain a second classification result. The first and second classification results are then fused using a linear confidence accumulation method to obtain the final classification result.

2. The CAPTCHA recognition method based on the improved YOLO9000 algorithm according to claim 1, characterized in that... Image transformation processing is performed on CAPTCHA images, including rotation, translation, shadow processing, and image enhancement.

3. The CAPTCHA recognition method based on the improved YOLO9000 algorithm according to claim 1, characterized in that... For the training samples and the CAPTCHA images to be recognized, data cleaning is performed to remove invalid and incomplete Chinese characters, English letters, and numbers.

4. The CAPTCHA recognition method based on the improved YOLO9000 algorithm according to claim 1, characterized in that... The convolutional layers are improved by reducing the construction of convolutions in the bottleneck structure, removing two 1×1 convolutional layers, reducing the number of detection heads from nine convolutional layers to six, and adding a dropout layer after each max pooling layer to prevent overfitting. The improved YOLO9000 model has fifteen convolutional layers.

5. The CAPTCHA recognition method based on the improved YOLO9000 algorithm according to claim 1, characterized in that... The DBN model consists of a Restricted Boltzmann Machine (RBM) and a top-level Backpropagation Network (BP). The RBM layer has three layers: a visible layer, a hidden layer, and a fully connected layer. The visible layer is used for input data, and the hidden layer is used for feature detection. The visible layer and the hidden layer are connected by a fully connected layer. Unsupervised machine learning training is performed through the RBM layer, with the lower RBM layer serving as the output of the upper layer. The BP neural network is used to train the output results, and the error between the actual output and the expected output is backpropagated layer by layer to adjust the weights of the network, ultimately resulting in a three-layer DBN model suitable for CAPTCHA recognition.

6. The CAPTCHA recognition method based on the improved YOLO9000 algorithm according to claim 1, characterized in that... p is defined by the following formula i To determine the recognition capability of the CAPTCHA recognition model for the i-th character, c i n represents the number of times the i-th character is correctly recognized. i This represents the total number of times the i-th character appears in the sample set, where N is the total number of character types. The first CAPTCHA recognition model's ability to recognize N types of characters can be represented by the following vector: in This represents the YOLO model's ability to recognize the Nth character. The second CAPTCHA recognition model's ability to recognize N types of characters is represented by the following vector: in This represents the YOLO model's ability to recognize the Nth character.

7. The CAPTCHA recognition method based on the improved YOLO9000 algorithm according to claim 1, characterized in that... The first classification result and the second classification result are fused by using a linear confidence accumulation method, including the following steps: The trained YOLO model is used to classify and recognize the CAPTCHA image to be identified, and the resulting score X is obtained. YOLO Represented as: The recognition score of the YOLO model is compared with the recognition ability vector p of the YOLO model. YOLO The result of the dot product is used as the model's final recognition score O. YOLO Represented as: The trained DBN model is used to classify and recognize the CAPTCHA image to be identified, and the resulting score X is obtained. DBN Represented as: The recognition score of the DBN model is compared with the recognition ability vector p of the DBN model. DBN The result of the dot product is used as the model's final recognition score O. DBN Represented as: Sort the components of the YOLO model score from largest to smallest, select the two largest components and record them. Sort the components of the DBN model score from largest to smallest, select the two largest components and record them. The results obtained by fusing the YOLO and DBN models are used to output the final classification result (class). Linear confidence accumulation (LCA) is employed, introducing α and β as weighting factors to fuse the recognition scores from the YOLO and DBN models. The sum of α and β equals 1. By adjusting the values ​​of α and β, the relative weights of the two models are balanced. The formula for calculating the recognition score is as follows: O = αX YOLO +βX DBN =(o1o2,…,o N ) Finally, the component with the highest probability in O is obtained, denoted as class, and output: class=argmax(o1o2,…,o N )。 8. A CAPTCHA recognition system based on an improved YOLO9000 algorithm, characterized in that... The system performs CAPTCHA recognition using the CAPTCHA recognition method based on the improved YOLO9000 algorithm as described in any one of claims 1-7, and the system comprises: The data acquisition module is used to collect CAPTCHA images to construct training samples. The CAPTCHA in the CAPTCHA images consists of Chinese characters, English letters, and numbers. The data preprocessing module is used to annotate the Chinese characters in the verification code, clean the verification code image to remove invalid and incomplete Chinese characters, English letters and numbers, and perform image transformation processing on the verification code image to obtain preprocessed training samples. The YOLO model building module is used to improve the YOLO9000 model by reducing the construction of convolutional layers in the bottleneck structure, and using the cross-entropy loss function plus the average difference similarity function as the loss function to obtain an improved YOLO9000 model. The improved YOLO9000 model retains the pre-training and multi-classification functions. A CAPTCHA recognition model is built based on the improved YOLO9000 model and denoted as the YOLO model. It is also used to train the YOLO model based on the preprocessed training samples to obtain the trained YOLO model. The DBN model building module is used to build a CAPTCHA recognition model based on the DBN model, and is denoted as DBN model; and is used to train the DBN model based on the preprocessed training samples to obtain the trained DBN model. The recognition preprocessing module is used to call the data preprocessing module. For the verification code image to be recognized, the data preprocessing module annotates the Chinese characters in the verification code and performs image transformation processing on the verification code image to obtain the preprocessed verification code image. The identification and classification module is used to identify and classify the preprocessed CAPTCHA image using a trained YOLO model to obtain a first classification result, and to identify and classify the preprocessed CAPTCHA image using a trained DBN model to obtain a second classification result. The first and second classification results are then fused using a linear confidence accumulation method to obtain a final classification result.

9. The CAPTCHA recognition system based on the improved YOLO9000 algorithm according to claim 8, characterized in that... The YOLO model building module is used to improve the convolutional layers by reducing the construction of convolutions in the bottleneck structure, removing two 1×1 convolutional layers, reducing the number of detection heads from nine convolutional layers to six, and adding a dropout layer after each max pooling layer to prevent overfitting. The improved YOLO9000 model has fifteen convolutional layers. The DBN model consists of a Restricted Boltzmann Machine (RBM) and a top-level Backpropagation Network (BP). The RBM layer has three layers: a visible layer, a hidden layer, and a fully connected layer. The visible layer is used for input data, and the hidden layer is used for feature detection. The visible layer and the hidden layer are connected by a fully connected layer. Unsupervised machine learning training is performed through the RBM layer, with the lower RBM layer serving as the output of the upper layer. The BP neural network is used to train the output results, and the error between the actual output and the expected output is backpropagated layer by layer to adjust the weights of the network, ultimately resulting in a three-layer DBN model suitable for CAPTCHA recognition.

10. The CAPTCHA recognition system based on the improved YOLO9000 algorithm according to claim 8 or 9, characterized in that... The identification and classification module defines p using the following formula. i To determine the recognition capability of the CAPTCHA recognition model for the i-th character, c i n represents the number of times the i-th character is correctly recognized. i This represents the total number of times the i-th character appears in the sample set, where N is the total number of character types. The first CAPTCHA recognition model's ability to recognize N types of characters can be represented by the following vector: in This represents the YOLO model's ability to recognize the Nth character. The second CAPTCHA recognition model's ability to recognize N types of characters is represented by the following vector: in This represents the YOLO model's ability to recognize the Nth character. The identification and classification module is used to fuse the first classification result and the second classification result by employing a linear confidence accumulation method, including the following steps: The trained YOLO model is used to classify and recognize the CAPTCHA image to be identified, and the resulting score X is obtained. YOLO Represented as: The recognition score of the YOLO model is compared with the recognition ability vector p of the YOLO model. YOLO The result of the dot product is used as the model's final recognition score O. YOLO Represented as: The trained DBN model is used to classify and recognize the CAPTCHA image to be identified, and the resulting score X is obtained. DBN Represented as: The recognition score of the DBN model is compared with the recognition ability vector p of the DBN model. DBN The result of the dot product is used as the model's final recognition score O. DBN Represented as: Sort the components of the YOLO model score from largest to smallest, select the two largest components and record them. Sort the components of the DBN model score from largest to smallest, select the two largest components and record them. The results obtained by fusing the YOLO and DBN models are used to output the final classification result (class). Linear confidence accumulation (LCA) is employed, introducing α and β as weighting factors to fuse the recognition scores from the YOLO and DBN models. The sum of α and β equals 1. By adjusting the values ​​of α and β, the relative weights of the two models are balanced. The formula for calculating the recognition score is as follows: O = αX YOLO +βX DBN =(o1o2,…,o N ) Finally, the component with the highest probability in O is obtained, denoted as class, and output: class=argmax(o1o2,…,o N )。

Images