Handwritten character recognition method and system based on template-centered deep discriminant analysis

Abstract

The application discloses a handwriting character recognition method and system based on template centralization deep discriminant analysis, which can effectively improve the character recognition accuracy compared with traditional character recognition methods. The handwriting character recognition method based on template centralization deep discriminant analysis comprises the following main steps: constructing a printed character template and a handwritten character training set, extracting features of the printed character template and the handwritten character by using a backbone network, constructing a template centralization discriminant analysis loss function, training and optimizing network parameters by using gradient back propagation, and effectively recognizing handwritten characters by using the trained model.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of pattern recognition, and particularly relates to a handwritten character recognition method and system based on template centralization deep discriminant analysis. BACKGROUND

[0002] Handwritten character recognition can be generally divided into online handwritten character recognition and offline handwritten character recognition. Online handwritten character recognition data is a coordinate sequence, and is usually applied to human-computer interaction devices such as a handwriting board, a tablet computer and the like. Offline handwritten character data is an image, and is widely applied to font recognition, check recognition and the like. Due to the fact that a large number of handwritten characters or related texts exist in life scenes, handwritten character recognition has wide application value. At present, most of the recognition technologies for offline handwritten characters adopt a classification method of a deep neural network, that is, a neural network is trained to reach a minimum classification error by constructing a softmax cross-entropy loss function, but the softmax cross-entropy loss function lacks the ability to learn information from different classes of samples, and is prone to misclassification when handwritten characters are small in difference. How to improve the accuracy of handwritten character recognition is one of the key problems to be solved in the field. SUMMARY

[0003] The application aims at solving the above problems, and provides a handwritten character recognition method and system based on template centralization deep discriminant analysis. A printed character and a handwritten character training set are constructed, a convolutional neural network is used to extract features of the printed character and the handwritten character, a loss function is constructed by using the idea of deep discriminant analysis, and the network is optimized through error back propagation. The system can effectively improve the accuracy of handwritten character recognition.

[0004] The application provides a handwritten character recognition method based on template centralization deep discriminant analysis, which comprises the following steps:

[0005] Step S1, constructing a handwritten character training set, and presetting the number of handwritten character categories as C;

[0006] Step S2, constructing a printed character template set by using C printed character images which are the same as the number of handwritten character categories;

[0007] Step S3, extracting features of images in the printed character template set and the handwritten character training set by using a backbone network, constructing a template centralization deep discriminant analysis model, taking the images in the printed character template set and the handwritten character training set as inputs, extracting features of the images as outputs, training the template centralization deep discriminant analysis model, and updating parameters of the backbone network through gradient back propagation until a loss function descending rate is lower than a set threshold value, so that a trained template centralization deep discriminant analysis model is obtained;

[0008] Step S4, using the trained template centralized deep discriminant analysis model to recognize new character images.

[0009] Further, in step S1, the MNIST dataset is used to construct a handwritten character training set, and the number of categories is 10.

[0010] Further, the specific steps of step S3 are as follows:

[0011] Step S3.1, randomly take n training set images to form a batch, and assume that there are n i samples belonging to the i-th character;

[0012] Step S3.2, the feature output of the template set after the backbone network is denoted as T i , T i ∈{T1,T2,…,T c}, and the feature output of each class of image in the batch after the backbone network is denoted as

[0013] Step S3.3, calculate the inter-class scatter matrix Sw and the intra-class scatter matrix Sb according to the above feature output:

[0014]

[0015]

[0016]

[0017] Wherein: represents the feature output of the j-th sample belonging to the i-th class, and T i represents the feature output of the template belonging to the i-th class.

[0018] For the Sw calculated, add a unit matrix of λ times to obtain a new intra-class matrix Sw new = Sw+λI.

[0019] Step S3.4: directly perform eigenvalue decomposition on Sw new -1 Sb to obtain eigenvalues v i and eigenvectors e i , take C-1 largest eigenvalues, and then use k smallest eigenvalues to construct a loss function expression:

[0020]

[0021] According to the loss function L, calculate the loss value of a batch, and update the backbone network parameters by gradient back propagation;

[0022] Step S3.5: Repeat S3.1 to S3.4 until the rate of loss function decrease is lower than a set threshold, and obtain the trained template-centered deep discriminant analysis model.

[0023] Further, the specific steps of step S4 are as follows:

[0024] Step S4.1: According to the trained template-centered deep discriminant analysis model, the feature mapping output of all training set samples is calculated, assuming that there are N i of each class, denoted as The feature mapping output T i of the template is consistent with step S3.2;

[0025] Step S4.2: Calculate the inter-class scatter matrix Sw and the intra-class scatter matrix Sb:

[0026]

[0027] Sw′=Sw+λI

[0028]

[0029]

[0030] Wherein: represents the feature output of the jth sample belonging to the ith class, T i represents the feature output of the template belonging to the ith class;

[0031] Eigenvalue decomposition is performed on Sw′ -1 and Sb to obtain eigenvectors e i and eigenvalues v i The eigenvalues are composed of the eigenvectors corresponding to the C-1 largest eigenvalues, and the stable projection matrix W=(e1, e2, …e C-1 ).

[0032] Step S4.3: Based on the projection matrix W obtained above and the template feature mapping output of step S3.2, Assuming that the output of the character image to be recognized after passing through the backbone network is X t , the distance to the linear decision hyperplane is:

[0033]

[0034]

[0035] Wherein: W T is the transpose of the projection matrix, and H is the normal vector of the decision hyperplane;

[0036] P C ′=1 / (1+e-d derive the probability of each class and determine the class of the sample by P C = P C ' / ∑P i ' is further normalized, and finally the class with the highest probability is determined as the class of the sample. i

[0037] In a second aspect, the present application provides a system for handwritten character recognition based on template-centered deep discriminant analysis, comprising:

[0038] a handwritten character acquisition module, configured to acquire a handwritten character picture to be recognized;

[0039] a feature extraction module, configured to input the handwritten character to be recognized into a pre-constructed discriminant model to perform feature extraction and acquire a feature vector;

[0040] a character recognition module, configured to calculate the probability of each class according to a decision function based on the obtained feature vector, and take the value with the highest probability as the recognition result of the character.

[0041] Advantages: Compared with the prior art, the present application has the following significant advantages: the present application provides a handwritten character recognition method and device based on template-centered deep discriminant analysis, and provides a loss function for template-centered discriminant analysis to train a convolutional neural network. The commonly used softmax cross-entropy loss function focuses on the deviation between the true value and the predicted value, but lacks the information between classes. The loss function provided by the present application enables the network to generate features that are clustered within the class and dispersed between the classes after training. Compared with the traditional discriminant analysis model, the present application is centered on the template rather than the mean, and the template participates in the calculation of the intra-class matrix, so that the image can generate a feature representation around the template. In addition, the hard negative mining method is used in the calculation of the inter-class matrix to increase the recognition degree between classes and achieve the effect of separating the classes as much as possible, thereby improving the accuracy of handwritten character recognition during discrimination. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a printed digital template sample picture;

[0043] Figure 2 is a deep convolutional neural network model training flowchart;

[0044] Figure 3 is a handwritten digital picture to be recognized. DETAILED DESCRIPTION

[0045] The technical solutions of the present application will be further described below in conjunction with the drawings.

[0046] ​A handwritten character recognition method based on template centering deep discriminant analysis, comprising the following steps:

[0047] Step S1: Construct a handwritten Arabic numeral character training set, in this example, directly use the public MNIST dataset, the number of categories is 10;

[0048] Step S2: Construct a template set using 10 printed character images corresponding to the handwritten character, as shown in Figure 1 ;

[0049] Step S3: Construct a template centering deep discriminant analysis model and train, as shown in Figure 2 ;

[0050] Step S4: Identify a handwritten character image as shown in Figure 3 ;

[0051] The specific steps of the step S3 are as follows:

[0052] Step S3.1: Take 1024 training set images to form a batch, and assume that there are n i samples belonging to the i-th character;

[0053] Step S3.2: The feature output of the template set after the backbone network is denoted as T i , T i ∈{T1,…,T 10}, and the feature output of each image in the batch after the backbone network is denoted as In this example, the backbone network is ResNet18;

[0054] Step S3.3: Calculate the inter-class scatter matrix Sw and the intra-class scatter matrix Sb according to the feature mapping output obtained above:

[0055]

[0056]

[0057] Wherein:

[0058]

[0059] For the Sw obtained by calculation, add a unit matrix with λ times to obtain a new intra-class matrix Sw′=Sw+λI;

[0060] Step S3.4: Perform eigenvalue decomposition on Sw′ -1 Sb to obtain eigenvalues v i and eigenvectors e i , take C-1 largest eigenvalues, and then use k smallest eigenvalues to construct a loss function expression:

[0061]

[0062] According to the loss function L, the loss value of a batch is calculated, and the gradient is back propagated to update the backbone network parameters;

[0063] Step S3.5: Repeat S3.1 to S3.4, train for 100 rounds, use the SGD optimizer, the initial learning rate is 0.001, the weight decay is set to 0.0005, and the loss function is reduced until the rate is lower than the set threshold;

[0064] The specific steps of the step S4 are as follows:

[0065] Step S4.1: According to the trained template centering deep discriminant analysis model, the feature mapping output of all training set samples is calculated, assuming that there are N i , denoted as The feature mapping output T i of the template is consistent with step S3.2;

[0066] Step S4.2: Calculate the inter-class scatter matrix Sw and the intra-class scatter matrix Sb:

[0067]

[0068] Sw=Sw+λI

[0069]

[0070] Wherein:

[0071]

[0072] The eigenvalue decomposition is performed on Sw -1 and Sb to obtain the eigenvector e i and the eigenvalue v i The eigenvalue is composed of the eigenvectors corresponding to the C-1 largest eigenvalues, and the stable projection matrix W=(e1, e2, …e C-1 ).

[0073] Step S4.3: Based on the projection matrix W obtained above and the template feature mapping output of step S3.2, Assuming that the output of the character image to be recognized through the backbone network is X t , the distance to the linear decision hyperplane is:

[0074]

[0075] Wherein:

[0076]

[0077] Use P C ′=1 / (1+e -d ) Calculate the probability of each class, and use P C =P C ′ / ∑P i 'Further normalize, and finally use argmaxP i The class with the highest probability is determined as the category of the sample. In this example, the predicted value for each class is [0,0,1,0,0,0,0,0,0,0]. The highest probability is obtained by the argmax function, which is the third position, corresponding to the third class of the label, i.e., the number 2.

[0078] This invention provides a device for handwritten character recognition based on template-centered deep discriminant analysis, the device comprising:

[0079] The handwritten character acquisition module is used to acquire images of handwritten characters to be recognized.

[0080] The feature extraction module takes the handwritten characters to be recognized as input into a pre-built discrimination model for feature extraction and obtains feature vectors.

[0081] The character recognition module calculates the probability of each class based on the obtained feature vector according to the decision function, and takes the value with the highest probability as the character recognition result.

[0082] Table 1 shows a comparison between the Template Centered Discriminant Analysis (TLDA) loss function and the softmax cross-entropy loss function (CCE) of this invention, demonstrating their accuracy on the training and test sets. The results of this invention are superior, highlighting its beneficial effects.

[0083] Table 1

[0084]

Claims

1. A handwritten character recognition method based on template-centered deep discriminant analysis, characterized in that, Includes the following steps: Step S1: Construct a handwritten character training set, and preset the number of handwritten character types to C; Step S2: Construct a set of printed character templates using C printed character images, which is the same number as the number of handwritten character types; Step S3: Use the backbone network to extract features from the images in the printed character template set and the handwritten character training set, and construct a template-centered deep discriminant analysis model. Take the images in the printed character template set and the handwritten character training set as input, extract the features of the images as output, train the template-centered deep discriminant analysis model, and use gradient backpropagation to update the backbone network parameters until the loss function decrease rate is lower than the set threshold, then the trained template-centered deep discriminant analysis model is obtained. The specific steps of step S3 are as follows: Step S3.1: Randomly select n images from the training set to form a batch. Assume that the number of images belonging to the i-th character class is... One sample; Step S3.2, the feature output of the printed character template set after passing through the backbone network is denoted as... , The feature output of each image class in a batch after passing through the backbone network is denoted as... ; Step S3.3: Calculate the inter-class scatter matrix based on the above features. and intra-class scatter matrix : in: This represents the feature output of the j-th sample belonging to the i-th class. This represents the feature output of the template belonging to the i-th class; For the calculated Increasing the identity matrix by a factor of λ yields a new within-class matrix. ; Step S3.4: Directly to Eigenvalues ​​are obtained by performing eigenvalue decomposition. and eigenvectors Take C-1 largest eigenvalues, and then use k smallest eigenvalues ​​to construct the loss function expression: The loss value of a batch is calculated based on the loss function L, and the backbone network parameters are updated by backpropagation of gradients. Step S3.5: Repeat steps S3.1 to S3.4 until the loss function decrease rate is lower than the set threshold to obtain the trained template-centered deep discriminant analysis model; Step S4: Use the trained template-centered deep discriminant analysis model to recognize the new character image.

2. The handwritten character recognition method based on template-centered deep discriminant analysis according to claim 1, characterized in that, In step S1, a handwritten character training set is constructed using the MNIST dataset, with 10 different character types.

3. The handwritten character recognition method based on template-centered deep discriminant analysis according to claim 1, characterized in that, The specific steps of step S4 are as follows: Step S4.1: Based on the trained template-centered deep discriminant analysis model, calculate the feature mapping output of all training set samples, assuming each class has N. i One, denoted as Template feature map output With step S3.2 Consistent; Step S4.2: Calculate the inter-class scatter matrix and intra-class scatter matrix : in: This represents the feature output of the j-th sample belonging to the i-th class. This represents the feature output of the template belonging to the i-th class; right Perform eigenvalue decomposition to obtain eigenvectors. and eigenvalues These are eigenvalues, and a stable projection matrix is ​​formed by the eigenvectors corresponding to the C-1 largest eigenvalues. ; Step S4.3: Based on the projection matrix obtained above And the template feature mapping output of step S3.2, Assuming the output of the image of the character to be recognized after passing through the backbone network is The distance to the linear decision hyperplane is: in: It is the transpose of the projection matrix. It is the normal vector of the decision hyperplane; use Calculate the probability of each class, and then... Further normalization, and finally using The class with the highest probability is determined as the category of the sample.

4. A system for handwritten character recognition based on template-centered deep discriminant analysis, characterized in that, It includes a handwritten character acquisition module, a feature extraction module, and a character recognition module, and executes the handwritten character recognition method based on template-centered deep discriminant analysis as described in claim 1 through the above modules; The handwritten character acquisition module is used to acquire the handwritten characters to be recognized; The feature extraction module inputs the handwritten characters to be recognized into a pre-built discrimination model for feature extraction to obtain feature vectors; The character recognition module calculates the probability of each class based on the obtained feature vector according to the decision function, and takes the value with the highest probability as the character recognition result.

Images