Character recognition device, character recognition method, and program

Abstract

To provide a character recognition device, character recognition method and program which further improve recognition accuracy after rejection processing.SOLUTION: A character recognition device 10 comprises a neural network which receives an input of an input image showing a character and outputs an identification score for each class indicating the character identified from the input image. The neural network comprises: a feature extraction unit 12 which extracts a feature vector indicating a feature of the character from the input image; and an identification unit 13 which calculates an identification score for each class by using a secondary identification function of the feature vector or a secondary identification function of the feature vector normalized to a unit length.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] Embodiments of the present invention relate to a character recognition device, a character recognition method, and a program. [Background technology]

[0002] For example, character recognition devices have been known for recognizing character types by extracting individual characters from an image, digitizing them as feature vectors using a feature extractor, calculating a recognition score for each target class (character type) using a classifier, and comparing the magnitudes of these recognition scores. Furthermore, techniques for rejecting unreliable character recognition results based on recognition scores have also been known. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 51-97340 [Patent Document 2] Japanese Patent Application Publication No. 4-58358 [Patent Document 3] Japanese Patent Application Publication No. 4-275690 [Patent Document 4] Japanese Patent Application Publication No. 4-294445 [Patent Document 5] Japanese Patent Publication No. 2010-39778 [Patent Document 6] Japanese Patent Publication No. 2012-181568 [Non-patent literature]

[0004] [Non-Patent Document 1] Yoshiaki Kurosawa, "Subspace Methods Derived from Spherical Gaussian Distributions," IEICE Transactions on Electronics, Information and Communication Engineers, Vol. J81-D2, No. 6, pp. 1205-1212 (1998). [Non-Patent Document 2] D. Ciresan and U. Meier, “Multi-Column Deep Neural Networks for offline handwritten Chinese character classification”, 2015 International Joint Conference on Neural Networks (IJCNN), 2015, pp. 1-6 [Non-Patent Document 3] Hideki Nakayama, “Image Feature Extraction and Transfer Learning Using Deep Convolutional Neural Networks”, The Institute of Electronics, Information and Communication Engineers, Audio Research Society, July 2015 [Non-Patent Document 4] C.M. Bishop, “Pattern Recognition and Machine Learning (Volumes 1 & 2)”, translated by Hiroshi Motoda et al., Maruzen Publishing Co., Ltd., 2012 [Non-Patent Document 5] E. Oya, “Pattern Recognition and Subspace Methods”, translated by Hideaki Ogawa et al., Sangyo Tosho Publishing Co., Ltd., 1986 [Summary of the Invention] [Problems to be Solved by the Invention]

[0005] In recent years, due to the improvement of neural network technology called so-called deep learning, the recognition accuracy of character recognition devices without reject processing has been significantly improved. However, with conventional technologies including neural networks, it has been difficult to further improve the recognition accuracy after reject processing. [Means for Solving the Problems]

[0006] The character recognition device according to the embodiment includes a neural network that receives an input of an input image indicating a character and outputs an identification score for each class indicating the character identified from the input image. The neural network includes a feature extraction unit that extracts a feature vector indicating the features of the character from the input image, and an identification unit that calculates the identification score for each class using a quadratic discriminant function of the feature vector or a quadratic discriminant function of the feature vector normalized to a unit length. [Brief Description of the Drawings]

[0007] [Figure 1] Figure 1 illustrates the processing images of classification using the subspace method and classification using a neural network. [Figure 2] Figure 2 is an illustrative diagram showing an example where patterns are concentrated near the identification boundary. [Figure 3A] Figure 3A is an illustrative diagram illustrating the recognition accuracy after rejection processing (in the case of the subspace method). [Figure 3B] Figure 3B is an illustrative diagram illustrating the recognition accuracy after rejection processing (in the case of a CNN). [Figure 4] Figure 4 shows an example of the functional configuration of the character recognition device according to the first embodiment. [Figure 5] Figure 5 is a flowchart showing an example of the character recognition process in the first embodiment. [Figure 6] Figure 6 shows an example of training data from the first embodiment. [Figure 7] Figure 7 shows an example of a neural network that implements the feature extraction unit 12 and the identification unit 13 of the first embodiment. [Figure 8] Figure 8 shows examples of the hardware configuration of the character recognition device according to the first to fourth embodiments. [Modes for carrying out the invention]

[0008] Embodiments of the character recognition device, character recognition method, and program will be described in detail below with reference to the attached drawings.

[0009] For example, Non-Patent Document 1 assumes a feature extractor that is manually designed, such as a gradient feature. Non-Patent Document 1 describes a series of quadratic classifiers (i.e., classifiers using quadratic forms) obtained by modeling the distribution of character patterns for each class on the feature vector space as a Gaussian distribution or an approximation thereof.

[0010] These discriminator methods include, for example, methods that model each class as belonging to a sub-vector space within the feature vector space (so-called subspace methods), and methods that apply ridge regularization to the estimation of the covariance matrix of a Gaussian distribution (so-called pseudo-Bayesian methods, also known as the Modified Quadratic Discriminant Function (MQDF)). Hereafter, these will be collectively referred to as "subspace method-based methods."

[0011] Furthermore, as described in Non-Patent Document 2, there is a method using a Convolutional Neural Network (CNN) for classifiers. The Convolutional Neural Network method employs a linear classifier as the classifier and trains the feature extractor and classifier together.

[0012] Generally, methods using convolutional neural networks are considered to be more accurate than methods using subspace methods. Linear classifiers are elementary and simple classifiers, so it is thought that training a feature extractor suitable for classification contributes significantly to their classification accuracy.

[0013] Furthermore, in the practical application of character recognition devices, in order to ensure reliability suitable for the field of use, there is a process to reject a character if the device determines that it cannot output a reliable recognition result for that character. For example, as described in Patent Document 1, there is a rejection process that utilizes the identification score for each class of the input character.

[0014] Furthermore, as described in Patent Documents 2 to 5, there are processes that utilize projection onto a subspace in the feature vector space as a preprocessing step for recognition and learning by neural networks.

[0015] Figure 1 illustrates the processing images of classification using the subspace method and classification using a neural network.

[0016] First, input unit 1 receives the input character 101.

[0017] Next, the feature extraction unit 2 extracts feature vectors from character 101 that represent the characteristics of character 101. In the subspace method, feature vectors are extracted by a pre-designed extractor (manual extraction). In the neural network, feature vectors are extracted by a learning-based extractor (learning-based extraction).

[0018] Next, the identification unit 3 identifies characters based on the feature vectors. In the subspace method, characters are identified by a quadratic classifier. In the neural network, characters are identified by a linear classifier. The example in Figure 1 shows that the identification score for character 101a is 0.6 and the identification score for character 101b is 0.35.

[0019] Next, the output unit 4 outputs the character identification result (recognition result) based on the identification score. In the example in Figure 1, the character 101a with the highest identification score is output as the identification result. Note that in the example in Figure 1, the output unit 4 does not perform any rejection processing of the identification result. Rejection processing, which rejects the identification result, is performed, for example, when the identification score is smaller than a predetermined threshold.

[0020] Subspace methods are inferior to convolutional neural network methods in terms of recognition accuracy. On the other hand, in convolutional neural network methods, the feature vector space obtained through learning tends to concentrate patterns (feature vectors) near the discrimination boundary more easily than the feature vector space obtained by manually designed feature extractors.

[0021] Figure 2 is an illustrative diagram showing an example where patterns are concentrated near the discrimination boundary. In the feature vector space, Figure 2 shows an example where patterns are concentrated at the discrimination boundary between the class representing the character "0" and the class representing the character "1". As shown in Figure 2, in the convolutional neural network (CNN) method, the frequency of close proximity of discrimination score values ​​for multiple classes is higher compared to subspace-based methods, and the recognition accuracy after rejection processing using discrimination scores cannot surpass that of subspace-based methods.

[0022] Figure 3A is an illustrative diagram illustrating the recognition accuracy after rejection processing (in the case of the subspace method). Region 111a indicates the region where the feature vector is recognized as character 101a. Region 111b indicates the region where the feature vector is recognized as character 101b. The discrimination boundary 112 indicates the discrimination boundary between character 101a and character 101b.

[0023] Figure 3B is an illustrative diagram to explain the recognition accuracy after rejection processing (in the case of CNN). Region 121a indicates the region where the feature vector is recognized as character 101a. Region 121b indicates the region where the feature vector is recognized as character 101b. The discrimination boundary 122 indicates the discrimination boundary between characters 101a and 101b. As shown in Figure 3B, in the case of CNN, patterns tend to concentrate around the discrimination boundary 122. Therefore, if we include the indeterminate processing results that were rejected by the rejection process for patterns present in region 123, the recognition accuracy in the case of CNN will be lower than the recognition accuracy in the case of the subspace method.

[0024] Furthermore, as described in Patent Documents 2 to 5, there are techniques that utilize projection onto subspaces in the feature vector space during the recognition and learning process using neural networks. However, since the subspace is not learned as part of the neural network, these techniques cannot be expected to separate each class in the feature vector space. Therefore, compared to subspace-based methods, more correct patterns must be rejected to reduce misreadings, and high accuracy after the rejection process cannot be expected.

[0025] (First Embodiment) In the first embodiment, a character recognition device, character recognition method, and program are described, which include a neural network incorporating a subspace method as a classifier, and an approximation method for learning it in a practical amount of time.

[0026] [Overview of the prefecture] In the character recognition device of the first embodiment, the classifier used in the identification unit of a typical convolutional neural network is replaced from a linear classifier to a quadratic classifier used in subspace method systems. This combines the high recognition accuracy achieved through feature extraction training, which is an advantage of convolutional neural networks, with the inter-class separation advantage of subspace method systems, making it possible to obtain recognition accuracy after rejection processing that surpasses the recognition accuracy of conventional technologies.

[0027] [Example of functional configuration] Figure 4 shows an example of the functional configuration of the character recognition device 10 of the first embodiment. The character recognition device 10 of the first embodiment includes an input unit 11, a feature extraction unit 12, an identification unit 13, and an output unit 14.

[0028] The input unit 11 receives an image separated into individual characters as the input image. The characters to be recognized can be arbitrary. For example, the characters may include hiragana, katakana, kanji, the alphabet, numbers, and symbols.

[0029] The feature extraction unit 12 extracts feature vectors that represent the features of the input image by converting the input image into feature vectors.

[0030] The identification unit 13 performs operations on the feature vector and calculates an identification score for each class of recognition target (for example, a class representing each character to be recognized).

[0031] The output unit 14 determines the character recognition result (recognition result) and whether or not a character was rejected due to the rejection process based on the recognition score, and outputs the recognition result and whether or not a character was rejected.

[0032] [Example of character recognition processing] Figure 5 is a flowchart showing an example of character recognition processing in the first embodiment. First, the input unit 11 receives an image separated into character units as the input image (step S101). Next, the feature extraction unit 12 extracts feature vectors that represent the features of the input image by converting the input image received through the processing in step S1 into feature vectors (step S102).

[0033] Next, the identification unit 13 performs calculations on the feature vectors extracted by the processing in step S102 to calculate an identification score for each class (step S103).

[0034] Next, the output unit 14 determines the recognition result from the identification score and further determines whether or not to perform a rejection judgment based on the setting information provided before the start of processing (step S104). The setting information is information indicating whether or not to perform a rejection judgment. If the setting information indicates that a rejection judgment should be performed, the rejection process described above is carried out.

[0035] If a rejection decision is made (Step S105, Yes), the output unit 14 makes a rejection decision based on the identification score (Step S105). If no rejection decision is made (Step S105, No), the process proceeds to Step S106.

[0036] Next, the output unit 14 outputs output information that includes at least the recognition result (step S106). Specifically, if a rejection determination is made, the output unit 14 outputs output information that includes the recognition result and the rejection determination result, and if no rejection determination is made, it outputs output information that includes the recognition result.

[0037] Furthermore, at this time, it is determined whether the correct answer expected for the recognition result is available based on the configuration information given before processing begins (step S107). The configuration information indicates, for example, a learning mode in which the correct answer is available as training data, or an estimation mode in which the character represented by an unknown input image for which there is no correct answer is estimated.

[0038] Figure 6 shows an example of training data for the first embodiment. The training data for the first embodiment includes training images and ground truth images. The training images are input images that represent characters and are associated with ground truth images. The ground truth images represent the correct characters corresponding to the training images. For example, the ground truth images may be the name of the ground truth class that represents the correct characters in the processing result of the character recognition process.

[0039] Returning to Figure 5, if the correct answer is not available (Step S107, No), the process ends. If the correct answer is available (Step S107, Yes), the output unit 14 passes the correct answer to the feature extraction unit 12 and the identification unit 13. The feature extraction unit 12 and the identification unit 13 calculate weight update values ​​from the correct answer and update the weights based on the weight update values ​​(Step S108).

[0040] [Details of each part] Figure 7 shows an example of a neural network 130 that implements the feature extraction unit 12 and the discrimination unit 13 of the first embodiment. The neural network in the upper part of Figure 7 is a prior art (Non-Patent Literature 3) that represents the processing of the feature extraction unit 12. The lower part of Figure 7 shows the neural network 130 of the first embodiment. In the neural network 130 of the first embodiment, a part of the neural network in the upper part of Figure 7 (the part other than the discrimination unit (fully connected layer and discrimination layer)) is used as a feature extractor.

[0041] The feature extraction unit 12 calculates feature vectors from the input image using a feature extractor implemented by a neural network 130. Details of the processing of the neural network that implements the feature extractor are described, for example, in Non-Patent Literature 2.

[0042] The identification unit 13 calculates the recognition score for each class c from the feature vector output by the feature extraction unit 12 using a quadratic discriminant function. Here, the quadratic discriminant function is a function given to each class c in the form of equation (1) below for the feature vector x.

[0043]

number

[0044] Alternatively, a quadratic discriminant function refers to a function given for each class c in the form of equation (3) below, for a feature vector represented by the following equation (2) after normalizing the feature vector x to a unit length.

[0045]

number

number

[0046] These functions are generalizations of the discriminant function forms of the subspace method system shown in Non-Patent Document 1, P c This corresponds to the covariance matrix or a constant multiple thereof when the distribution of the feature vectors of the class is estimated as a Gaussian distribution. The name of the quadratic discriminant function is that these discriminant functions are P c This originates from the fact that it contains a feature vector x whose coefficient matrix is ​​, or a term in the quadratic form of the feature vector represented by equation (2) above.

[0047] In a typical neural network, the discriminant function is the linear discriminant function given by equation (4) below, which is a key difference between this embodiment and conventional technology.

[0048]

number

[0049] Furthermore, in this embodiment, the symbols of the discriminant functions in equations (1) and (3) above are determined such that a larger value makes the hypothesis that the input character belongs to that class more plausible. In the following, the quadratic discriminant function is assumed to be the functional form of equation (1) or (3) above, but the discriminant unit can be constructed using the same procedure even if a function that gives exactly or approximately equivalent discriminant results is used, such as a function obtained by giving the function values ​​of these functional forms to a monotonically increasing function, or a function to which a small correction is made to any term.

[0050] The output unit 14 determines the class with the highest identification score, i.e., the value of the identification function calculated by the identification unit 13, as the identification result. Furthermore, if rejection processing was enabled in the configuration information provided before processing began, the output unit 14 performs a rejection judgment based on the identification score of each class.

[0051] As a method for rejection determination based on the identification score, for example, the method described in Patent Document 1 can be used. Next, the output unit 14 outputs the recognition result and the rejection determination result if a rejection determination has been made, and outputs the recognition result if no rejection determination has been made. At this time, it also determines whether the correct answer expected in the recognition result is available based on the setting information given before the start of processing, and if the correct answer is not available, it terminates the processing, and if it is available, it passes the correct answer to the feature extraction unit 12 and the identification unit 13.

[0052] The following describes how to handle situations where the correct answer is available.

[0053] The identification unit 13, which has been returned to the processing unit, calculates the loss function and its gradient based on the correct answer passed from the output unit 14, sends the gradient to the feature extraction unit 12, and updates the weights of the identification unit 13 based on the gradient. Here, the loss function is the identification function f of equation (1) above. c Alternatively, the discriminant function g of equation (3) above cWhen it is defined using [a certain method] and the discriminant function is a function obtained by adding a constant to a constant multiple of the log-likelihood, like the discriminant function appearing in Non-Patent Document 1, the loss function of the prior art can be used as it is. For example, when using the softmax cross entropy loss function shown in Non-Patent Document 4, the loss function is defined as in the following equations (5) to (7).

[0054] [Number] [Number] [Number]

[0055] However, X = (x1, ···, x N ) represents the entire set of feature vectors, and Y = (y1, ···, y N ) is the correct answer corresponding to X. Θ = (θ1, ···, θ C ), and θ c = (P c , q c , r c ) are the parameters of the discriminant function, that is, the weights of the discriminant unit 13. p c is the log-likelihood, and a and b are constants for converting the discriminant function value to the log-likelihood.

[0056] In the above example, the discriminant function uses the form of f c . Also, R is a regularization term. As methods for updating the weights, the stochastic gradient method shown in Non-Patent Document 4, and improved methods of the stochastic gradient method in the prior art, etc. can be used. As the regularization term, in addition to the obvious example of setting R to 0, conventional techniques such as the L p regularization shown in Non-Patent Document 4 can be used.

[0057] The feature extraction unit 12 updates its own weights based on the gradient input from the discrimination unit 13. That is, the weights of the neural network 130 are determined by optimizing a loss function based on the correct character class represented by the training image and the output of the neural network 130 when the training image is input to the neural network 130. By obtaining the weights of the neural network 130 through learning, a feature extractor and a discriminator with the desired properties can be concretely realized.

[0058] As for specific weight update methods, for example, the backpropagation method shown in Non-Patent Document 4, and improved methods of the backpropagation method in the prior art can be used.

[0059] Furthermore, in the first embodiment, the discriminant function is defined as f in the definition of the loss function. c Although we used g c The loss function can be similarly defined for discriminant functions of the form shown.

[0060] As described above, the character recognition device 10 of the first embodiment includes a neural network 130 that receives an input image showing characters and outputs a class-specific identification score for the characters identified from the input image. The neural network 130 includes a feature extraction unit 12 and an identification unit 13. The feature extraction unit 12 extracts feature vectors that show the characteristics of the characters from the input image. The identification unit 13 calculates a class-specific identification score using a quadratic discriminant function of the feature vectors, or a quadratic discriminant function of the feature vectors normalized to a unit length.

[0061] In other words, the character recognition device 10 of the first embodiment combines the advantages of a convolutional neural network, namely high recognition accuracy due to the learning of the feature extractor, with the advantages of a quadratic discriminant function system used as a classifier, namely interclass separation.

[0062] As a result, the character recognition device 10 of the first embodiment can further improve the recognition accuracy after rejection processing. Specifically, by replacing the identification unit 13 with a linear classifier such as a conventional CNN with a subspace method or a derived method, the patterns of each class to be recognized can be represented in a subvector space within the feature vector space. Therefore, the phenomenon of the pattern distribution concentrating at the identification boundary in the feature vector space and the problem of identification scores being close together between classes are mitigated. As a result, the accuracy of rejection processing using the identification score is improved, and the recognition accuracy after rejection processing can be further improved compared to conventional methods.

[0063] (Second Embodiment) Next, a second embodiment will be described. In the description of the second embodiment, explanations similar to those of the first embodiment will be omitted, and the differences from the first embodiment will be described.

[0064] [Overview of the prefecture] In the second embodiment, constraints are placed on the form of the discriminant function of the first embodiment, and its weights are replaced with different representations as needed. This allows the use of a general derived method of the subspace method system as the discriminator, thereby improving recognition accuracy by preventing overfitting.

[0065] The functional configuration of the character recognition device 10 in the second embodiment is the same as in the first embodiment (see Figure 4). Furthermore, the overall flow diagram showing the character recognition process in the second embodiment is also the same as in the first embodiment (see Figure 5).

[0066] [Details of each part] The details of the feature extraction unit 12 and the output unit 14 are the same as in the first embodiment.

[0067] In the identification unit 13 of the second embodiment, the parameter P in the identification function of the first embodiment c ,q c ,r c ,s c and t c Instead of making all of them learnable weights, certain constraints are placed on them. For example, the discriminant function gc The discriminant function form can be restricted to the subspace method by imposing the constraint condition in equation (8) below.

[0068]

number

[0069] Also, the discriminant function g c By imposing the constraint condition in equation (9) below, the discriminant function form can be restricted to a pseudo-Bayesian method.

[0070]

number

[0071] At this time, as seen in Non-Patent Document 1, parameter P c This can be seen as a low-rank approximation of the covariance matrix of the distribution of the feature vectors of the class in question, or a constant multiple thereof. Furthermore, the parameter U c D is a matrix consisting of eigenvectors associated with the top eigenvalues ​​of the covariance matrix. Top eigenvalues ​​are the eigenvalues ​​with the larger values ​​among the multiple eigenvalues ​​of the covariance matrix. c This is a diagonal matrix determined by the eigenvalues ​​of the covariance matrix.

[0072] Here, the method described in Non-Patent Document 1 is D c The components of this matrix are equivalent to replacing the lower eigenvalues ​​of a diagonal matrix obtained by arranging the above eigenvalues ​​in descending order with constants. However, other low-rank approximation methods can also be used, such as simply adding a constant multiple of the identity matrix to a diagonal matrix obtained by arranging the above eigenvalues ​​in descending order.

[0073] Under these constraints, the weights actually learned are the parameters P of the quadratic discriminant function. c ,q c ,r c ,s c and t c Rather, a portion of the parameter U that appears in the above constraints is used. c and D cIt can be done as follows: Also, for example, P c =expΛ c The antisymmetric matrix Λ such that c parameter P c It can also be used as a substitute.

[0074] In the second embodiment, the value of the quadratic discriminant function is calculated using a weighted sum of the inner products of the feature vector or normalized feature vector and the eigenvectors of the quadratic coefficient matrix in the quadratic discriminant function. The coefficients of the weighted sum are determined using the eigenvalues ​​of the coefficient matrix. The coefficient matrix included in the quadratic discriminant function that identifies each class is determined based on a covariance matrix that shows the distribution of the feature vectors belonging to each class.

[0075] In this case, as with the first embodiment, if the weights are updated using conventional techniques such as the stochastic gradient method, it is not necessarily guaranteed that the updated weights will satisfy nonlinear constraints such as those shown in equation (10) below.

[0076]

number

[0077] In such cases, a weight correction process that satisfies the constraints may be added to the weight update process, for example, by the learning subspace method shown in Non-Patent Document 5, the averaging learning subspace method, and the method shown in Patent Document 6. Alternatively, the weight update method may be modified so that the constraints are automatically and strictly or approximately satisfied.

[0078] As explained above, in the second embodiment, constraints are placed on the form of the discriminant function of the first embodiment, and a general derivation method for quadratic discriminant functions is used. This allows the use of a general derivation method of the subspace method system as the discriminator, and the effect of improving recognition accuracy by preventing overfitting can be obtained.

[0079] (Third embodiment) Next, the third embodiment will be described. In the description of the third embodiment, explanations similar to those of the first or second embodiment will be omitted, and only the differences from the first or second embodiment will be described.

[0080] [Overview of the prefecture] In the third embodiment, a regularization term equivalent to the constraints imposed on the discriminant function is added to the loss function of the second embodiment. As a result, the constraints are automatically approximately satisfied even when conventional techniques are used for updating the weights, thereby improving the learning speed.

[0081] The functional configuration of the character recognition device 10 in the third embodiment is the same as in the first embodiment (see Figure 4). Furthermore, the overall flow diagram showing the character recognition process in the third embodiment is also the same as in the first embodiment (see Figure 5).

[0082] [Details of each part] The details of the feature extraction unit 12 and the output unit 14 are the same as in the second embodiment.

[0083] In the identification unit 13 of the third embodiment, a regularization term corresponding to a nonlinear constraint is added to the loss function of the second embodiment. For example, as a regularization term corresponding to the constraint in equation (10) above, the regularization term of equation (11) below is provided.

[0084]

number

[0085] By adding a regularization term (equation (11) above) to the loss function that evaluates the degree to which each of the multiple eigenvectors of the covariance matrix is ​​orthonormal to each other, the constraints can be automatically and approximately satisfied even when the weights are updated using conventional techniques such as stochastic gradient methods, as in the first embodiment. This avoids the increase in computational complexity that would result from adding a weight correction process to the weight update process to satisfy the constraints, or from modifying the weight update method so that the constraints are automatically satisfied. In this case, the weight correction may be performed only when the weight update is completed or at an arbitrarily determined point during the update, so that the constraints are strictly satisfied at that point.

[0086] As explained above, in the third embodiment, a regularization term equivalent to the constraints added to the discriminant function is added to the loss function of the second embodiment. As a result, even when using conventional techniques for updating weights, the constraints are automatically approximately satisfied, and the learning speed can be improved.

[0087] (Fourth Embodiment) Next, the fourth embodiment will be described. In the description of the fourth embodiment, explanations similar to those of the first to third embodiments will be omitted, and the differences from the first to third embodiments will be described.

[0088] [Overview of the prefecture] In the fourth embodiment, the regularization term of the third embodiment is extended so that the distributions of each class diverge in the feature vector space. This further improves the classification accuracy, including rejection.

[0089] The functional configuration of the character recognition device 10 in the fourth embodiment is the same as in the first embodiment (see Figure 4). Furthermore, the overall flow diagram showing the character recognition processing in the fourth embodiment is also the same as in the first embodiment (see Figure 5).

[0090] [Details of each part] The details of the feature extraction unit 12 and the output unit 14 are the same as in the third embodiment.

[0091] In the identification unit 13 of the fourth embodiment, the regularization term in the loss function of the third embodiment is modified to separate the dictionary subspaces corresponding to each class. For example, the constraint condition in equation (10) above is that the pattern of class c is in the dictionary subspace Im(U c U c T This means that the distribution is concentrated around ). Therefore, in order to separate the dictionary subspaces between different classes, the regularization term is modified to equation (12) below.

[0092]

number

[0093] The loss function of the fourth embodiment includes a regularization term that evaluates the degree to which the top eigenvectors corresponding to a predetermined number of top eigenvalues, selected in descending order of eigenvalue magnitude from among the multiple eigenvalues ​​of the covariance matrix, are orthonormal to each other among the classes corresponding to their respective covariance matrices.

[0094] As a result, the spaces spanned by the top eigenvectors corresponding to the top eigenvalues ​​of each class are approximately orthogonal to each other. Therefore, in addition to the effects of the third embodiment, as learning progresses, the dictionary subspaces approach a state where they are orthogonal to each other and automatically begin to diverge.

[0095] As explained above, according to the fourth embodiment, the dictionary subspaces corresponding to different classes in the learned feature vector space diverge from each other and approach a state of orthogonality. As a result, the difference in discrimination scores increases, which further improves recognition accuracy, including rejections.

[0096] Finally, examples of the hardware configuration of the character recognition device 10 according to the first to fourth embodiments will be described.

[0097] [Example hardware configuration] Figure 8 shows examples of the hardware configuration of the character recognition device 10 according to the first to fourth embodiments. The character recognition device 10 according to the first to fourth embodiments comprises a processor 201, a main memory 202, an auxiliary storage device 203, a display device 204, an input device 205, and a communication device 206. The processor 201, main memory 202, auxiliary storage device 203, display device 204, input device 205, and communication device 206 are connected via a bus 210.

[0098] Note that the character recognition device 10 may not be equipped with some of the above-described components. For example, if the character recognition device 10 can utilize the input and display functions of an external device, the character recognition device 10 may not be equipped with a display device 204 and an input device 205.

[0099] The processor 201 executes the program read from the auxiliary storage device 203 into the main memory device 202. The main memory device 202 is memory such as ROM and RAM. The auxiliary storage device 203 is such as an HDD (Hard Disk Drive) and a memory card.

[0100] The display device 204 is, for example, a liquid crystal display. The input device 205 is an interface for operating the character recognition device 10. The display device 204 and the input device 205 may be implemented by a touch panel or the like that has both display and input functions. The communication device 206 is an interface for communicating with other devices.

[0101] The program executed by the character recognition device 10 is recorded in an installable or executable file format on a computer-readable storage medium such as a CD-ROM, memory card, CD-R, or DVD and provided as a computer program product.

[0102] Alternatively, the program executed by the character recognition device 10 may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Alternatively, the program executed by the character recognition device 10 may be provided via a network such as the Internet without requiring downloads.

[0103] Alternatively, the program for the character recognition device 10 may be pre-installed and provided in ROM or the like.

[0104] The program executed by the character recognition device 10 has a modular configuration that includes functions that can also be implemented by the program, as shown in the functional configuration of Figure 4 above. In actual hardware terms, each of these functions is loaded onto the main memory 202 by the processor 201 reading and executing the program from the storage medium. In other words, each of the above function blocks is generated on the main memory 202.

[0105] It should be noted that some or all of the functions shown in Figure 4 above may be implemented using hardware such as ICs instead of software.

[0106] Alternatively, multiple processors 201 may be used to implement each function, in which case each processor 201 may implement one of the functions, or two or more of the functions.

[0107] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of symbols]

[0108] 1 Input section 2. Feature extraction unit 3. Identification Unit 4 Output section 10 Character recognition device 11 Input section 12 Feature Extraction Unit 13 Identification unit 14 Output section 130 Neural Networks 201 Processor 202 Main storage 203 Auxiliary storage device 204 Display device 205 Input device 206 Communication equipment 210 Bus

Claims

1. The system includes a neural network that accepts an input image representing characters and outputs a class-specific identification score for each character identified from the input image. The aforementioned neural network is A feature extraction unit extracts feature vectors representing the characteristics of the characters from the input image, The system includes an identification unit that calculates an identification score for each class using a quadratic discriminant function of the feature vector, or a quadratic discriminant function of the feature vector normalized to a unit length, The value of the quadratic discriminant function is calculated using a weighted sum of the inner products of the feature vector or the normalized feature vector and the eigenvectors of the quadratic coefficient matrix in the quadratic discriminant function, and the coefficients of the weighted sum are determined using the eigenvalues ​​of the coefficient matrix. The coefficient matrix is ​​determined based on a covariance matrix that shows the distribution of the feature vectors belonging to each of the classes. A character recognition device equipped with the following features.

2. The weights of the neural network are determined by optimizing a loss function based on the correct character class represented by the training image and the output of the neural network when the training image is input to the neural network. The character recognition device according to claim 1.

3. The coefficient matrix is ​​included in the quadratic discriminant function that identifies each of the aforementioned classes, The character recognition device according to claim 1.

4. The loss function includes a regularization term that evaluates the degree to which each of the multiple eigenvectors of the covariance matrix is ​​orthonormal to each other. The character recognition device according to claim 2.

5. The loss function includes a regularization term that evaluates the degree to which the top eigenvectors corresponding to a predetermined number of top eigenvalues ​​selected in descending order of magnitude from among the eigenvalues ​​of the covariance matrix are orthonormal to each other within the classes corresponding to each covariance matrix. The character recognition device according to claim 2.

6. An output unit that outputs the character recognition result based on the aforementioned identification score. A character recognition device according to any one of claims 1 to 5, further comprising:

7. The output unit performs a rejection process, which does not output the character recognition result, if the identification score is less than the threshold. The character recognition device according to claim 6.

8. A character recognition method for a character recognition device comprising a neural network that accepts an input image showing characters and outputs an identification score for each class showing the characters identified from the input image, The steps include extracting a feature vector representing the characteristics of the characters from the input image, The process includes the step of calculating a class-specific discrimination score using a quadratic discriminant function of the feature vector, or a quadratic discriminant function of the feature vector normalized to a unit length, The value of the quadratic discriminant function is calculated using a weighted sum of the inner products of the feature vector or the normalized feature vector and the eigenvectors of the quadratic coefficient matrix in the quadratic discriminant function, and the coefficients of the weighted sum are determined using the eigenvalues ​​of the coefficient matrix. The coefficient matrix is ​​determined based on a covariance matrix that shows the distribution of the feature vectors belonging to each of the classes. Character recognition method.

9. Computers, It functions as a neural network that accepts an input image representing characters and outputs a class-specific identification score for each character identified from the input image. The aforementioned neural network is A feature extraction unit extracts feature vectors representing the characteristics of the characters from the input image, The system includes an identification unit that calculates an identification score for each class using a quadratic discriminant function of the feature vector, or a quadratic discriminant function of the feature vector normalized to a unit length, The value of the quadratic discriminant function is calculated using a weighted sum of the inner products of the feature vector or the normalized feature vector and the eigenvectors of the quadratic coefficient matrix in the quadratic discriminant function, and the coefficients of the weighted sum are determined using the eigenvalues ​​of the coefficient matrix. The coefficient matrix is ​​determined based on a covariance matrix that shows the distribution of the feature vectors belonging to each of the classes. program.

Images