Character recognition device, character recognition method, and program

Abstract

To provide a character recognition apparatus, character recognition method, and program capable of detecting cancellation and reflecting it in a detection result.SOLUTION: A character recognition apparatus of the embodiment comprises an image acquisition unit, a character score calculation unit, a cancellation score calculation unit, a correction unit, and a decoding unit. The image acquisition unit acquires an image in which characters are written. The character score calculation unit calculates a character class score representing a character class like to be recognized for a frame in which the image acquired by the image acquisition unit is divided in a predetermined direction. The cancellation score calculation unit calculates a cancellation score representing a cancellation character like for the frame. The correction unit corrects the character class score so that the higher the cancellation score is, the smaller the character class score is. The decoding unit estimates a plausible recognition result based on the character class score for each frame after correction.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] An embodiment of the present invention relates to a character recognition device, a character recognition method, and a program. [Background technology]

[0002] There are data entry systems that convert character strings handwritten or printed on paper forms into digital data. Such data entry systems digitize the form by performing character recognition processing on the image in the form using a character recognizer that uses, for example, optical character recognition (OCR) technology. A user may strike through character strings that have been handwritten or printed. Since a character string with a strikethrough indicates that the user has cancelled the string, even if the character string is recognized in the character recognition processing, the recognition result must be rejected. For this reason, technology to detect strikethroughs has been developed. However, conventional technology only detects strikethroughs, and manual corrections during digitization are required. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 4693825 [Patent Document 2] JP 2020-119289 A Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide a character recognition device, a character recognition method, and a program that are capable of detecting cancellation and reflecting it in the detection result. [Means for solving the problem]

[0005] A character recognition device according to an embodiment includes an image acquisition unit, a character score calculation unit, a cancellation score calculation unit, a correction unit, and a decoding unit. The image acquisition unit acquires an image on which characters are written. The character score calculation unit calculates a character class score representing a likelihood of a character class to be recognized for each frame obtained by dividing the image acquired by the image acquisition unit in a predetermined direction. The cancellation score calculation unit calculates a cancellation score representing a likelihood of the frame being a cancellation character for the image acquired by the image acquisition unit. The correction unit corrects the character class score such that the character class score becomes relatively smaller for a frame having a higher cancellation score. The decoding unit estimates a likely recognition result based on the character class score for each frame after the correction. [Brief description of the drawings]

[0006] [Figure 1] 1 is a block diagram showing an example of the configuration of a character recognition device according to a first embodiment. [Diagram 2] FIG. 13 is a diagram showing an example of character recognition when a character is not erased or corrected. [Diagram 3] 11A and 11B are diagrams showing examples of recognition results before and after correction in a case where cancellation has been performed. [Figure 4] 4 is a flowchart showing an example of a processing procedure of the character recognition device according to the first embodiment. [Diagram 5] 10 is a flowchart showing an example of a processing procedure of a character recognition device according to a second embodiment. [Figure 6] 11A and 11B are diagrams for explaining the advantage of changing weights at the time of correction for each character type. [Figure 7] FIG. 13 is a block diagram showing an example of the configuration of a character recognition device according to a third embodiment. [Figure 8] FIG. 13 is a diagram for explaining an example of a decoding algorithm of a decoding unit according to the third embodiment. [Figure 9] 13 is a flowchart showing an example of a processing procedure of a character recognition device according to a third embodiment. [Figure 10] FIG. 13 is a block diagram showing an example of the configuration of a character recognition device according to a fourth embodiment. [Figure 11] 13 is a flowchart showing an example of a processing procedure of a character recognition device according to a fourth embodiment. [Figure 12] FIG. [Figure 13] 13 is a flowchart showing an example of a processing procedure of a character recognition device according to a fifth embodiment. [Figure 14] 13 is a flowchart showing an example of a processing procedure of a character recognition device according to a sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, a character recognition device, a character recognition method, and a program according to an embodiment will be described with reference to the drawings.

[0008] (First embodiment) First, an example of the configuration of a character recognition device will be described. Fig. 1 is a block diagram showing an example of the configuration of a character recognition device according to this embodiment. As shown in Fig. 1, the character recognition device 1 includes an image acquisition unit 2, a character score calculation unit 4, a revocation score calculation unit 5, a correction unit 6, a decoding unit 7, and an output unit 8. The character score calculation unit 4 includes a character recognition network 41. The revocation score calculation unit 5 includes a revocation likelihood prediction network 51.

[0009] The character recognition device 1 determines the canceled characters and reflects them in the character recognition result.

[0010] The image acquisition unit 2 acquires an image on which characters are written, and outputs the acquired image to the character score calculation unit 4 and the cancellation score calculation unit 5.

[0011] The character score calculation unit 4 inputs the image output by the image acquisition unit 2 to the character recognition network 41 and calculates the feature amount of the entire image. Next, the character score calculation unit 4 divides the feature amount of the entire image into frames of a number corresponding to the width of the input image. The character score calculation unit 4 divides the image at a predetermined ratio, for example, into an input image of 64 pixels in height and 8 pixels in width. After that, the character score calculation unit 4 calculates the character likelihood (character class score) M of class k (k is an integer of 2 or greater) in frame t (t is an integer of 1 or greater) from the feature amount of each divided frame. t,k Here, class k is, for example, A to Z in the alphabet. That is, for the image acquired by image acquisition unit 2, character score calculation unit 4 calculates a character class score that indicates the likelihood of the recognition target being a character class for each frame divided in a predetermined direction. Furthermore, character score calculation unit 4 inputs the image output by image acquisition unit 2 to character recognition network 41, and calculates the blank likelihood (blank score) B of the blank class in frame t. t The character score calculation unit 4 calculates the character likelihood M t,k and the blank likelihood B t and is output to the correction unit 6.

[0012] The character recognition network 41 calculates the character likelihood M t,k The character recognition network 41 also outputs a blank likelihood B t The character recognition network 41 has previously learned the character likelihood and the blank likelihood using character images and blank images.

[0013] The revocation score calculation unit 5 inputs the image output by the image acquisition unit 2 to the revocation likelihood prediction network 51, calculates the feature amount of the entire image, and divides it into feature amounts of the same number of frames as the character score calculation unit 4. Then, the revocation score calculation unit 5 calculates the revocation likelihood (revocation score) R tThat is, the cancellation score calculation unit 5 calculates a cancellation score representing the likelihood of a frame being a cancelled character. The character score calculation unit 4 calculates the cancellation likelihood R t is output to the correction unit 6.

[0014] The cancellation likelihood prediction network 51 calculates the cancellation likelihood R, which is the likelihood that the character included in the input image is cancelled, in an image in which the character is cancelled. t The cancellation likelihood prediction network 51 has already learned the cancellation likelihood using an image in which characters have been cancelled.

[0015] The correction unit 6 receives the character likelihood M t,k and the blank likelihood B t , and the cancellation likelihood R t The correction unit 6 inputs the cancellation likelihood R t Based on the character likelihood M t,k After correcting, the corrected character likelihood M' t,k In this way, the correction unit 6 corrects the character class scores so that the character class scores become relatively smaller for frames with higher cancellation scores. The correction unit 6 calculates the blank likelihood B t and the corrected character likelihood for each frame M' t,k and output to the decoding unit 7.

[0016] The decoding unit 7 converts the blank likelihood B t and the character likelihood M' t,k and calculates a character string with a high likelihood (score) based on a predetermined rule. The calculated character string candidates may be one or more in descending order of likelihood, and may be two or more. The calculated character string may be one character. The predetermined rule is, for example, to delete consecutive labels and blanks. The decoding unit 7 may take into account knowledge processing such as bigrams and word lists when calculating the character string with the highest likelihood.

[0017] The output unit 8 outputs the recognized character string (or characters) to an external device, reflecting the result calculated by the decoding unit 7. The external device is, for example, an image display device, a mobile terminal, etc. The output information is the character string excluding the characters struck out with a strikethrough, that is, information on the character string that is not struck out.

[0018] Here, an example of the correction method performed by the correction unit 6 will be described. The correction unit 6 calculates the character likelihood M t,k Correct the following.

[0019]

number

[0020] In equation (1), α is a constant between 0 and 1. The meaning of this equation is that the revocation score (likelihood of revocation R t ) has a higher character class score (character likelihood M t,k The character class scores are adjusted so that the difference between the

[0021] Alternatively, the correction unit 6 may use, for example, the following equation (2) to calculate the character likelihood M t,k Correct the following.

[0022]

number

[0023] In formula (2), α is a constant in the range of 0≦α≦1, β is a constant in the range of 0≦β≦1, and u is a step function. When formula (2) is used, the exponent part is a discrete value, so the state after correction is set according to the discrete value.

[0024] Next, an overview of the character recognition process of this embodiment will be described with reference to Fig. 2 and Fig. 3. Fig. 2 is a diagram showing an example of character recognition when characters are not erased and no correction is performed. Note that in each of the following embodiments, an example will be described in which an image on which characters are written (or printed) is divided into frames in the horizontal direction as an example of a predetermined direction.

[0025] 2, "A" and "B" are written (or printed) on an input image g10. In this example, an example will be described in which the input image g10 is divided horizontally into three frames f1 to f3 for processing.

[0026] As shown in g20, the result of character recognition (including blank recognition) for each frame, for frame f1, the character likelihood g21 of class A is highest, followed by the blank likelihood g23, and the character likelihood g22 of class B is lowest. For frame f2, the blank likelihood is highest, followed by the character likelihood of class B, and the character likelihood of class A is lowest. For frame f3, the character likelihood of class B is highest, followed by the character likelihood of class A, and the blank likelihood is lowest.

[0027] In such a case, "AB" is output, which is the recognition result g30 obtained by removing the blanks from "A", the blanks, and "B", which have the highest likelihood in frame order. In this way, if cancellation is not performed, characters can be properly recognized without correction.

[0028] Next, an example of the recognition result before and after correction when cancellation is performed will be described. Fig. 3 is a diagram showing an example of the recognition result before and after correction when cancellation is performed. In the example of Fig. 3, "A" and "B cancelled with two horizontal lines" are written (or printed) in the input image g50.

[0029] As shown in g50, which is a result of performing character recognition (including blank recognition) and cancellation prediction on each frame, for frame f1, the character likelihood g61 of class A is the highest, followed by the blank likelihood g63, then the character likelihood g63 of class B, and the cancellation likelihood g64 is the lowest. For frame f2, the blank likelihood and cancellation likelihood are equally high, and the character likelihood of class B and the character likelihood of class A are equally low. For frame f3, the cancellation likelihood and the character likelihood of class B are equally high, and the blank likelihood and the character likelihood of class A are equally low.

[0030] Thus, in the pre-correction state, it is difficult to determine whether frame f2 is cancelled or blank. For this reason, in this embodiment, a correction g66 is performed on the likelihood of each class using the cancellation likelihood. As shown in the correction result g70, for frame f1, the character likelihood g71 of class A is the highest, followed by the blank likelihood g73, and then the character likelihood g73 of class B. For frame f2, the blank likelihood is the highest, and the character likelihoods of class B and class A are equally low. For frame f3, the blank likelihood is the highest, and the character likelihoods of class B and class A are equally low.

[0031] As a result, the character recognition device 1 outputs "A" having the highest likelihood in frame order, a blank, and "A" which is the recognition result g80 obtained by removing the blank from the blank. In this way, even if cancellation has been performed, the character can be appropriately recognized by performing correction according to this embodiment. In the example shown in Fig. 3, the character is crossed out with a double line, but the crossing out is not limited to this and may be blacked out, crossed out with a diagonal line, etc. The cancellation likelihood prediction network 51 learns such various cancellation states in advance.

[0032] Next, a description will be given of an example of a processing procedure of the character recognition device 1 according to this embodiment. Fig. 4 is a flowchart showing an example of a processing procedure of the character recognition device according to this embodiment.

[0033] The image acquisition unit 2 acquires an image on which text is written (step S11).

[0034] The character score calculation unit 4 and the cancellation score calculation unit 5 calculate the feature amount of the entire input image, and divide it into feature amounts the number of which corresponds to the width of the input image (step S12).

[0035] The character score calculation unit 4 calculates the character likelihood M of class k in frame t from the divided features. t,k and the blank likelihood B of the blank class in frame t t (Step S13).

[0036] The cancellation score calculation unit 5 calculates the cancellation likelihood R t is calculated (step S14).

[0037] The correction unit 6 calculates the cancellation likelihood R t Based on the character likelihood M t,k After correcting, the corrected character likelihood M' t,k is calculated (step S15).

[0038] The decoding unit 7 converts the blank likelihood B t and the character likelihood M' t,k and estimate a character string with a high likelihood based on a predetermined rule (step S16).

[0039] The output unit 8 outputs the recognized character string, reflecting the result estimated by the decoding unit 7, to an external device (step S17).

[0040] As described above, in this embodiment, the character likelihood for each class is corrected using the character likelihood and cancellation likelihood for each class. Also, in this embodiment, the character class score is corrected so that the character class score becomes relatively smaller for a frame with a higher cancellation score. As a result, according to this embodiment, it is possible to determine the canceled characters and reflect them in the recognition result.

[0041] Second Embodiment Next, a second embodiment will be described. The configuration example of the character recognition device 1 of this embodiment is the same as that of the first embodiment. In this embodiment, the weight of correction is changed for each character type during correction.

[0042] Next, a description will be given of an example of a processing procedure of the character recognition device 1 in this embodiment. Fig. 5 is a flowchart showing an example of a processing procedure of the character recognition device according to this embodiment.

[0043] The character recognition device 1 performs the processes of steps S11 to S14 in the same manner as in the first embodiment.

[0044] Next, the correction unit 6 sets different intensities for each class and calculates the cancellation likelihood R t Based on the character likelihood M t,k After correcting, the corrected character likelihood M' t,k is calculated (step S21).

[0045] Next, the character recognition device 1 performs the processes of steps S16 to S17 in the same manner as in the first embodiment.

[0046] Here, an example of the correction performed by the correction unit 6 in step S21 will be described. The correction unit 6 suppresses erroneous deletion of characters including horizontal lines by setting different strengths for each class using, for example, the following formula (3). Characters including horizontal lines that are easily recognized erroneously include, for example, "￥" (en mark) and "ki" (katakana "ki").

[0047]

number

[0048] In formula (3), α for each character class k k is 0≦α k It is a constant less than or equal to 1. The correction unit 6 adjusts the constant α k Make it larger.

[0049] Here, the constant α k An example of how to set this will be explained. The correction unit 6 calculates the cancellation likelihood for each image of one character of each character type using the following formula (4) to obtain a constant α k Set up.

[0050]

number

[0051] In formula (4), R t (x k ) is the image x of class k k In equation (4), the likelihood of cancellation is calculated when N images x k,1 ,…,x k,N Alternatively, the average value, maximum value, median value, etc. may be used.

[0052] Here, the advantage of changing the weighting strength at the time of correction for each character type will be further explained. Fig. 6 is a diagram for explaining the advantage of changing the weighting strength at the time of correction for each character type. The example in Fig. 6 is an example of processing and likelihood when performing character recognition on an input image g101 of "￥", "3", "0", and "cancel 3" to recognize it as "￥30".

[0053] Table g111 shows the likelihood of each frame when the weights at the time of correction are not changed. The horizontal direction corresponds to frames f1 to f4, and the vertical direction corresponds to the likelihood M t,1 , the likelihood of the character "3" M t,2 , the character likelihood of "0" M t,3 , the blank likelihood B t , and the revocation likelihood R t As shown in Table g111, if the weights at the time of correction are not changed, the cancellation likelihood R t will be high, resulting in erroneous recognition.

[0054] Table g112 shows the weights for the correction of "￥" and "3" as α ￥ The horizontal direction of table g112 corresponds to frames f1 to f4, and the vertical direction is the likelihood of the character "￥" M t,1, the likelihood of the character "3" M t,2 , the character likelihood of "0" M t,3 , and the blank likelihood B t In the case of table g112, the recognition result of image g101 after correction is that the likelihood of "￥30" is 0.009 (=0.1×0.09) and the likelihood of "30" is 0.01 (=0.1×0.1). In this example, if the weights at the time of correction are the same, the first candidate will be "30" and the "￥" will be deleted and the recognition will be erroneous.

[0055] Table g113 shows the weighting for the correction of "￥" as α ￥ = 0.9, and the weighting for "3" is changed to α3 = 0.1 for correction. The horizontal and vertical directions of table g113 are the same as those of table g113. In the case of table g113, the recognition result of image g101 after correction is that the likelihood of "￥30" is 0.081 (= 0.81 x 0.1), and the likelihood of "30" is 0.01 (= 0.1 x 0.1). As in this example, by appropriately setting the weighting for correction, it is possible to prevent "￥" from being deleted as the first candidate, by setting the weighting for correction appropriately.

[0056] As described above, in this embodiment, the character likelihood for each class is corrected by setting different intensities for each class. As a result, according to this embodiment, it is possible to control correction for character types that are easily mistaken for cancellation by a horizontal line, for example, and therefore it is possible to reduce erroneous recognition.

[0057] (Third embodiment) Next, a third embodiment will be described. In this embodiment, each image is input, and the likelihood (score) is calculated for each character, not for each frame, to estimate the most likely character.

[0058] Fig. 7 is a block diagram showing an example of the configuration of a character recognition device according to this embodiment. As shown in Fig. 7, character recognition device 1A includes image acquisition unit 2, character score calculation unit 4A, revocation score calculation unit 5A, decoding unit 7A, and output unit 8. Character score calculation unit 4A includes character recognition network 41. Revocation score calculation unit 5A includes revocation likelihood prediction network 51.

[0059] The character score calculation unit 4A inputs the image information output by the image acquisition unit 2 to the character recognition network 41 and calculates the character likelihood (score) M k The character score calculation unit 4A calculates the calculated character likelihood M k to the decoding unit 7A. The character recognition device 1A extracts an image for each character (each class) from the image by a known method.

[0060] The revocation score calculation unit 5A inputs the image information output by the image acquisition unit 2 to the revocation likelihood prediction network 51 and calculates the revocation likelihood (score) R of the class. k The character score calculation unit 4A calculates the calculated cancellation likelihood R t to the decoding unit 7A.

[0061] The decoding unit 7A calculates the character likelihood M k and the revocation likelihood R t and estimate the most likely character.

[0062] Next, an example of the decoding algorithm of the decoding section 7A will be described. FIG. 8 is a diagram for explaining an example of a decoding algorithm of the decoding section 7A according to the present embodiment. In this embodiment, the score without cancellation is M k Let the score with cancellation be γM k R k (where γ is a constant equal to or greater than 1.) The example in Fig. 8 is a recognition example when an image of "￥" is input.

[0063] The decoding unit 7A calculates the score when all classes are cancelled and the score when all classes are not cancelled, and outputs the scores in descending order. k and the revocation likelihood R calculated by the revocation score calculation unit 5A. t Calculation results are shown below.

[0064] Table g202 shows the candidates with the first to fourth highest scores when γ is 2.0 and 10.0. As shown in table g202, when γ is 2.0, the order of highest score is no cancellation of “￥”, cancellation of “Y”, no cancellation of “Y”, and cancellation of “￥”. Also, as shown in table g202, when γ is 10.0, the order of highest scores is "Y" with cancellation, "￥" without cancellation, "Y" without cancellation, and "￥" with cancellation.

[0065] In this manner, in this embodiment, the decoding unit 7A calculates the score without cancellation by multiplying the weighting coefficient γ and the character likelihood M k and the cancellation likelihood R k Then, based on the score with cancellation and the score without cancellation, a likely character or cancellation is estimated.

[0066] Next, a description will be given of an example of a processing procedure of the character recognition device 1 in this embodiment. Fig. 9 is a flowchart showing an example of a processing procedure of the character recognition device according to this embodiment.

[0067] The image acquiring unit 2 acquires an image with characters written thereon (for example, image g211) (step S31).

[0068] The character score calculation unit 4A inputs the image information output by the image acquisition unit 2 to the character recognition network 41 and calculates the character likelihood M k is calculated (step S32).

[0069] The cancellation score calculation unit 5A inputs the image information output by the image acquisition unit 2 to the cancellation likelihood prediction network 51 to calculate the cancellation likelihood Rt is calculated (step S33).

[0070] The decoding unit 7A calculates the score without cancellation by the weighting factor γ and the character likelihood M k and the cancellation likelihood R k is calculated, and a likely character or cancellation is estimated based on the score with cancellation and the score without cancellation (step S34).

[0071] The output unit 8 outputs the recognized character string, reflecting the result calculated and estimated by the decoding unit 7A, to an external device (step S35).

[0072] In this embodiment, a score with cancellation and a score without cancellation are calculated for each character, and based on the calculated score and the weighting coefficient γ, a likely character and cancellation information are estimated and output. As a result, according to this embodiment, it is possible to appropriately recognize whether the acquired image is a character or a strikeout.

[0073] (Fourth embodiment) Next, a fourth embodiment will be described. In this embodiment, a character string is estimated based on the corrected character likelihood as in the first embodiment, and a character string that is not expected to be cancelled is also estimated based on the character likelihood before correction and the blank likelihood, and the estimation results are compared to perform character recognition.

[0074] Fig. 10 is a block diagram showing an example of the configuration of a character recognition device according to this embodiment. As shown in Fig. 10, character recognition device 1B includes image acquisition unit 2, character score calculation unit 4, revocation score calculation unit 5, correction unit 6, decoding unit 7B, comparison unit 9, and output unit 8B. Character score calculation unit 4 includes character recognition network 41. Revocation score calculation unit 5 includes revocation likelihood prediction network 51.

[0075] The decoder 7B calculates the corrected character likelihood M' t、k and the blank likelihood B tBased on the character likelihood M t、k and the blank likelihood B t Based on this, we estimate a likely string when cancellation is not assumed.

[0076] The comparison unit 9 compares the corrected character likelihood M' t、k and the blank likelihood B t The character string estimated based on and the character likelihood M t、k and the blank likelihood B t and compare the estimated string based on the

[0077] The output unit 8B outputs the redundant character string before the cancellation is reflected and the cancellation information to an external device based on the result of the comparison by the comparison unit.

[0078] Next, an example of the processing procedure of the character recognition device 1B in this embodiment will be described with reference to the flowchart of FIG 11.

[0079] The image acquiring unit 2 acquires an image with characters written thereon (for example, image g311) (step S41).

[0080] Subsequently, the character recognition device 1B performs the processes of steps S12 to S16 in the same manner as in the first embodiment.

[0081] Next, the decoding unit 7B calculates the character likelihood M t、k and the blank likelihood B t Based on this, the decoding unit 7B estimates a likely character string in the case where cancellation is not assumed (step S42). Note that the decoding unit 7B may perform the process of step S42 in parallel with steps S12 to S16, or may perform the process before steps S12 to S16.

[0082] The comparison unit 9 compares the corrected character likelihood M' t、k and the blank likelihood B t The character string estimated based on and the character likelihood M t、k and the blank likelihood B tThe comparison unit 9 compares the character string estimated based on ...

[0083] Based on the result of the comparison by the comparison unit, the output unit 8B outputs the redundant character string before the cancellation is reflected and the cancellation information to an external device (step S44). t、k and the blank likelihood B t The estimated character string is “￥30” based on the character likelihood M t、k and the blank likelihood B t If the character string estimated based on the above is "¥300", the output unit 8B outputs "¥300" together with the fact that the fourth character is estimated to be a cancellation.

[0084] As described above, in this embodiment, a character string is estimated based on the corrected character likelihood, and a character string that is not intended to be canceled is also estimated based on the character likelihood before correction and the blank likelihood, and the estimation results are compared to perform character recognition. As a result, according to this embodiment, by presenting such recognition results to, for example, the user who wrote the handwriting, it is possible to provide clues as to whether recognition was correct or not, or how to correct the recognition if it was incorrect.

[0085] Fifth embodiment Next, a fifth embodiment will be described. In each of the above-mentioned embodiments, the cancellation by two horizontal lines has been described as an example of cancellation, but the cancellation method includes a strikethrough line g401, a blackout g402, a diagonal line g403, and a cross mark g404 as shown in Fig. 12. Fig. 12 is a diagram showing an example of cancellation.

[0086] As shown in Fig. 12, when multiple cancellations are expected, five classes of cancellations are required: no cancellation, cancellation by strikethrough, cancellation by blackening, cancellation by diagonal line, and cancellation by cross, rather than two classes of cancellations with and without cancellation, as in the case of Fig. 12. For this reason, in this embodiment, recognition is performed using three or more classes for cancellation. Note that the configuration example of the character recognition device 1 of this embodiment is the same as that of the first embodiment.

[0087] The revocation score calculation unit 5 inputs the image output by the image acquisition unit 2 to the revocation likelihood prediction network 51 and calculates the feature amount of the entire image. Next, the feature amount of the entire image is divided into frames of a number corresponding to the width of the input image. After that, the character score calculation unit 4 calculates the revocation likelihood (score) R for frame t and revocation class j from the feature amount of each divided frame. t、j The character score calculation unit 4 calculates the cancellation likelihood R t、j is output to the correction unit 6.

[0088] The correction unit 6 calculates the cancellation likelihood R t、j Based on the character likelihood M t,k After correcting, the corrected character likelihood M' t,k Calculate.

[0089] Here, an example of a sewing method performed by the correction unit 6 will be described. The correction unit 6 sets different thresholds and weights for each cancellation class j using, for example, the following equation (5). For example, the weight of the blackened cancellation class is lowered (β j (Make larger).

[0090]

number

[0091] In equation (5), u is a step function and β j is the weight for each cancellation class j.

[0092] Next, an example of the processing procedure of the character recognition device according to this embodiment will be described with reference to the flowchart of FIG 13.

[0093] The character recognition device 1 performs the processes of steps S11 to S13 in the same manner as in the first embodiment.

[0094] The cancellation score calculation unit 5 calculates the cancellation likelihood R t、j is calculated (step S51).

[0095] The correction unit 6 calculates the cancellation likelihood R t、j Based on the character likelihood M t,k After correcting, the corrected character likelihood M' t,k is calculated (step S52).

[0096] Subsequently, the character recognition device 1 performs the processes of steps S16 to S17 in the same manner as in the first embodiment.

[0097] The method of this embodiment may be combined with the method of the second embodiment in which the weight at the time of correction is changed for each character type.

[0098] As described above, in this embodiment, the cancellation likelihood is calculated using cancellation classes in accordance with a plurality of cancellation patterns. As a result, according to this embodiment, it is possible to improve the accuracy of cancellation likelihood estimation and the explainability.

[0099] Sixth embodiment Next, a sixth embodiment will be described. The configuration example of the character recognition device 1 of this embodiment is the same as that of the first embodiment. In this embodiment, recognition is performed taking into consideration the cancellation likelihood of adjacent frames, based on the fact that whether an adjacent frame is cancelled or not changes frequently.

[0100] During the correction, the correction unit 6 calculates, for example, the cancellation likelihood R tIn the above, smoothing is performed using a Gaussian function with respect to t. Alternatively, during correction, the correction unit 6 performs an OR operation using the following formula (6) on adjacent frames, for example.

[0101]

number

[0102] Next, an example of the processing procedure of the character recognition device according to this embodiment will be described with reference to the flowchart of FIG.

[0103] The character recognition device 1 performs the processes of steps S11 to S14 in the same manner as in the first embodiment.

[0104] The correction unit 6 takes into account adjacent frames during correction and calculates the cancellation likelihood R t Based on the character likelihood M t,k After correcting, the corrected character likelihood M' t,k is calculated (step S61).

[0105] Subsequently, the character recognition device 1 performs the processes of steps S16 to S17 in the same manner as in the first embodiment.

[0106] As described above, in this embodiment, the likelihood of cancellation of adjacent frames is also taken into consideration for recognition. As a result, according to this embodiment, it is possible to improve the accuracy of the cancellation determination.

[0107] In the above embodiments, examples of recognizing the cancellation of characters have been described, but the present invention is not limited to this. What is written (or printed) may be a symbol, a mathematical formula, etc. Thus, in the present embodiment, characters include, for example, alphabets, numbers, symbols, hiragana, katakana, kanji, etc.

[0108] In some of the above embodiments, the character score calculation unit 4 calculates the likelihood B tIn the above description, the decoding unit 7 calculates the likelihood of a character string based on the corrected character likelihood and the cancellation likelihood, but the estimation may be performed without using the blank likelihood.

[0109] Furthermore, in the above embodiment, an example has been described in which an image on which characters are written (or printed) is divided into frames in the horizontal direction, but the direction of division may be, for example, the vertical direction.

[0110] In each of the above embodiments, the character recognition device 1 (1A, 1B) is a hardware functional unit such as an LSI, but may be a software functional unit.

[0111] According to at least one of the embodiments described above, it is possible to determine a cancelled character and reflect it in the recognition result by having the cancellation score calculation unit 5. Also, according to at least one of the embodiments described above, it is possible to determine a cancelled character and reflect it in the recognition result by correcting the character likelihood for each class using the character likelihood for each class and the cancellation likelihood by having the correction unit 6.

[0112] In addition, a program for realizing all or part of the functions of the character recognition device 1 (1A, 1B) of the present invention may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to perform all or part of the processing of the character recognition device 1 (1A, 1B). Note that the term "computer system" here includes hardware such as an OS and peripheral devices. The term "computer system" also includes a WWW system equipped with a homepage providing environment (or display environment). The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. The term "computer-readable recording medium" also includes those that hold a program for a certain period of time, such as volatile memory (RAM) inside a computer system that becomes a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line.

[0113] The above program may also be transmitted from a computer system in which the program is stored in a storage device or the like to another computer system via a transmission medium, or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The above program may also be for realizing part of the above-mentioned functions. Furthermore, it may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0114] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0115] 1, 1A, 1B... character recognition device, 2... image acquisition unit, 4, 4A... character score calculation unit, 5, 5A... cancellation score calculation unit, 6... correction unit, 7, 7A, 7B... decoding unit, 8... output unit, 9... comparison unit, 41... character recognition network, 51... cancellation likelihood prediction network

Claims

1. an image acquisition unit for acquiring an image having characters written thereon; a character score calculation unit that calculates a character class score representing a likelihood of a character class of a recognition target for each frame obtained by dividing the image acquired by the image acquisition unit in a predetermined direction; a cancellation score calculation unit that calculates a cancellation score representing a likelihood of the frame being a cancelled character; a correction unit that corrects the scores of the character classes so that the scores of the character classes become relatively smaller for frames having higher cancellation scores; a decoding unit that estimates a likely recognition result based on the corrected character class score for each frame; A character recognition device comprising:

2. an image acquisition unit for acquiring an image having characters written thereon; a character score calculation unit that calculates a character class score representing a likelihood of each character class to be recognized for the image acquired by the image acquisition unit; a cancellation score calculation unit that calculates a cancellation score representing a likelihood of a cancellation character for each of the character classes; a decoding unit for estimating a likely recognition result based on the character class score and the revocation score; A character recognition device comprising:

3. the correction unit corrects the character class score by using the character class score and the cancellation score.

2. The character recognition device according to claim 1.

4. the correction unit sets a weight for correction for each type of the character to be recognized.

4. The character recognition device according to claim 3.

5. A comparison unit is further provided, The decoding unit estimates a likely recognition result in a case where cancellation is not assumed based on the character class score; the comparison unit compares a likely recognition result estimated based on the corrected character class score with a likely recognition result estimated based on the character class score in a case where cancellation is not assumed, and estimates a redundant recognition result and cancellation information before cancellation reflection based on a result of the comparison.

4. The character recognition device according to claim 3.

6. The revocation comprises a plurality of revocation classes; the cancellation score calculation unit calculates a cancellation class score representing a likelihood of a canceled character for the frame and the cancellation class; the correction unit corrects the character class score by using the character class score and the cancellation class score.

2. The character recognition device according to claim 1.

7. the correction unit corrects the character class score by using the character class score and the cancellation score between adjacent frames.

2. The character recognition device according to claim 1.

8. The character score calculation unit calculates a blank score for a blank between characters, the decoding unit estimates a likely recognition result based on the corrected character class score and the blank score.

8. The character recognition device according to claim 1, 3, 4, 6, or 7.

9. The character score calculation unit calculates a blank score for a blank between characters, the decoding unit estimates a likely recognition result based on the character class score and the blank score.

3. The character recognition device according to claim 2.

10. The image acquisition unit acquires an image having characters written thereon, a character score calculation unit calculates a character class score representing a likelihood of a character class of a recognition target for each frame obtained by dividing the image acquired by the image acquisition unit in a predetermined direction; a cancellation score calculation unit calculates a cancellation score representing a likelihood of the frame being a cancelled character; a correction unit corrects the scores of the character classes so that the scores of the character classes become relatively smaller for frames having higher cancellation scores; a decoding unit that estimates a likely recognition result based on the corrected character class score for each frame; Character recognition method.

11. The image acquisition unit acquires an image having characters written thereon, a character score calculation unit calculates a character class score representing a likelihood of each character class to be recognized for the image acquired by the image acquisition unit; a cancellation score calculation unit calculates a cancellation score representing a likelihood of a cancellation character for each of the character classes; a decoding unit that estimates a likely recognition result based on the character class score and the revocation score; Character recognition method.

12. On the computer, Obtain an image with text written on it, Calculating a character class score representing a likelihood of the character class being recognized for each frame obtained by dividing the acquired image in a predetermined direction; Calculating a cancellation score representing a likelihood of the frame being a cancelled character; correcting the character class scores so that the character class scores become relatively smaller for frames with higher cancellation scores; estimating a likely recognition result based on the corrected character class scores for each frame; program.

13. On the computer, Obtain an image with text written on it, calculating a character class score representing a likelihood of each character class to be recognized for the acquired image; calculating a cancellation score representing a likelihood of a cancellation character for each of the character classes; A plausible recognition result is estimated based on the character class score and the cancellation score. program.

Images