Information processing device and program

Abstract

To increase character recognition accuracy for a document including a numeric character.SOLUTION: A character recognition device 10, an example of an information processing device, includes: a numeric character recognition unit 126 which recognizes a numeric character included in an image as a subject to be processed; and a symbol recognition unit 127 which recognizes a symbol included in the image as the subject to be processed. The symbol recognition unit 127 recognizes partial images C1, C4, C8 corresponding to the symbol from a paragraph image P4 showing numeric characters and other symbols. The numeric character recognition unit 126 recognizes numeric characters included in the paragraph image P4A obtained by deleting the partial images C1, C4, C8 corresponding to the symbols from the paragraph image.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to an information processing device and a program. [Background technology]

[0002] In information processing devices such as character recognition devices, optical character recognition (OCR) technology is generally adopted. OCR technology is a technology that captures characters written on documents such as forms as images using optical means such as a camera or image scanner, and converts the characters in the captured image into character information (e.g., character code) that can be used by a computer or the like. For example, the following Patent Document 1 discloses a technology for generating an image from a processing target image that includes characters and lines, with lines removed in a set length direction. It is expected that the accuracy of recognizing characters included in the processing target image will be improved by removing lines that become noise in character recognition from the processing target image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-142628 A Summary of the Invention [Problem to be solved by the invention]

[0004] When performing character recognition on a document including numbers using an information processing device, symbols used in conjunction with the numbers, such as commas indicating digits and dots indicating decimal points, may be erroneously recognized as numbers or parts of numbers, resulting in a decrease in the accuracy of the recognition of the numbers. The above-mentioned conventional technology can remove the ruled lines in the length direction, but cannot prevent symbols used in conjunction with the numbers from being erroneously recognized as numbers, resulting in a problem that the decrease in the accuracy of the recognition of the numbers cannot be suppressed. [Means for solving the problem]

[0005] An information processing device according to a preferred embodiment of the present invention includes a generation unit that generates a second image from a first image representing numbers and symbols other than numbers by erasing a first portion corresponding to the symbols, and a number recognition unit that recognizes numbers included in the second image generated by the generation unit.

[0006] A program according to a preferred embodiment of the present invention causes a processor to function as a generation unit that generates a second image from a first image representing numbers and symbols other than numbers by erasing a first portion corresponding to the symbols, and a number recognition unit that recognizes numbers contained in the second image generated by the generation unit. Effect of the Invention

[0007] According to the present invention, it is possible to improve the accuracy of character recognition for documents that include numbers. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a configuration of a character recognition system according to an embodiment. [Diagram 2] FIG. 13 is a diagram showing an example of a receipt image. [Diagram 3] FIG. 11 is a diagram illustrating an example of recognition result data. [Figure 4] FIG. 2 is a block diagram showing an example of a hardware configuration of a terminal device. [Diagram 5] FIG. 2 is a block diagram showing an example of a functional configuration of a terminal device. [Figure 6] FIG. 2 is a block diagram showing an example of a hardware configuration of the character recognition device. [Figure 7] FIG. 2 is a block diagram showing an example of a functional configuration of a character recognition device. [Figure 8] FIG. 13 is a diagram showing an example of a receipt image after layout analysis. [Figure 9] 4A to 4C are diagrams illustrating a character recognition method of a second character recognition unit. [Figure 10] 4 is a flowchart showing a procedure of a character recognition process performed by the character recognition device. [Figure 11]13 is a flowchart showing a procedure of a second character recognition process performed by a second character recognition unit. [Figure 12] FIG. 11 is a diagram showing an example of recognition result data in the second embodiment. [Figure 13] 13A to 13C are diagrams illustrating a character recognition method of a second character recognition unit in a third embodiment. [Figure 14] FIG. 13 is a block diagram showing a functional configuration of a character recognition device according to a fourth embodiment. [Figure 15] FIG. 13 is a diagram showing an example of a receipt image to which the fourth embodiment is suitable. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. Note that the dimensions or scale of each part in the drawings may differ from the actual dimensions, and some parts are shown diagrammatically to facilitate understanding. Furthermore, the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description to the effect that the present invention is limited thereto.

[0010] In this embodiment, a notation refers to a line or a dot that is distinguished from the background in a document or an image obtained by scanning a document. A notation may refer to a character or a ruled line. A notation may also refer to a design, or may be a line or a dot that is unintentionally written by a writing implement in contact with the writing paper. In this embodiment, a character refers to a notation artificially defined using lines and dots to communicate and record words or language, such as kanji, hiragana, katakana, the alphabet, numbers, symbols, etc. Therefore, for example, lines, dots, and ruled lines unintentionally drawn by a writing implement in contact with writing paper are not included in characters. In this embodiment, the numerals are characters that indicate numbers. In this embodiment, the numerals refer to, for example, Arabic numerals (0, 1...9). In this embodiment, language characters are characters used to write languages, such as kanji, hiragana, katakana, the alphabet, etc. For example, dots, commas, periods, and commas are used together with language characters, but in this embodiment, dots, commas, periods, and commas are classified as symbols. In this embodiment, a symbol refers to a notation such as a dot, a comma, a period, a comma, a hyphen, an arithmetic symbol, etc., which is generally used in conjunction with a linguistic character or a number to indicate a specific meaning. Note that a symbol may be used alone.

[0011] [First embodiment] [Character Recognition System Overview] FIG. 1 is a diagram showing a configuration of a character recognition system 1 according to an embodiment. The character recognition system 1 includes a character recognition device 10 and a terminal device 20. The character recognition device 10 is an example of an information processing device. The character recognition device 10 and the terminal device 20 are connected via a network NW. The network NW may include the Internet and a local area network. For example, the network NW includes one or both of a wired network and a wireless network. In addition, the connection between the network NW and the character recognition device 10 may be, for example, a connection that enables communication between multiple elements, and may be a connection using either a wired or wireless connection, or a connection using both a wired and wireless connection.

[0012] Any information processing device connectable to a network NW can be used as the character recognition device 10. The character recognition device 10 executes a character recognition process that recognizes characters included in an image, for example, by using optical character recognition (OCR) technology. An example of a receipt image RI that is the subject of the character recognition process is described later in FIG. 2. The configuration of the character recognition device 10 is described later in FIG. 6 and FIG. 7.

[0013] Any information processing device connectable to the network NW can be adopted as the terminal device 20. Specifically, the terminal device 20 may be, for example, a stationary information device such as a personal computer, or a portable information terminal such as a notebook personal computer, a tablet terminal, or a smartphone. The configuration of the terminal device 20 will be described later with reference to Figs. 4 and 5.

[0014] In this embodiment, the character recognition device 10 is an information processing device owned by a business that provides a receipt digitization service. The terminal device 20 is an information processing device owned by a user who uses the receipt digitization service. The receipt digitization service is a service that performs character recognition processing on an image of a receipt (hereinafter referred to as a "receipt image") and converts the characters written on the receipt into computer-processable data. That is, in this embodiment, the image to be processed is a receipt image RI (see FIG. 2). The computer-processable data is, for example, text data such as Shift-JIS code. A user generates a receipt image RI by reading a receipt to be digitized with a scanner or by photographing it with a camera. The user transmits the receipt image RI to the character recognition device 10 using the terminal device 20. The character recognition device 10 performs character recognition processing on the receipt image RI transmitted from the terminal device 20, and transmits recognition result data RD (see FIG. 3) including the result of the character recognition processing to the terminal device 20. The user inputs the transmitted recognition result data into, for example, an accounting management application to manage expenses, etc. By using the receipt digitization service, the contents written on paper receipts are automatically digitized, which improves the efficiency of expense management compared to, for example, manually inputting the contents of receipts into a computer.

[0015] Fig. 2 is a diagram showing an example of a receipt image RI. The receipt image RI shown in Fig. 2 includes a document name notation N1, a date notation N2, a destination notation N3, an amount notation N4, a proviso notation N5, a receipt confirmation statement notation N6, an address notation N7, and an issuer name notation N8. Each notation N1 to N8 is composed of lines or dots written in a color that can be distinguished from the background color of the receipt. Hereinafter, in the receipt image RI, the color that displays the background color of the receipt is referred to as the background color, and the colors that display each notation N1 to N8 are referred to as the notation color.

[0016] The document name N1 is a notation that indicates that the document is a receipt. The date notation N2 is a notation that indicates the date the receipt was issued. The destination notation N3 is a notation that indicates the destination of the receipt. The amount notation N4 is a notation that indicates the amount (value) of money received. The proviso notation N5 is a notation that indicates the name of the product or service provided in exchange for money. The receipt confirmation notation N6 is a notation of wording that confirms that the amount stated in the amount notation N4 has been received under the name of the proviso notation N5. The address notation N7 is a notation that indicates the address of the issuer of the receipt. The issuer name notation N8 is a notation that indicates the name of the issuer of the receipt.

[0017] In the receipt image RI shown in FIG. 2, the document name N1 and the receipt confirmation statement N6 are written in type. In addition, the destination N3, the amount N4, the address N7, and the issuer name N8 are written in handwritten characters. The date N2 and the proviso N5 are a mixture of handwritten and typed characters. Generally, in receipt paper commercially available for handwriting, items common to all receipts are printed in type in advance, and items that differ depending on each receipt are written by hand. The receipt image RI shown in FIG. 2 is an image of a receipt with necessary parts handwritten on receipt paper commercially available for handwriting. Note that the receipt image RI shown in FIG. 2 is an example, and for example, all items in the receipt image may be printed in type, or all items in the receipt image may be handwritten.

[0018] FIG. 3 is a diagram showing an example of the recognition result data RD. The recognition result data RD shown in FIG. 3 shows the result of character recognition of the receipt image RI shown in FIG. 2. The recognition result data RD includes a date recognition result D1, a destination recognition result D2, an amount recognition result D3, a proviso recognition result D4, and an issuer recognition result D5. The date recognition result D1 shows the result of character recognition of the date notation N2. The destination recognition result D2 shows the result of character recognition of the destination notation N3. The amount recognition result D3 shows the result of character recognition of the amount notation N4. The proviso recognition result D4 shows the name of the product or service among the results of character recognition of the proviso notation N5. The issuer recognition result D5 shows the result of character recognition of the issuer name notation N8. Note that the recognition result data RD shown in FIG. 3 does not include the results of character recognition corresponding to the document name notation N1, the receipt confirmation statement notation N6, and the address notation N7. The document name notation N1 and the receipt confirmation statement notation N6 are omitted because they are self-evident in the receipt digitalization service. In addition, the address notation N7 is omitted because it is not necessary for expense management. The information to be transmitted as the recognition result data RD may be specified by the user. For example, the user may specify to the character recognition device 10 that the recognition result data RD should include the address notation N7.

[0019] [System Configuration] Next, the configurations of the terminal device 20 and the character recognition device 10 will be described.

[0020] 4 is a block diagram showing an example of a hardware configuration of the terminal device 20. The terminal device 20 includes a processor 22 that controls each unit of the terminal device 20, a memory 24 that stores various information, a communication device 26, an operation device 28, and a display device 29.

[0021] The memory 24 includes, for example, one or both of a volatile memory such as a random access memory (RAM) that functions as a working area for the processor 22 and a non-volatile memory such as an electrically erasable programmable read-only memory (EEPROM) that stores various information such as the control program PG2, and functions as a storage unit 240 (see FIG. 5) described later. The memory 24 may be detachable from the terminal device 20. Specifically, the memory 24 may be a storage medium such as a memory card that is detachable from the terminal device 20. The memory 24 may also be, for example, a storage device (e.g., online storage) communicatively connected to the terminal device 20 via a network NW or the like.

[0022] The memory 24 stores, for example, a control program PG2. In this embodiment, the control program PG2 includes, for example, an application program for transmitting a receipt image RI to the character recognition device 10 and obtaining recognition result data RD. The control program PG2 may include, for example, an operating system program for the processor 22 to control each part of the terminal device 20.

[0023] The processor 22 includes, for example, one or more CPUs (Central Processing Units). The processor 22 executes, for example, a control program PG2 stored in the memory 24 and operates in accordance with the control program PG2 to function as a control unit 220 (see FIG. 5) described later.

[0024] Furthermore, for example, when the processor 22 is configured to include a plurality of CPUs, some or all of the functions of the control unit 220 may be realized by the plurality of CPUs operating in cooperation with each other in accordance with a program such as the control program PG2. Furthermore, the processor 22 may be configured to include hardware such as a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array) in addition to one or more CPUs, or in place of some or all of the one or more CPUs. In this case, some or all of the control unit 220 realized by the processor 22 may be realized by hardware such as a DSP.

[0025] The communication device 26 is hardware for communicating with an external device located outside the terminal device 20 via one or both of a wired network and a wireless network, and functions as a communication unit 260.

[0026] The operation device 28 is hardware for accepting operations by the user of the terminal device 20, and functions as an operation unit 280. For example, the operation device 28 may be configured from one or more devices including some or all of an operation button, a touch panel, a keyboard, and a mouse.

[0027] The display device 29 is hardware for displaying various information to a user of the terminal device 20, and functions as a display unit 290. For example, the display device 29 may be a display included in the terminal device 20 or a display connected to the terminal device 20.

[0028] The hardware configuration of the terminal device 20 is not limited to the example shown in Fig. 4. For example, the terminal device 20 may have an optical device such as a camera or an image scanner for generating a receipt image RI from a receipt, which is a paper medium.

[0029] 5 is a block diagram showing an example of a functional configuration of the terminal device 20. The terminal device 20 has a control unit 220 that controls each unit of the terminal device 20, a storage unit 240 that stores various information, a communication unit 260 for executing communication with an external device such as the character recognition device 10, an operation unit 280 for accepting an operation by a user of the terminal device 20, and a display unit 290 for displaying various information.

[0030] The control unit 220 includes an image transmission unit 222 and a recognition result data receiving unit 224. The image transmission unit 222 transmits a receipt image RI to the character recognition device 10. The recognition result data receiving unit 224 receives the recognition result data RD from the character recognition device 10.

[0031] 6 is a block diagram showing an example of a hardware configuration of the character recognition device 10. The character recognition device 10 includes a processor 12 that controls each unit of the character recognition device 10, a memory 14 that stores various information, and a communication device 16.

[0032] Memory 14 includes, for example, one or both of a volatile memory such as a RAM that functions as a working area for processor 12 and a non-volatile memory such as an EEPROM that stores various information such as the control program PG1, and functions as a storage unit 140 (see FIG. 7) described below. Note that memory 14 may be detachable from character recognition device 10, similar to memory 24 of terminal device 20 described in FIG. 4, or may be a storage device (for example, online storage) communicatively connected to character recognition device 10 via a network NW or the like.

[0033] In this embodiment, the control program PG1 includes, for example, an application program for the character recognition device 10 to execute the character recognition process. The control program PG1 may include, for example, an operating system program for the control unit 120 to control each unit of the character recognition device 10.

[0034] The processor 12 is configured similarly to the processor 22 of the terminal device 20 described in Fig. 4. For example, the processor 12 includes one or more CPUs. The processor 12 executes a control program PG1 stored in the memory 14 and operates according to the control program PG1, thereby functioning as a control unit 120 (see Fig. 7) described later.

[0035] Also, for example, when the processor 12 is configured to include multiple CPUs, some or all of the functions of the control unit 120 may be realized by these multiple CPUs operating in cooperation with each other in accordance with a program such as the control program PG1. Also, the processor 12 may be configured to include hardware such as a GPU, a DSP, or an FPGA in addition to one or more CPUs, or in place of some or all of the one or more CPUs. In this case, some or all of the control unit 120 realized by the processor 12 may be realized by hardware such as a DSP.

[0036] The communication device 16 is hardware for communicating with an external device that exists outside the character recognition device 10 via one or both of a wired network and a wireless network, and functions as a communication unit 160 .

[0037] 7 is a block diagram showing an example of a functional configuration of the character recognition device 10. The character recognition device 10 has a control unit 120 that controls each unit of the character recognition device 10, a storage unit 140 that stores various information, and a communication unit 160 for executing communication with an external device such as a terminal device 20.

[0038] [Character recognition details] Below, we will explain each component of control unit 120 of character recognition device 10, and also explain the details of character recognition processing by character recognition device 10. Control unit 120 includes an image acquisition unit 121, a character area specification unit 122, a character type determination unit 123, a first character recognition unit 124, a second character recognition unit 125 (a number recognition unit 126, a symbol recognition unit 127, a symbol erasure unit 128, and an output unit 129), and a recognition result generation unit 130.

[0039] The image acquisition unit 121 acquires image data to be processed. In this embodiment, the image acquisition unit 121 receives the receipt image RI transmitted from the terminal device 20 via the network NW.

[0040] The character area specifying unit 122 divides the receipt image RI acquired by the image acquiring unit 121 into components such as characters and lines, specifies the areas to be read as characters for each paragraph, and divides the image into paragraph images P (P1 to P8). The process performed by the character area specifying unit 122 is generally called layout analysis. FIG. 8 is a diagram showing an example of the receipt image RI after layout analysis. Through the layout analysis, the receipt image RI is specified to have a paragraph image P1 that is a rectangular area surrounding the document name notation N1, a paragraph image P2 that is a rectangular area surrounding the date notation N2, a paragraph image P3 that is a rectangular area surrounding the destination notation N3, a paragraph image P4 that is a rectangular area surrounding the amount notation N4, a paragraph image P5 that is a rectangular area surrounding the proviso notation N5 and the receipt confirmation statement notation N6, and a paragraph image P6 that is a rectangular area surrounding the address notation N7 and the issuer name notation N8. Each of the paragraph images P1 to P6 is a part of the receipt image RI.

[0041] The character type determination unit 123 determines the character type of the notation included in each paragraph image P identified by the character area identification unit 122. In this embodiment, the character type determination unit 123 determines whether each paragraph image P included in the receipt image RI corresponds to a number paragraph image predicted to include only numerical notation, or a mixed paragraph image predicted to include a mixture of notations such as kanji, hiragana, katakana, symbols, and numbers.

[0042] In this embodiment, the character type determination unit 123 determines the paragraph image P4 corresponding to the amount notation N4 as a numeric paragraph image, and determines the remaining paragraph images P1 to P3 and P5 to P6 as mixed paragraph images. The character type determination unit 123 identifies the paragraph image P4 corresponding to the amount notation N4 from the receipt image RI, for example, using pattern matching. The layout of commercially available receipt paper and receipts printed out from store cash registers is almost fixed. For this reason, the character type determination unit 123 holds general receipt layouts as templates, and determines which template the receipt image RI to be processed this time matches. In each template, the position of the paragraph corresponding to the amount display is specified in advance. The character type determination unit 123 compares the position of the paragraph of the amount display in the template with the position of each paragraph image P included in the receipt image RI, and identifies the paragraph image P corresponding to the amount display in the receipt image RI.

[0043] Both the first character recognition unit 124 and the second character recognition unit 125 recognize notations included in the processing target image as characters. In this embodiment, the first character recognition unit 124 and the second character recognition unit 125 perform character recognition by, for example, AI (Artificial Intelligence)-OCR. In AI-OCR, the first character recognition unit 124 and the second character recognition unit 125 recognize characters included in the processing target image using a learning model that has learned the relationship between an image including characters and the characters included in the image. The learning model is configured by a multi-layer neural network. By using AI-OCR, characters, especially handwritten characters, can be recognized with high accuracy. The first character recognition unit 124 and the second character recognition unit 125 have different character recognition algorithms. The first character recognition unit 124 uses a standard algorithm for recognizing Japanese that is suitable for recognizing documents containing a mixture of kanji, hiragana, katakana, symbols, numbers, etc. In addition, the second character recognition unit 125 uses an algorithm specialized for recognizing numbers in the number recognition unit 126 described below, and uses an algorithm specialized for recognizing symbols in the symbol recognition unit 127 described below.

[0044] The first character recognition unit 124 performs character recognition on the paragraph image P determined to be a mixed paragraph image by the character type determination unit 123. In this embodiment, the first character recognition unit 124 performs character recognition on notations included in the paragraph image P1 corresponding to the document name notation N1, the paragraph image P2 corresponding to the date notation N2, the paragraph image P3 corresponding to the destination notation N3, the paragraph image P5 corresponding to the proviso notation N5 and the receipt confirmation statement notation N6, and the paragraph image P6 corresponding to the address notation N7 and the issuer name notation N8. When notations arranged in each of the paragraph images P1 to P3 and P5 to P6 span multiple lines, the first character recognition unit 124 divides each of the paragraph images P1 to P3 and P5 to P6 into lines, and further divides them into partial images in character units that are estimated to be one character. Then, the first character recognition unit 124 recognizes notations included in the partial images cut out in character units as one character. The recognition result in the first character recognition unit 124 is output to the recognition result generation unit 130. In this embodiment, the recognition results of first character recognition unit 124 and second character recognition unit 125 are output as text data such as Shift-JIS code. In this embodiment, first character recognition unit 124 uses a standard algorithm for recognizing Japanese, but if there is a paragraph image P that contains only a specific character type (or is highly likely to contain only a specific character type) among paragraph images P1-P3 and P5-P6, an algorithm specialized for recognizing that character type may be applied to that paragraph image P.

[0045] The second character recognition unit 125 performs character recognition on the paragraph image P that is determined to be a numeric paragraph image by the character type determination unit 123. In this embodiment, the second character recognition unit 125 performs character recognition on the paragraph image P4 corresponding to the amount notation N4. The paragraph image P4 of the receipt image RI is an example of a first image representing numbers and symbols other than numbers. As described above, the amount notation N4 is written in handwritten characters. Therefore, the paragraph image P4 is an image obtained by reading a document including handwritten numbers. The second character recognition unit 125 includes a number recognition unit 126, a symbol recognition unit 127, a symbol erasure unit 128, and an output unit 129.

[0046] The digit recognition unit 126 performs character recognition processing using an algorithm specialized for digit recognition (hereinafter referred to as the "digit algorithm"). The symbol recognition unit 127 performs character recognition processing using an algorithm specialized for symbol recognition (hereinafter referred to as the "symbol algorithm"). The symbol recognition unit 127 may perform character recognition processing using, for example, an algorithm specialized for recognizing symbols that are frequently used in association with digits among symbols. Symbols that are frequently used in association with digits are, for example, commas (,), dots (.), hyphens (-), currency symbols (￥, $, etc.). Further, particularly when the digit represents an amount of money, Chinese characters (language characters) such as "yen", "gold", and "ye" that are frequently used in association with the digit may be treated as symbols and made recognizable by the symbol recognition unit 127.

[0047] The symbol elimination unit 128 eliminates the notation corresponding to the symbol from the processing target image (paragraph image P4 in the present embodiment). The symbol elimination unit 128 eliminates the notation indicating the symbol, for example, by replacing the partial image C including the notation corresponding to the symbol with a plain image having the same color as the background color of the receipt image RI.

[0048] The output unit 129 outputs the recognition result of the notation included in the processing target image (paragraph image P4 in the present embodiment) based on the recognition result by the digit recognition unit 126. Further, the output unit 129 may output the recognition result of the notation included in the processing target image based on the recognition result by the digit recognition unit 126 and the recognition result by the symbol recognition unit 127.

[0049] A character recognition method of the second character recognition unit 125 will be specifically described with reference to FIG. 9. First, as shown in FIG. 9A, the second character recognition unit 125 divides notations included in the paragraph image P4 into units estimated to be one character (hereinafter, referred to as "character units"). The regions divided into character units are called partial images C (C1 to C8). In this embodiment, the partial image C is a rectangular region including notations estimated to be one character. In this embodiment, the partial image C1 including the notation "￥", the partial image C2 including the notation "2", the partial image C3 including the notation "5", the partial image C4 including the notation ", (comma)", the partial image C5 including the notation "7", the partial image C6 including the notation "6", the partial image C7 including the notation "0", and the partial image C8 including the notation "- (hyphen)" are cut out from the paragraph image P4.

[0050] Next, the second character recognition unit 125 performs character recognition processing of the notation included in each of the partial images C1 to C8 by the symbol recognition unit 127. That is, the symbol recognition unit 127 recognizes the symbol included in the paragraph image P4. For example, the second character recognition unit 125 determines that an notation whose accuracy of the recognition result by the symbol recognition unit 127 as a symbol is a predetermined value or more is a symbol, and determines that a partial image C whose accuracy of the recognition result is less than the predetermined value is not a symbol. The accuracy of the recognition result is an index value indicating the accuracy of the recognition result, and in this embodiment, the higher the accuracy, the higher the accuracy of the recognition result. In this embodiment, the partial image C1 including the notation of "￥", the partial image C4 including the notation of ",", and the partial image C8 including the notation of "-" are determined to include an notation indicating a symbol. In the first embodiment, it is sufficient to specify the partial image C including the notation of the symbol, and the recognition result of the notation of the symbol does not need to be stored.

[0051] Next, the second character recognition unit 125 erases the notation indicating the symbol by the symbol erasing unit 128 as shown in FIG. 9B. In this embodiment, the symbol erasing unit 128 erases the notation indicating the symbol by, for example, replacing the images of the partial images C1, C4, and C8 with a plain image having the same color as the background color of the receipt image RI. The paragraph image P4 from which the notation indicating the symbol has been deleted is set as the paragraph image P4A. The paragraph image P4A is shown in FIG. 9C. That is, the symbol erasing unit 128 is an example of a generating unit that generates the paragraph image P4A by erasing the partial images C1, C4, and C8 corresponding to the symbol from the paragraph image P4. More specifically, the symbol erasing unit 128 erases the portion corresponding to the symbol recognized by the symbol recognition unit 127 to generate the paragraph image P4A. The partial images C1, C4, and C8 are an example of the first portion, and the paragraph image P4A is an example of the second image.

[0052] Then, the second character recognition unit 125 performs character recognition processing on the notations included in the paragraph image P4A using the number recognition unit 126. That is, the number recognition unit 126 recognizes the numbers included in the paragraph image P4A generated by the symbol erasure unit 128. In this embodiment, the paragraph image P4A includes partial images C2 to C3 and C5 to C7. The number recognition unit 126 recognizes that the notation in partial image C2 indicates "2," the notation in partial image C3 indicates "5," the notation in partial image C5 indicates "7," the notation in partial image C6 indicates "6," and the notation in partial image C7 indicates "0."

[0053] The second character recognition unit 125 outputs the recognition result of the paragraph image P4 through the output unit 129. In this embodiment, the output unit 129 arranges the recognition results of the notations in the partial images C2 to C3, C5 to C7 in the order of arrangement of the partial images C2 to C3, C5 to C7 in the paragraph image P4, and outputs them as the recognition results of the notations included in the paragraph image P4. That is, the output unit 129 outputs a character string in which only the recognition results of the numbers are arranged among the notations included in the paragraph image P4. At this time, the places in the partial image C where the symbols were written may not be left blank, and the numbers may be arranged by filling them. Specifically, the output unit 129 outputs "25760" as the recognition result U of the notations included in the paragraph image P4, as shown in FIG. 9E. The recognition result by the second character recognition unit 125 is output to the recognition result generating unit 130.

[0054] The recognition result generating unit 130 generates the recognition result data by inputting the recognition results output by the first character recognition unit 124 and the second character recognition unit 125 into a predetermined format. In this embodiment, the predetermined format is the format of the recognition result data RD shown in FIG. 3. The recognition result generating unit 130 generates the recognition result data by inputting the recognition result of the paragraph image P2 into the date recognition result D1, the recognition result of the paragraph image P3 into the destination recognition result D2, the recognition result of the paragraph image P4 into the amount recognition result D3, the character string of the recognition result of the paragraph image P5 excluding unnecessary parts such as "However" and "As" into the proviso recognition result D4, and the character string of the paragraph image P6 corresponding to the issuer name notation N8 into the issuer recognition result D5. The recognition result generating unit 130 also transmits the generated recognition result data to the terminal device 20.

[0055] [flowchart] FIG. 10 is a flowchart showing the procedure of the character recognition process by the character recognition device 10. The control unit 120 of the character recognition device 10, functioning as the image acquisition unit 121, receives the receipt image RI to be processed from the terminal device 20 (step S100). Next, the control unit 120, functioning as the character region identification unit 122, performs layout analysis on the receipt image RI acquired in step S100 and identifies paragraph images P included in the receipt image RI (step S102). Next, the control unit 120, functioning as the character type determination unit 123, determines the character type of each paragraph image P identified in step S102 (step S104). More specifically, the control unit 120 determines whether each paragraph image P is a numeric paragraph image or a mixed paragraph image.

[0056] The control unit 120 sequentially performs character recognition on each paragraph image P identified in step S102. If the paragraph image P to be processed is a numeric paragraph image (step S106: YES), the control unit 120 functions as the second character recognition unit 125 and performs character recognition on the notation in the paragraph image to be processed using a numeric algorithm and a symbol algorithm (step S108: second character recognition process). Details of step S108 are shown in FIG. 11 This will be described later using a flowchart.

[0057] Also, if the paragraph image to be processed is not a numeric paragraph image (step S106: NO), i.e., if it is a mixed paragraph image, the control unit 120 functions as a first character recognition unit 124 and performs character recognition on the notation in the paragraph image to be processed using a standard algorithm for recognizing Japanese (step S110: first character recognition processing).

[0058] The control unit 120 returns to step S106 and continues the character recognition process until the character recognition process for all paragraph images is completed (step S112: NO). When the character recognition process for all paragraph images is completed (step S112: YES), the control unit 120 functions as the recognition result generating unit 130, inputs the recognition result for each paragraph image P into a predetermined format to generate recognition result data (step S114), transmits the generated recognition result data to the terminal device 20 (step S116), and ends the process according to this flowchart.

[0059] Next, the process of step S108 (second character recognition process by second character recognition unit 125) will be described with reference to Fig. 11. As described above, when the paragraph image P to be processed is a numeric paragraph, control unit 120 functions as second character recognition unit 125 and performs the following process. Control unit 120 divides notation included in the paragraph image P to be processed (paragraph image P4 in this embodiment) into character units (see Fig. 9A) and cuts out multiple partial images C (step S200).

[0060] The control unit 120 functions as the symbol recognition unit 127 to perform character recognition processing using a symbol algorithm for each partial image C included in the paragraph image P (step S202: symbol recognition processing). If the paragraph image P to be processed has a partial image C including a notation indicating a symbol (step S204: YES), the control unit 120 functions as the symbol deletion unit 128 to delete the partial image C including the notation indicating a symbol (see FIG. 9B) (step S206). If the paragraph to be processed does not have a partial image C including a notation indicating a symbol (step S204: NO), the control unit 120 proceeds to step S208.

[0061] The control unit 120 functions as the number recognition unit 126 to perform character recognition (see FIG. 9C) using a number algorithm for each partial image C for the notation included in the paragraph image P (step S208: number recognition process). The control unit 120 functions as the output unit 129 to output the recognition result in step S208 to the recognition result generation unit 130 (see FIG. 9E) as the recognition result for the paragraph image P (step S210), and ends the processing of this flowchart.

[0062] As described above, in the first embodiment, the symbol erasing unit 128 generates a paragraph image P4A (see FIG. 9C) by erasing the partial images C1, C4, and C8 corresponding to symbols from a paragraph image P4 (see FIG. 8) representing numbers and symbols other than numbers, and the number recognition unit 126 recognizes numbers included in the paragraph image P4A. Since the number recognition unit 126 recognizes numbers from the paragraph image P4A from which symbols other than numbers have been erased in advance, the possibility of misrecognizing symbols as numbers is reduced, and the accuracy of number recognition can be improved compared to when numbers are recognized from the paragraph image P4 in which symbols other than numbers are mixed.

[0063] In the first embodiment, the symbol recognition unit 127 recognizes symbols included in the paragraph image P4, and the symbol deletion unit 128 deletes the partial images C1, C4, and C8 corresponding to the symbols recognized by the symbol recognition unit 127 to generate the paragraph image P4A. By recognizing the symbols included in the paragraph image P4 by the symbol recognition unit 127, the partial image C corresponding to the symbols can be reliably specified and deleted from the paragraph image P4. For example, compared to the case where symbols having a smaller area than numbers, such as commas and dots, are deleted as noise by image processing, when the symbol recognition unit 127 is used, symbols having a size similar to that of numbers, such as "￥", can also be deleted, and symbols can be deleted more reliably from the paragraph image P4.

[0064] In the first embodiment, the output unit 129 outputs a character string in which only the recognition results of numbers among the notations included in the paragraph image P4 are arranged as the recognition result of the paragraph image P4. This unifies the character types included in the recognition result of the paragraph image P4 to only numbers, and reduces the processing load using the recognition result data. For example, when the character recognition result for the receipt image RI is used in an accounting application, the recognition result of the amount notation N4 is unified to only numbers, so that it can be used directly for processing such as calculation without performing preprocessing such as deleting unnecessary symbols.

[0065] In the first embodiment, the number recognition unit 126 performs character recognition processing using a number algorithm specialized for recognizing numbers. In general, when the character type included in the processing target image is fixed, the accuracy of character recognition can be improved by using an algorithm specialized for recognizing the character type. On the other hand, an algorithm specialized for recognizing a specific character type tends to recognize any character or line as one character in the specific character type, and recognition accuracy is likely to decrease when other character types are mixed in the processing target image. As in the present embodiment, by erasing in advance notations other than the specific character type (numbers) from the processing target image, the accuracy of character recognition using an algorithm specialized for recognizing the specific character type can be further improved.

[0066] [Second embodiment] Next, a second embodiment will be described. In the following examples, for elements whose functions are similar to those of the first embodiment, the reference symbols used in the description of the first embodiment will be used and detailed descriptions of each will be omitted as appropriate. In the first embodiment, the recognition result output by the output unit 129 did not include the recognition result of the notation recognized as a symbol. In the second embodiment, the recognition result output by the output unit 129 includes the recognition result of the notation recognized as a symbol.

[0067] The character recognition method of second character recognition unit 125 in the second embodiment will be described with reference to Fig. 9. In the second embodiment, second character recognition unit 125 first divides writings included in paragraph image P4 into character units, as shown in Fig. 9A. In this embodiment, partial images C1 to C8 are cut out from paragraph image P4.

[0068] Next, the second character recognition unit 125 performs character recognition processing of the notation included in each of the partial images C1 to C8 by the symbol recognition unit 127. In this embodiment, the notation included in the partial image C1 is recognized as "￥", the notation included in the partial image C4 is recognized as ",", and the notation included in the partial image C8 is recognized as "-", and the notations included in the other partial images C2 to C3 and C5 to C7 are recognized as not symbols. In the second embodiment, the output unit 129 saves the position of the partial image C including the notation of a symbol and the recognition result of the notation of the symbol.

[0069] Next, the second character recognition unit 125 uses the symbol erasing unit 128 to erase the notations indicating symbols as shown in FIG. 9B. In this embodiment, the notations in the partial images C1, C4, and C8 are erased. The paragraph image P4 from which the notations indicating symbols have been deleted is set as a paragraph image P4A, which is shown in FIG. 9C. Then, the second character recognition unit 125 uses the number recognition unit 126 to perform character recognition processing on the notations included in the partial images C2 to C3 and C5 to C7 included in the paragraph image P4A, and recognizes the numbers indicated by each notation. In this embodiment, as shown in FIG. 9D, the notation in the partial image C2 is recognized as indicating "2," the notation in the partial image C3 as indicating "5," the notation in the partial image C5 as indicating "7," the notation in the partial image C6 as indicating "6," and the notation in the partial image C7 as indicating "0."

[0070] When the recognition results of all partial images C1 to C8 are collected, the second character recognition unit 125 generates a character string in which the recognition results of all partial images C1 to C8 are arranged according to the arrangement of partial images C1 to C8 in paragraph image P4, by means of the output unit 129. Then, the output unit 129 outputs this character string as the recognition result for paragraph image P4. That is, the second character recognition unit 125 outputs the character string "￥25,760-" in which the notation "￥" in partial image C1, the notation "2" in partial image C2, the notation "5" in partial image C3, the notation "," in partial image C4, the notation "7" in partial image C5, the notation "6" in partial image C6, the notation "0" in partial image C7, and the notation "-" in partial image C8 are arranged in this order as the recognition result for paragraph image P4. That is, the output unit 129 outputs a character string in which the numbers identified by the number recognition unit 126 and the symbols identified by the symbol recognition unit 127 are arranged. The arrangement of the numbers and symbols included in this character string corresponds to the arrangement of the numbers and symbols in the paragraph image P4.

[0071] Fig. 12 is a diagram showing an example of recognition result data in the second embodiment. The recognition result data RDA shown in Fig. 12 shows the result of character recognition of the receipt image RI in Fig. 2. In the recognition result data RDA, the date recognition result D1, the destination recognition result D2, the proviso recognition result D4, and the issuer recognition result D5 are the same as the recognition result data RD shown in Fig. 3. On the other hand, in the recognition result data RD shown in Fig. 3, the amount recognition result D3 is "25760", which is a character string consisting of numbers only. In contrast, in the recognition result data RDA shown in Fig. 12, the amount recognition result D3 is "￥25,760-", which includes numbers and symbols.

[0072] As described above, in the second embodiment, the output unit 129 outputs a character string in which the numbers identified by the number recognition unit 126 and the symbols identified by the symbol recognition unit 127 are arranged. The arrangement of the numbers and symbols included in this character string corresponds to the arrangement of the numbers and symbols in the paragraph image P4. That is, in the second embodiment, the output unit 129 outputs a character string in which all the recognition results of the notations included in the paragraph image P4 are arranged as the recognition result of the paragraph image P4. As a result, the recognition results of the notations included in the paragraph image P4 are recorded in the recognition result data without exception. Therefore, it is possible to improve convenience in cases such as when the notation of the amount in the paragraph image P4 includes a number after the decimal point, or when the type of currency in the notation of the amount is unknown and needs to be identified by the currency notation such as ￥.

[0073] The output unit 129 may be set so that only specific symbols among those identified by the symbol recognition unit 127 are included in the character string of the recognition result, or so that only specific symbols are not included in the character string of the recognition result. For example, when the numeral notation "1,234,567.89" is character-recognized, even if commas separating digits are not necessary, the recognition result as a numerical value may differ if the dot corresponding to the decimal point is not included in the recognition result. In this case, the output unit 129 is set in advance so that the dot among the recognized symbols is included in the recognition result, and the other symbols are not included in the recognition result. As a result, the recognition result of the numeral notation is output as "1234567.89".

[0074] [Third embodiment] Next, the third embodiment will be described. In the following examples, for elements whose functions are similar to those of the first or second embodiment, the symbols used in the description of the first or second embodiment will be used, and detailed descriptions of each will be omitted as appropriate. In the first and second embodiments, the second character recognition unit 125 erases the symbols recognized by the symbol recognition unit 127 from the image, and then performs character recognition of the numbers using the number recognition unit 126. In the third embodiment, the number recognition unit 126 and the symbol recognition unit 127 each perform character recognition for all notations included in the paragraph image P4, and determine the notation of the symbol based on the accuracy of the recognition result. Thereafter, the notations recognized as symbols are erased from the paragraph, and then the number recognition unit 126 performs character recognition of the numbers.

[0075] The character recognition method of the second character recognition unit 125 in the third embodiment will be described with reference to Fig. 13. In the third embodiment, the second character recognition unit 125 first classifies writings included in the paragraph image P4 into character units, as shown in Fig. 13A. In this embodiment, partial images C1 to C8 are cut out from the paragraph image P4.

[0076] Next, the second character recognition unit 125 performs character recognition of the notation included in the partial images C1 to C8 by the symbol recognition unit 127. In the third embodiment, the symbol recognition unit 127 outputs the recognition result of the notation of each of the partial images C1 to C8 together with the accuracy information of each recognition result. Hereinafter, the recognition result by the symbol recognition unit 127 is referred to as the "second recognition result", and the accuracy of the second recognition result is referred to as the "second accuracy". In this embodiment, as shown in FIG. 13B, the second recognition result of the notation of the partial image C1 is "￥" and the second accuracy is 95%. The second character recognition result of the notation of the partial image C2 is ">" and the second accuracy is 15%. The second recognition result of the notation of the partial image C3 is "$" and the second accuracy is 18%. The second recognition result of the notation of the partial image C4 is "," and the second accuracy is 93%. The second recognition result of the notation of the partial image C5 is " / " and the second accuracy is 28%. The second recognition result of the notation in partial image C6 is "&" and the second accuracy is 11%. The second recognition result of the notation in partial image C7 is "@" and the second accuracy is 20%. The second recognition result of the notation in partial image C8 is "-" and the second accuracy is 95%. Note that the accuracy percentages are omitted in FIG. 13.

[0077] In addition, the second character recognition unit 125 performs character recognition of the notation included in the partial images C1 to C8 by the number recognition unit 126. The number recognition unit 126 also outputs the recognition result of the notation of each partial image C1 to C8, together with the accuracy information of each recognition result. Hereinafter, the recognition result by the number recognition unit 126 is referred to as the "first recognition result", and the accuracy of the first recognition result is referred to as the "first accuracy". In this embodiment, as shown in FIG. 13C, the first recognition result of the notation of the partial image C1 is "7" and the first accuracy is 28%. The first recognition result of the notation of the partial image C2 is "2" and the first accuracy is 93%. The first recognition result of the notation of the partial image C3 is "5" and the first accuracy is 95%. The first recognition result of the notation of the partial image C4 is "1" and the first accuracy is 20%. The first recognition result of the notation of the partial image C5 is "7" and the first accuracy is 94%. The first recognition result of the notation in partial image C6 is "6" and the first accuracy is 95%. The first recognition result of the notation in partial image C7 is "0" and the first accuracy is 96%. The first recognition result of the notation in partial image C8 is "7" and the first accuracy is 11%.

[0078] The second character recognition unit 125 uses the symbol erasing unit 128 to compare the first accuracy and the second accuracy for each of the partial images C1 to C8, and determines that the notation in the partial image C with the higher second accuracy is a symbol. In this embodiment, the notations in the partial images C1, C4, and C8 are determined to be symbols. Thereafter, similar to the first embodiment, the second character recognition unit 125 uses the symbol erasing unit 128 to erase the notation indicating a symbol as shown in FIG. 9C. The second character recognition unit 125 performs character recognition on the paragraph image P4A from which the notation indicating a symbol has been deleted by the number recognition unit 126. Then, the output unit 129 outputs the recognition result by the number recognition unit 126 as the recognition result for the paragraph image P4.

[0079] That is, in the third embodiment, the paragraph image P4 is divided into a plurality of partial images C1 to C8, and the number recognition unit 126 recognizes the notation in each of the partial images C1 to C8 as a number and calculates a first accuracy indicating the accuracy of the number recognition result. Also, the symbol recognition unit 127 recognizes the notation in each of the partial images C1 to C8 as a symbol and calculates a second accuracy indicating the accuracy of the symbol recognition result. The symbol erasure unit 128 identifies a partial image C including a symbol notation based on a comparison result between the first accuracy and the second accuracy, and erases the partial image C including the symbol notation from the paragraph image P4.

[0080] In this way, in the third embodiment, the spelling of symbols included in the paragraph image P4 is specified using not only the recognition result by the symbol recognition unit 127 but also the recognition result by the number recognition unit 126, so that the spelling of symbols can be specified with higher accuracy. For example, even for a symbol having a shape similar to a number, it can be determined whether it is a symbol or a number based on the recognition results and accuracy of the symbol recognition unit 127 and the number recognition unit 126, respectively.

[0081] In the above description, the second character recognition unit 125 determines the partial image C including the symbol notation based on the first accuracy and the second accuracy, erases the partial image C including the symbol notation, and then performs character recognition by the number recognition unit 126. Without being limited to this, the second character recognition unit 125 may select whether to adopt the first recognition result or the second recognition result for the notation of each of the partial images C1 to C8 based on the first accuracy and the second accuracy. In this case, the second character recognition unit 125 arranges the selected recognition result based on the arrangement of the partial images C1 to C8 in the paragraph image P4, and outputs it as the recognition result for the paragraph image P4.

[0082] For example, in the example of FIG. 13, the first accuracy of the representation of the partial image C1 is 28 %, and the second accuracy is 95%. Therefore, the more likely recognition result of the notation in partial image C1 is the second recognition result "￥". Similarly, the notation in partial image C2 is "2", the notation in partial image C3 is "5", the notation in partial image C4 is ",", the notation in partial image C5 is "7", the notation in partial image C6 is "6", the notation in partial image C7 is "0", and the notation in partial image C8 is "-" are selected as more likely recognition results. The second character recognition unit 125 outputs the recognition results of the notation in each of the partial images C1 to C8 by the output unit 129 in the same manner as the arrangement of the partial images C1 to C8 in paragraph image P4 as "￥25,760-", and as the recognition result in paragraph image P4.

[0083] If it is not desired to include symbols in the recognition result of paragraph image P4, the output unit 129 may be configured to delete the second recognition result when the recognition result of paragraph image P4 includes the second recognition result, and to generate a character string consisting of only the first recognition result.

[0084] [Fourth embodiment] Next, the fourth embodiment will be described. In the following examples, for elements whose functions are similar to those of the first to third embodiments, the reference numerals used in the description of the first to third embodiments will be used, and detailed description of each will be omitted as appropriate. In the first to third embodiments, the second character recognition unit 125 includes the number recognition unit 126, the symbol recognition unit 127, and the symbol deletion unit 128, and the symbol notation recognized by the symbol recognition unit 127 is deleted by the symbol deletion unit 128, and then the number recognition unit 126 performs character recognition of the numbers. In the fourth embodiment, the symbol recognition unit 127 and the symbol deletion unit 128 are not used, and a noise removal process, which is a type of image processing, is performed on the paragraph image P4, thereby removing the symbols from the paragraph image P4. Then, the number recognition unit 126 performs character recognition of the numbers on the paragraph image P4 after the noise removal process, and outputs the recognition result in the paragraph image P4.

[0085] 14 is a block diagram showing a functional configuration of a character recognition device 10A in the fourth embodiment. The second character recognition unit 125A of the character recognition device 10A includes a number recognition unit 126, an image processing unit 132, and an output unit 129. The image processing unit 132 performs a noise removal process to remove noise from the paragraph image P4 to be processed. The noise removal process for the image is a conventional technique, so a detailed description will be omitted. However, the image processing unit 132 applies, for example, a median filter or a smoothing filter to the paragraph image P4 to remove symbols included in the paragraph image P4. That is, the image processing unit 132 is an example of a generating unit that performs a noise removal process on the paragraph image P4 to generate an image in which a partial image C corresponding to a symbol is erased from the paragraph image P4.

[0086] Fig. 15 is a diagram showing an example of a receipt image to which the fourth embodiment is suitable. In addition to the number "25760", the amount notation N4 of the receipt image RIA in Fig. 15 includes a comma between the "5" and "2" as a digit separator, and a hyphen after the "0" as a symbol indicating the end of the amount. These symbols occupy a smaller area in the paragraph image P4 than the numbers, and can be removed as noise.

[0087] 9A, when the notation included in the paragraph image P4 is divided into character-based partial images C, if there is a partial image C whose size is equal to or smaller than a predetermined value, the partial image C may be removed as noise. Removing the partial image C as noise may mean, for example, replacing the partial image C with a plain image having the same color as the background color of the receipt image RI. The size of the partial image C being equal to or smaller than a predetermined value may mean, for example, that the number of pixels constituting the partial image C is equal to or smaller than a predetermined number. In this case, the predetermined number may be a fixed value determined in advance, or may be determined based on the number of pixels of other partial images C included in the same paragraph image P.

[0088] As described above, in the fourth embodiment, the image processing unit 132 performs noise removal processing on the paragraph image P4 to generate an image in which the partial image C corresponding to the symbol is erased from the paragraph image P4. This makes it possible to erase the partial image C corresponding to the symbol from the paragraph image P4 by simple processing, and reduces the processing load on the character recognition device 10.

[0089] [Variations] The present invention is not limited to the above-described embodiments. Specific modified embodiments are exemplified below. Two or more embodiments selected from the following examples may be combined.

[0090] [First Modification] In the above-described embodiment, the receipt image RI (particularly the paragraph image P4 showing numbers and symbols other than numbers) is an image obtained by scanning a receipt including handwritten numbers. However, the above-described embodiment can also be applied to cases where the numbers included in the receipt image RI are characters printed using type (various fonts used in text editors and cash register terminals, character shapes in letterpress printing, etc.).

[0091] [Second modified example] In the above-described embodiment, the image to be processed by the character recognition device 10 is a receipt image RI. However, the image to be processed by the character recognition device 10 is not limited to this, and may be an image generated by reading a medium (e.g., a document) containing numbers with a scanner or photographing it with a camera. Media containing numbers include documents in a wide range of fields, such as invoices, financial statements, transfer forms, estimates, and other accounting documents, and recording sheets on which measured values ​​from experiments and observations are recorded.

[0092] [Third Modification] In the above-described embodiment, the character recognition device 10 performs character recognition processing on the receipt image RI transmitted from the terminal device 20, and returns the recognition result data RD to the terminal device 20. This is not limiting, and for example, the character recognition device 10 may generate an image to be processed. Specifically, for example, a user may mail a receipt he or she owns to a provider of a receipt digitization service, and the provider may read the receipt with a scanner connected to the character recognition device 10 to generate an image to be processed. Also, for example, a program for executing the character recognition processing (for example, the control program PG1 in FIG. 6) may be installed in the terminal device 20. In this case, the terminal device 20 functions as the character recognition device 10. [Explanation of symbols]

[0093] 1...character recognition system, 10, 10A...character recognition device, 120...control unit, 121...image acquisition unit, 122...character area identification unit, 123...character type determination unit, 124...first character recognition unit, 125, 125A...second character recognition unit, 126...number recognition unit, 127...symbol recognition unit, 128...symbol erasure unit, 129...output unit, 130...recognition result generation unit, 132...image processing unit, 140...memory unit, 160...communication unit, 20...terminal device, 220...control unit, 222...image transmission unit, 224...recognition result data receiving unit, 240...memory unit, 260...communication unit, 280...operation unit, 290...display unit, NW...network, RD, RDA...recognition result data, RI, RIA...receipt image.

Claims

1. a symbol recognition unit that recognizes symbols included in a first image representing numbers and symbols other than numbers; a generating unit that generates a second image by erasing a first portion corresponding to the symbol from the first image; a number recognition unit that recognizes numbers included in the second image generated by the generation unit, the generation unit erases a portion corresponding to the symbol recognized by the symbol recognition unit as the first portion to generate the second image.

23. An information processing apparatus comprising:

2. an output unit that outputs a character string in which the numbers identified by the number recognition unit and the symbols identified by the symbol recognition unit are arranged, The sequence of the numbers and the symbols included in the character string is corresponding to the arrangement of the numbers and the symbols in the first image; 2. The information processing apparatus according to claim 1,

3. The first image is divided into a plurality of portions; The number recognition unit is Recognizing the representation in each of the plurality of portions as the number, and calculating a first accuracy indicating an accuracy of the recognition result of the number; The symbol recognition unit includes: Recognizing the representation in each of the plurality of portions as the symbol, and calculating a second accuracy indicating an accuracy of the recognition result of the symbol; The generation unit is identifying the first portion in the first image based on a comparison result between the first likelihood and the second likelihood, and erasing the first portion from the first image; 3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

4. A generating unit that generates a second image by erasing a first portion corresponding to a symbol from a first image representing a number and a symbol other than a number; a number recognition unit that recognizes numbers included in the second image generated by the generation unit, The generation unit generates the second image by performing a noise removal process on the first image.

23. An information processing apparatus comprising:

5. The first image is an image obtained by scanning a document including the handwritten numbers.

5. The information processing apparatus according to claim 1,

6. The processor, a symbol recognition unit that recognizes symbols included in a first image representing numbers and symbols other than numbers; a generating unit that generates a second image by erasing a first portion corresponding to the symbol from the first image; a number recognition unit that recognizes numbers included in the second image generated by the generation unit; and make it work. the generation unit erases a portion corresponding to the symbol recognized by the symbol recognition unit as the first portion to generate the second image. A program characterized by:

Images