Image processing device, image processing method, and program

Abstract

To enable identification of the classification type of a document image using predefined rule-based conditions.SOLUTION: An image processing device disclosed herein is configured to obtain a character recognition result by performing character recognition processing on a document image, and identify the classification type of the document image on the basis of character strings included in the character recognition result and predefined conditions.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a document image Information about the company that issued the The present invention relates to an image processing device for specifying the [Background technology]

[0002] In recent years, it has become common to digitize documents by scanning them using image scanners attached to devices such as multifunction printers (MFPs: multifunction devices with functions such as printing, copying, and faxing). It has also become common to digitize documents by photographing them using digital cameras or the camera functions of mobile devices such as smartphones. Thus, it has become easy to obtain document images (scanned document images) by optically scanning or photographing documents containing handwritten or printed characters. Furthermore, by performing optical character recognition (OCR) on these document images, the character images in the document images can be converted into character codes that can be used by a computer. Such character recognition processing has led to the automation of tasks such as converting paper documents (receipts, invoices, etc.) into digital data and inputting them into systems (e.g., expense reimbursement). This is expected to improve productivity in data entry tasks.

[0003] Figure 1(a) shows the flow of information between the accounting system and various linked systems.

[0004] The accounting system 101 is a system that supports accounting and bookkeeping work. The accounting system 101 is used by accounting and bookkeeping staff to record transactions of cash, deposits, assets, products, etc. within a company while classifying them from a management or tax perspective, and is an electronic version of the various forms of ledgers that were previously used. The recorded contents are stored as accounting processing results 102 and are output as various ledgers 107 as needed. Furthermore, when a transaction is made, the accounting system must classify expenses according to the content and purpose of the transaction, and expense item codes are used to identify each expense item.

[0005] There are also cash receipts and withdrawals management systems 103 that manage cash and deposits, budget management systems 104 that manage the budgets of each department, inventory management systems 105 that manage product inventory, and asset management systems 106 that manage various assets. When transactions of cash, deposits, assets, products, etc. occur, the increase or decrease is recorded in each management system (103, 105, 106). Conventionally, accountants and bookkeepers have referenced the information on each transaction recorded in each management system (103, 105, 106) and recorded it in accounting system 102. Furthermore, budget management system 104 uses budget codes to identify each budget when managing budgets.

[0006] The form document 110 in FIG. 1(b) is an example of a receipt (form document) issued when office supplies were purchased. This is evidence that the office supplies were purchased. Various information is recorded in each section of the form document 110. For example, the document title 111 is the title of the form document 110 and indicates that this form document is a "receipt." The issue date 112 indicates the date on which this form document was issued. In accounting and finance work, when a receipt for this form document is saved as evidence, the issue date 302, "November 12, 2020," may be used to identify this form document.

[0007] Issuer 113 contains information about the company (issuer) that created and issued this form document, such as the company name, address, and telephone number. When receiving a form document and processing accounting or bookkeeping, it is necessary to clarify what was purchased and for what purpose in order to classify expenses, and the company name information of issuer 113, "AAA Office Machines Co., Ltd.", may be used for this purpose. In addition, when a receipt for this form document is saved as supporting evidence, the company name information may also be used to identify this form document.

[0008] The recipient 114 contains the name of the company that made the purchase and payment. The total amount 115 contains the total amount of the purchase and payment. When accounting or bookkeeping is performed on the document, this amount (and the tax amount, if necessary) will be used as the transaction amount. The details 116 contains detailed information about the invoice, and for each item, information such as the unit price, quantity, and price is listed. The summary 117 also shows that the total amount 115 is calculated by adding information such as tax to the subtotal of each price.

[0009] FIG. 1(c) shows an example of ledger information that is typically recorded in the accounting system 101 by an accountant or bookkeeper. The ledger transaction record 120 is a ledger-style record of transactions used in accounting and bookkeeping, and is shown in the form of a table. Each line in this table represents one transaction, with the transaction date listed first, followed by each piece of information.

[0010] The following describes an example of recording transaction information for a certain department (division). For example, the budget management system 104 pre-records the purchase details and amounts of office supplies used in business operations as part of the department's budget. The department then purchases office supplies from a company named "AAA Office Machines Co., Ltd." on November 12, 2020, and pays in cash through the cash flow management system 108. In this case, it is necessary to record that the purchase amount has been withdrawn. Furthermore, it is assumed that the company named "AAA Office Machines Co., Ltd." has issued and obtained a receipt 110 as evidence of the purchase and payment. In this case, the following information is recorded in the three columns (6th to 8th columns from the left) after the "Credit" column in the transaction record 120 in the ledger. Specifically, the payment record in the cash receipt management system records "cash" and its type, code "100," as well as the budget information from the budget management system 104, which includes the department name "Kamata Branch" and budget code "221," and the amount of "7,700" paid. Furthermore, the following information is recorded in the three columns (second through fourth columns from the left) after the "debit" side of the ledger transaction record 120: It records that "Kamata Branch" spent "7,700" yen on office supplies. Therefore, the accounting staff records the expense item "Office Supplies" and the expense item code "300" in the "debit" side, the department name "Kamata Branch" and budget code "221" in the "department" side, and "7,700" in the "amount." Furthermore, to facilitate linking to the report document 110 as the basis for classifying the expense item, "AAA Office Machines Co., Ltd." is entered in the "Summary." By recording in this way, it becomes possible to correlate and record individual facts, such as which department's budget was used to withdraw cash from assets and purchase office supplies, as transactions used as office supply expenses. When recording, the accounting and bookkeeping staff confirm each fact, including supporting documents, classify the expenses, and record them in the accounting system 101. The accumulated contents of these records are the accounting processing results 102.

[0011] When performing such work, it would be possible to reduce the workload of accounting and bookkeeping staff if there was a function that automatically transcribes the contents of the form document 110 (date, amount, company name, etc.) into the accounting system 102. For this reason, in recent years, studies have been conducted to read documents such as supporting documents as digital images using an image scanner, and perform character recognition processing to extract and transcribe the information written therein.

[0012] However, in documents such as receipts and invoices, the company name may be written in a logo or special font, or may be stamped with an unclear character image, which can result in incorrect recognition during character recognition processing.

[0013] In Patent Document 1, sorting elements including at least date, business partner, amount, application, and appearance such as size and color are extracted from a learning image, and accounting items for the sorting elements are machine-learned to create a journalizing AI. Then, when processing evidence images, sorting elements are extracted from the evidence images, and the sorting AI is used to select accounting items. [Prior art documents] [Patent documents]

[0014] [Patent Document 1] Japanese Patent Application Publication No. 2018-097813 Summary of the Invention [Problem to be solved by the invention]

[0015] Patent Document 1 The document did not disclose an image processing device that includes a first identification means that identifies information about the company that issued the document image based on one or more character strings recognized from the document image through character recognition processing of the document image and predefined conditions, a second identification means that identifies the company name of the company that issued the document image contained in the one or more character strings, and an output means that outputs company name information, wherein if the company name of the company is identified by the second identification means, the output means outputs the identified company name as the company name information, and if the company name of the company is not identified by the second identification means and information about the company is identified by the first identification means, the output means outputs information about the identified company instead of the company name as the company name information. [Means for solving the problem]

[0016] In order to solve the above problems, the image processing device of the present invention comprises: The document image processing device includes a first identification means for identifying information about the company that issued the document image based on one or more character strings recognized from the document image through character recognition processing on the document image and predefined conditions, a second identification means for identifying the company name of the company that issued the document image contained in the one or more character strings, and an output means for outputting company name information, wherein if the company name of the company is identified by the second identification means, the output means outputs the identified company name as the company name information, and if the company name of the company is not identified by the second identification means and information about the company is identified by the first identification means, the output means outputs information about the identified company instead of the company name as the company name information. [Effects of the Invention]

[0017] In the present invention, If the company name is not specified but information about the company is specified, the information about the specified company is output instead of the company name as information about the company name. [Brief explanation of the drawings]

[0018] [Figure 1] This is a diagram showing an overview of the process in accounting and bookkeeping work, an example of a document image, and an example of the input content for that process. [Figure 2] FIG. 1 is a diagram illustrating a system configuration of an image processing system. [Figure 3] FIG. 2 is a diagram illustrating a hardware configuration of an image forming apparatus 101. [Figure 4] FIG. 2 is a diagram illustrating the hardware configuration of an image processing server 102 and a user terminal 103. [Figure 5] FIG. 10 is a diagram illustrating an example of a receipt according to the first embodiment. [Figure 6] FIG. 2 is a diagram showing an overall flow of processing according to the first embodiment. [Figure 7] FIG. 4 is a diagram illustrating an example of a processing rule according to the first embodiment. [Figure 8] FIG. 10 is a diagram showing the flow of an item value output process according to the first embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of a receipt according to the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of input information according to the second embodiment. [Figure 11] FIG. 10 is a diagram showing the flow of an item value output process according to the second embodiment. [Figure 12] FIG. 11 is a diagram illustrating an example of a receipt according to the third embodiment. [Figure 13] FIG. 11 is a diagram illustrating an example of a rule according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0019] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited to the embodiments, and all of the configurations described in the embodiments are not necessarily essential means for solving the problems of the present invention.

[0020] First Embodiment In this embodiment, a data input support device that extracts and displays item names and item values ​​from a document image will be described.

[0021] 2 is a diagram showing an example of the configuration of an image processing system 200 according to the first embodiment. The image processing system 200 includes an image forming apparatus 201, an image processing server 202, and a user terminal 203, which are interconnected via a network 204 and are capable of communicating with each other.

[0022] The image forming apparatus 201 is capable of receiving and printing a print request (print data) for image data from the user terminal 203, reading image data using a scanner provided in the image forming apparatus 201, and printing the image data read by the scanner. The image forming apparatus 201 is also capable of saving print data received from the user terminal 203 and transmitting image data read by the scanner of the image forming apparatus 201 to the user terminal 203 or the image processing server 202. The image forming apparatus 201 is also capable of implementing functions possessed by well-known image forming apparatuses such as an MFP (Multifunction Peripheral). The user terminal 203 is also capable of displaying the image processing results received from the image processing server 202 using an application with a user interface and interactively processing the results according to instructions from the user. The image processing server 202 may be located in the cloud, i.e., on the Internet.

[0023] In this embodiment, the user terminal 203 is assumed to be a general PC equipped with a display, keyboard, and mouse, but may also be, for example, a mobile terminal equipped with a touch panel.

[0024] The following describes a series of data input support processes in which an image forming device 201 scans a document such as a receipt, an image processing server 202 extracts information from the image of the document, and a user confirms and corrects the extracted information on a user terminal 203.

[0025] 3 is a diagram showing an example of the configuration of an image forming apparatus 201. The image forming apparatus 201 has a controller 301, a printer 302, a scanner 303, and an operation unit 304. The controller 301 has a CPU 311, a RAM 312, an HDD 313, a network I / F 314, a printer I / F 315, a scanner I / F 316, an operation unit I / F 317, and an expansion I / F 318.

[0026] The CPU 311 controls the overall operation of the image forming apparatus 201. The CPU 311 can control the exchange of data with the RAM 312, HDD 313, network I / F 314, printer I / F 315, scanner I / F 316, operation unit I / F 317, and expansion I / F 318. The CPU 311 also loads a control program (instructions) read from the HDD 313 into the RAM 312 and executes the instructions loaded into the RAM 312.

[0027] The HDD 313 stores control programs executable by the CPU 311, setting values ​​used in the image forming apparatus 201, data related to processing requested by the user, etc. The RAM 312 has an area for temporarily storing instructions read by the CPU 311 from the HDD 313. The RAM 312 can also store various data necessary for executing instructions. For example, in image processing, the CPU 311 can perform processing by expanding input data in the RAM 312.

[0028] The network I / F 314 is an interface for performing network communication with devices within the image processing system 200. The network I / F 314 can notify the CPU 311 that data has been received, and can transmit data on the RAM 312 to the network 204.

[0029] The printer I / F 315 can transmit print data sent from the CPU 311 to the printer 302 and can transfer printer status information received from the printer 302 to the CPU 311. The scanner I / F 316 can transmit image reading instructions sent from the CPU 311 to the scanner 303, transfer image data received from the scanner 303 to the CPU 311, and transfer status information received from the scanner 303 to the CPU 311.

[0030] The operation unit I / F 317 can transmit user instructions input from the operation unit 304 to the CPU 311 and can transmit screen information for user operation to the operation unit 304. The expansion I / F 318 is an interface that enables an external device to be connected to the image forming apparatus 201. The expansion I / F 318 includes, for example, a USB (Universal Serial Bus) type interface. By connecting an external storage device such as a USB memory to the expansion I / F 318, the image forming apparatus 201 can read data stored in the external storage device and write data to the external storage device.

[0031] The printer 302 can print image data received from the printer I / F 315 onto paper and can transmit the status of the printer 302 to the printer I / F 315. The scanner 303 can read information displayed on paper placed on it, digitize it, and transmit it to the scanner I / F 316 in accordance with an image reading instruction received from the scanner I / F 316. The scanner 303 can also transmit its own status to the scanner I / F 316.

[0032] The operation unit 304 is an interface that allows a user to perform operations to give various instructions to the image forming apparatus 201. For example, the operation unit 304 is equipped with a liquid crystal display with a touch panel, and provides an operation screen to the user and accepts operations from the user. Details of the operation unit 304 will be described later with reference to FIG. 5.

[0033] 4(a) is a diagram showing an example of the configuration of the image processing server 202. The image processing server 202 has a CPU 401, a RAM 402, a HDD 403, and a network I / F 404. The CPU 401 controls the entire image processing server 202. The CPU 401 can control the exchange of data with the RAM 402, the HDD 403, and the network I / F 404. The CPU 401 also loads a program (instructions) read from the HDD 403 into the RAM 402, and executes the instructions loaded into the RAM 402, thereby functioning as a processing unit that executes the processing of the flowcharts described below.

[0034] FIG. 4B shows an example of the configuration of the user terminal 203. The user terminal 203 includes a CPU 411, a RAM 412, a HDD 413, a network I / F 414, and an input / output I / F 415. The CPU 411 controls the entire user terminal 203. The CPU 411 can control data exchange with the RAM 412, the HDD 413, the network I / F 414, and the input / output I / F 415. The display 420 is configured with a display device such as a liquid crystal display, and displays display information received from the input / output I / F 415. The input device 430 is configured with a pointing device such as a mouse or a touch panel, and a keyboard, and receives operations from the user and transmits operation information to the input / output I / F 415. The HDD 413 can store image processing results received from the image processing server 202 via the network I / F 414. In this embodiment, the CPU 411 loads an application program read from the HDD 413 into the RAM 412 and executes it, thereby displaying display information and accepting user operations on the operation unit I / F 415.

[0035] 5A and 5B are examples of document images 500 generated by scanning a document with an image forming device. The example of document image 500 in Fig. 5A is an image obtained by reading a taxi receipt with image forming device 201.

[0036] In this embodiment, a process for extracting company name information from a document image will be described. In a receipt such as that shown in FIG. 5 , the company name information refers to the issuer of the receipt, which is used to estimate expense items such as transportation expenses in accounting systems and expense settlement processes. While the company name information typically indicates a unique company name, in this embodiment, if a unique company name cannot be identified, information on the company's classification type (company name type information), such as a railway company, airline, taxi, or toll road, is output. If the classification type of the receipt can be determined to be a railway company, airline, taxi, or other type, it is possible to infer that the payment is related to transportation expenses, facilitating input operations into the accounting system. In other words, by displaying company name information in the summary field of the accounting system when a unique company name can be identified, or by displaying company classification type information even when a unique company name cannot be identified, the user's input operations can be supported. Ideally, displaying the unique company name information is more detailed information, so it is desirable to do so. However, if it is difficult to extract a unique company name for various reasons, information other than the company name is used to identify and display the company's classification type. There are several ways to extract company names, for example, by searching the document recognition results using a dictionary of company names, or by extracting patterns from the phone numbers in the document recognition results using character string rules and searching a dictionary that links company names and phone numbers.With either method, if the characters corresponding to the company name or phone number cannot be recognized or are recognized incorrectly, it will be impossible to identify a specific company name.

[0037] 6 is a flowchart illustrating a company name type extraction process (a process for determining a company classification type) according to this embodiment. The following describes a process in which, for example, the document image 500 in FIG. 5(a) is input and the company name type (a company classification type) is determined to be "taxi."

[0038] In S601, the image processing server 202 acquires the document image 500 obtained by scanning from the image forming apparatus 201.

[0039] In S602, the CPU 401 of the image processing server 202 analyzes the document image 500 to detect character areas and performs character recognition processing on the character areas. As a result of the character recognition processing, the CPU 401 identifies the coordinates of the character areas, the coordinates of each character in the character areas, and the character codes of the character recognition results. The array of character codes obtained here for each character area is called an OCR string.

[0040] In S603, the CPU 401 of the image processing server 202 loads the information extraction rules stored in the HDD 403 into the RAM 402. The information extraction rules include a dictionary for extracting item values, pattern information, and conditions for outputting item values.

[0041] FIG. 7 shows an example of an information extraction rule. Table 700 in FIG. 7(a) is a table listing dictionaries defined as information extraction rules. Each row represents a dictionary definition, and includes a column for number, a column for dictionary name, and a column listing the corresponding search strings. Dictionary 701 is named "Total Key" and is associated with a list of search strings that may provide clues for the total amount field, such as "Total Amount," "Payment Amount," and "Receipt Amount." Dictionary 702 is named "Telephone Key" and is associated with a list of search strings that may provide clues for the telephone number field, such as "TEL" and "Telephone." Dictionary 703 is named "Taxi Term A" and is associated with a list of search strings that may provide clues for taxi receipts, such as "taxi" and "hire car." Dictionary 704 is named "Taxi Term B" and is associated with a list of search strings that may provide clues for taxi receipts, such as "Vehicle Number," "Vehicle Number," and "Radio Number." Dictionary 705 is named "Taxi Term C" and is associated with a list of search strings that may be clues to taxi receipts, such as "fare" and "meter charge." Note that dictionaries 701 to 705 in this embodiment are examples for the purpose of explanation, and are not limited to these.

[0042] Table 710 in FIG. 7(b) is a diagram showing a list of patterns defined as information extraction rules in a table. Each row shows the definition of a pattern, and there are columns for the number, the pattern name, and the regular expression showing the search pattern. Pattern 711 has the name "amount" and extracts a string pattern that matches the regular expression "￥?[￥d,]+円". The regular expression for this pattern 711 means a string pattern where the "￥" symbol exists 0 or 1 at the beginning, numbers and commas are consecutive 1 or more times, and the character "円" exists at the end. For example, a string such as "￥1,000円" matches this pattern. The regular expression for the amount is an example for explanation, and other patterns for extracting strings representing the value of the amount, patterns considering misrecognition and fluctuations in the character recognition results, or patterns having multiple search patterns may be used. Also, the method of expressing the search pattern is not limited to regular expressions. Pattern 712 has the name "phone number" and has a string pattern that matches the regular expression "0￥d{1,3}[-(]￥d{2,4}[-)]￥d{4}". The regular expression for this pattern 712 means a string pattern where the first character is "0", then numbers are consecutive 1 to 3 times, then there is 1 character of "-" or "(", then numbers are consecutive 2 to 4 times, then there is 1 character of "-" or ")", and finally numbers are consecutive 4 times. For example, a string representing a phone number such as "03(1234)5678" matches this pattern. The regular expression for the phone number is an example for explanation, and like the search pattern for the amount, it is not limited to this.

[0043] In S604, the CPU 401 of the image processing server 202 performs a text search for a character string that matches the search character string conditions of the dictionary list 700 loaded in S603 from among the OCR character strings of the character recognition results obtained in S602. FIG. 5(b) shows the results of a text search from the OCR character string. Search result 511 is the result of searching for "vehicle number" from taxi term B in dictionary 704. Search result 512 is the result of searching for "fare" from taxi term C in dictionary 705. Search result 513 is the result of searching for "total" from the total key in dictionary 701. Search result 514 is the result of searching for "TEL" from the telephone key in dictionary 702.

[0044] In S605, the CPU 401 of the image processing server 202 performs a text search for character strings that match the search patterns in the pattern list 710 loaded in S603 from the OCR character strings of the character recognition results obtained in S602. Search results 515 and 516 in Fig. 5(b) are the results of searching for amounts that match the search pattern in pattern 711. Search result 517 in Fig. 5(b) is the result of searching for phone numbers that match the search pattern in pattern 712.

[0045] Table 720 in FIG. 7(c) is an example of an output condition for an item value based on the text search results of S604 to S605. Table 720 has a column for the output condition number, a column for the item name to be output, a column for type, a column for the output condition, a column indicating the text search result to be judged, and a column for output value. The output value indicates what item value is output when the output condition is met, and may be the "value search result value (the value of the text search result judged to be a value)" or a character string of the item value such as "taxi." The "search result position" listed in the type column in table 720 indicates that the position of the text search result is used for condition judgment. Furthermore, the "search result logical operation" listed in the type column indicates that the result is calculated by a logical operation using the text search result.

[0046] When the type column in table 720 is "Search result position," the condition column indicates under what conditions the positional relationship is output. For example, the condition column "Value is to the right of the key" indicates that the condition is met when the two text search result positions, the key and the value, have a value to the right of the key. For example, output condition 721 in FIG. 7(c) is an output condition that outputs a value to the right of the key, with the item name "Total Amount" and the text search result of "Total Key" as the key and the text search result of "Amount" as the value. This output condition 721 describes as a rule that, on a receipt, the value of the "Amount Pattern" 711, written to the right of the character strings such as "Total Amount" and "Payment Amount" in the "Total Key" dictionary 701, is likely to be the total amount. Similarly, output condition 722 for phone number is an output condition that outputs a value to the right of the key, with the text search result of "Phone Key" as the key and the text search result of "Phone Number" as the value.

[0047] Furthermore, when the type column in table 720 is "Search Result Logical Operation," the condition column indicates the logical operation expression that the search results must satisfy before they are output. "Logical operation result is true" indicates that the search results are output when the logical operation result of the specified logical expression is true. For example, output condition 723 in FIG. 7(c) uses search result logical operation as the type, and outputs a taxi if the logical expression "taxi term A | (taxi term B & taxi term C)" is true. This output condition 723 describes a rule that specifies that if "taxi term A" in dictionary 703 exists or if a string belonging to "taxi term B" in dictionary 704 and "taxi term C" in dictionary 705 exists, the output value of the company name type item is a taxi. In other words, "taxi term A" in dictionary 703 is a strong term that can be used alone to identify a taxi, while "taxi term B" in dictionary 704 and "taxi term C" in dictionary 705 are terms that, although individually weak evidence of a taxi, are highly likely to be a taxi when they appear together.

[0048] In S606, the CPU 401 of the image processing server 202 determines and outputs the item values ​​to be output from the output conditions of the rules loaded in S603, i.e., the output values ​​corresponding to the item names of the total amount, telephone number, and company name type. The item value output process in S606 will be described in detail using the flowchart in FIG. 8.

[0049] FIG. 8 is a flowchart illustrating the details of the item value output process in this embodiment.

[0050] In S801, the CPU 401 of the image processing server 202 sequentially selects the output conditions of the rules loaded in S603, and proceeds to S802. For example, it is assumed that the output condition 721 in Fig. 7C is selected first.

[0051] In S802, the CPU 401 of the image processing server 202 acquires the text search results that are the target of the output condition from the text search results of S604 and S605. Since the output condition 721 targets the text search results of the "total key" that serves as the key and the text search results of the "amount" that serves as the value, the CPU 401 acquires the corresponding text search results 513 of the "total key" and the text search results 515 and 516 of the amount, respectively.

[0052] In S803, the CPU 401 of the image processing server 202 determines the type of output condition, and if it is "search result position", proceeds to S804, and if it is "search result logical operation", proceeds to S809. If the output condition 721 is selected, the type is "search result position", so proceeds to S804.

[0053] In S804, the CPU 401 of the image processing server 202 selects one text search result that can be a key and one text search result that can be a value respectively, and creates a combination. Since the key in the output condition 721 is the text search result 513, and the values are the text search results 515 and 516, there are two combinations: the text search result 513 and the text search result 515, and the text search result 513 and the text search result 516. First, select the text search result 513 and the text search result 515, and proceed to S805.

[0054] In S805, the CPU 401 of the image processing server 202 determines whether the combination of the key and the value selected in S805 meets the position condition. If it meets, proceed to S806; if not, proceed to S807. Since the condition of the output condition 721 is that "the value is to the right of the key", it is determined whether the text search result 515 is located to the right of the text search result 513 using the coordinate values of the text search results. In this embodiment, the determination of whether "the value is to the right of the key" is as follows in the coordinate system with the upper left origin. When the rectangular coordinates of the text search result of the key are the upper left coordinates (KX1, KY1) - the lower right coordinates (KX2, KY2), and the rectangular coordinates of the value are the upper left coordinates (VX1, VY1) - the lower right coordinates (VX2, VY2), it is sufficient to satisfy "KX2 < VX1", "KY2 > VY1", and "KY1 < VY2". That is, the left end X coordinate of the text search result of the value is larger than the right end X coordinate of the text search result of the key, and the Y coordinate values of the rectangles of the text search results of the key and the value overlap. The method for determining whether it is to the right in this embodiment is an example for explanation, and other methods may be used. Since the text search result 513 and the text search result 515 do not meet the condition, proceed to S807.

[0055] In S807, the CPU 401 of the image processing server 202 determines whether there are any remaining key and value combinations, and if there are, proceeds to S804, and if not, proceeds to S808. After determining the combination of text search result 513 and text search result 515, there are still combinations remaining, so proceeds to S804, selects the next combination, text search result 513 and text search result 517, and proceeds to S805. In S805, key text search result 513 and value text search result 516 are located to the right, so proceeds to S806.

[0056] In S806, the CPU 401 of the image processing server 202 determines the output value because the text search result 513 and the value text search result 516 satisfy the output condition, and proceeds to S807. That is, because the text search result 513 as the key and the text search result 516 as the value satisfy the conditions of the output condition 721, the CPU 401 determines that the text search result 516 is to be output as "total amount." Thereafter, in S807, because all combinations of the text search results for "total amount" have been processed, the process proceeds to S808.

[0057] In S808, the CPU 401 of the image processing server 202 determines whether all output conditions have been processed, and if there are unprocessed output conditions, the process proceeds to S801, and if not, the process ends.

[0058] When the determination of output condition 721 is completed, processing of output conditions 722 and 723 still remains, so the process proceeds to S801, and the next output condition 722 is set as the processing target, and the process proceeds to S802. Since output condition 722 is a search result position like output target 721, processing is performed in the same manner from S802 to S808. The combination of text search result 514 as the key and text search result 517 as the value is evaluated, and as this combination matches the condition, text search result 517 is determined as the output value for "telephone number".

[0059] After the determination process for output condition 722 is completed, in S801 output condition 723 is set as the processing target and the process proceeds to S802. In S802, text search results for "taxi term A" in dictionary 703, "taxi term B" in dictionary 704, and "taxi term C" in dictionary 705 are obtained from the text search results of S604 and S605. As a result, no text search results are obtained for "taxi term A" in dictionary 703, text search result 511 is obtained for "taxi term B" in dictionary 704, and text search result 512 is obtained for "taxi term C" in dictionary 705, and the process proceeds to S803. In S803, the type of output condition 723 is a search result logical operation, so the process proceeds to S809.

[0060] In S809, the CPU 401 of the image processing server 202 determines whether the logical expression of the output condition 723 is met, and if so, proceeds to S810, otherwise proceeds to S808. The output condition 723 is conditioned on whether the logical expression "Taxi term A | (Taxi term B & Taxi term C)" is true. When the number of text search results acquired in S802 is applied to this logical expression, the result is "0 | (1 & 1)", which means that the logical expression is true and the condition is met, so proceeds to S810.

[0061] In S810, the CPU 401 of the image processing server 202 determines the output value because the output condition is satisfied, and proceeds to S808. In the case of output condition 723, "taxi" is determined as the output value for "company name type (company classification type)". In the subsequent S808, all output conditions have been processed, so the processing ends.

[0062] In this embodiment, for the sake of explanation, one output condition (output condition 723) is set for the item name of the company name type (company classification type), but normally, output conditions should be added in the same way as the number of types of company name types to be distinguished.

[0063] Returning to the description of the flowchart in FIG. 6 , in S607, the CPU 401 of the image processing server 202 checks whether the telephone number output in S605 exists in the company name dictionary. If it does, it identifies the company name from the search results and outputs the search results as the item name "company name." The company name dictionary is a database that links telephone numbers with company names, and it is possible to identify a company name from a telephone number. However, it is difficult to cover all company names, and if there is an OCR recognition error, the company name cannot be searched. For example, in the receipt 500, the telephone number "03-1234-5678" is obtained, but if the company name cannot be obtained from the company name dictionary, the company name is left unnamed, and the process proceeds to S608. Note that while the method of searching for a company name based on a telephone number is used here, this is not limiting. As mentioned above, it is also possible to search for the company name itself from the character recognition results, in addition to searching based on a telephone number.

[0064] In S608, the CPU 401 of the image processing server 202 determines final company name information based on the "company name" and "company name type (company classification type)" determined in S601 to S608. If there is an output value for "company name", that value is output; if there is no output value for "company name", and if there is an output value for "company name type (company classification type)", the value of "company name type" is output; if there is neither, no output is made. In this case, since there is no output for "company name" and "company name type" has been output, "taxi", which is the "company name type (company classification type)", is determined as the final output of "company name information".

[0065] As a result of applying the processing of the flowcharts in Figures 6 and 8, in the example of receipt 500, three items and their values ​​are output: "Total amount" is text search result 516, "Telephone number" is text search result 517, and "Company name information" is "Taxi."

[0066] 9 is an example of a receipt from a gas station. A case where the processes of S601 to S608 are performed on this receipt 900 using the information extraction rules of FIG. 7 will be described. Performing text searches in S603 and S604 on the document recognition results results in the following: no results for "taxi term A" in dictionary 703, "taxi term B" in dictionary 704 results in "vehicle number" in text search result 901, and no results for "taxi term C" in dictionary 705. Applying the number of text search results to the logical expression of output condition 723 results in "0|(1&0)," which does not match the condition in S809, and therefore "company name type" is not output.

[0067] As explained above, by applying this embodiment, it is possible to search a dictionary and patterns from the character string of the document recognition result, and then perform a logical operation on the result to output "taxi" as the company name type (company classification type). Also, for receipts other than taxis that have similar terms, it is possible to prevent erroneous determination and not output the company name type.

[0068] <Second embodiment> In the first embodiment, the company name type (company classification type) was determined from the occurrence of terms in the document recognition results and the number of results of their logical operations, but other conditions, such as input image information such as the size of a receipt, may also be used.

[0069] FIG. 10(a) shows an example of an input document image, which is a railway receipt 1000. The receipt 1000 only contains a logo 1001, which is the name of the railway company and a clue that it is a railway receipt. Images with decorative elements such as logos are usually difficult to recognize using OCR, so it is difficult to directly identify the company name from the OCR string. However, receipts issued by railway companies are sometimes output in the same size as tickets, and the size of the receipt may be distinctive. Below, we will explain a method of using image size as an output condition type.

[0070] Table 1010 in Fig. 10(b) shows the information extraction rules in this embodiment. Output conditions 721 to 723 are the same as the information extraction rules in Fig. 7(c). Output condition 1011 is a condition for identifying a company name type, and if the image size is 85 mm and the height is 58 mm, "railway" is output. "Image size" is a new type, and if the size information of the input image is the specified size, "railway" is output.

[0071] 10(c) shows the image information acquired in S601 of the receipt 1000. The image information 1020 has a resolution of 300 dpi in both the width and height directions, and is 1000 pixels wide and 680 pixels high.

[0072] Fig. 11 is a flowchart illustrating the item value output process using the image processing size of this embodiment. The steps (S801 to S810) with the same numbers as those in the flowchart of Fig. 8 are the same as the corresponding steps in Fig. 8, and therefore detailed description thereof will be omitted.

[0073] In S1101, the CPU 401 of the image processing server 202 determines the type of output condition. If it is "search result position", the process proceeds to S804; if it is "search result logical operation", the process proceeds to S809; if it is "image size", the process proceeds to S1102.

[0074] In S1102, the CPU 401 of the image processing server 202 determines whether the size of the detected document image matches the conditions, and if it does match, proceeds to S1103, and if it does not match, proceeds to S808. Note that in this embodiment, the image to be processed is assumed to be an image that has been cropped according to the outline of the document, but if the image to be processed is an uncropped image, it may be configured to detect the outline of the document to obtain the document size and determine whether it matches the conditions.

[0075] In S1103, the CPU 401 of the image processing server 202 determines an output value. When the receipt 1000 is processed according to the flowchart of Fig. 11, in S1102 it is determined whether the size meets the output condition 1011. A width of 1000 pixels is 85 mm at a resolution of 300 dpi, and a height of 680 pixels is 58 mm at a resolution of 300 dpi, which meets the image size condition, so "railway" is output as the company name type (company classification type).

[0076] In the example of output condition 1011, only image size was used as a condition, but this is not limited to this. A composite condition that combines not only simple image size but also logical operation results from text search results from OCR character strings may also be used.

[0077] In addition to the image size, the characteristics of the original document, such as the color of the paper and the background pattern, may also be used.

[0078] As described above, by applying this embodiment, even if the characteristics of the company name type cannot be obtained from the OCR character string, the company name type can be output by using image information such as image size.

[0079] <Third embodiment> In the first embodiment, if both the company name type (company classification type) and the unique company name were identified, the unique company name was output as information related to the final company name in S608. Normally, between the company name type and the unique company name, the unique company name provides more detailed information, so in expense settlement applications, the company name type is used as an aid when it is difficult to extract the company name. However, in certain cases, it may be more desirable to output the company name type (classification type).

[0080] Receipt 1200 in FIG. 12 is a receipt for an airline ticket in this embodiment. Receipt 1200 is a receipt for an airline ticket paid at a convenience store, and the company name and telephone number listed are those of the convenience store. If a telephone number or company name dictionary is used to output the company name, the company name will be "CamonMart," the name of this convenience store. However, in expense settlement operations, outputting the type "air ticket" rather than the specific company name of the convenience store is more appropriate as support information for inputting the purpose of use.

[0081] FIG. 13 shows an example of a rule in this embodiment. Table 1300 in FIG. 13(a) is a list of dictionaries having dictionaries 1301, 1302, 1303, and 1304. Dictionary 1301 is a dictionary of airline ticket terms with search strings "AIRLINE" and "airline", which are clues that the company name type (purchase type) is an airline ticket. Dictionary 1302 is a dictionary of convenience store terms with search strings "convenience store", which are clues that the receipt was issued at a convenience store. Dictionary 1303 is a dictionary of web terms with "display date", which are clues that the receipt was issued on the web. Dictionary 1304 is a dictionary of agency terms with "travel", "agency", and "agency", which are clues that the receipt was issued by a travel agency.

[0082] Table 1310 in FIG. 13(b) is a list of output conditions using these terms. Output condition 1311 has an item name of "company name type (purchase type)," a condition type of "search result logical operation," a condition of "airline ticket term & convenience store term," and outputs "airline ticket (convenience store)" as the output value. Output condition 1312 has an item name of "company name type (purchase type)," a condition type of "search result logical operation," a condition of "airline ticket term & web term," and outputs "airline ticket (online)" as the output value. Output condition 1313 has an item name of "company name type (purchase type)," a condition type of "search result logical operation," a condition of "airline ticket term & agency term," and outputs "airline ticket (agency)" as the output value. The item names of output conditions 1311, 1312, and 1313 are defined as company name type (purchase type) because, in addition to company name type information, supplier information indicating where the item was purchased, i.e., where payment was made, is also output. Output condition 1311 outputs "airline ticket (convenience store)", i.e., an air ticket paid for at a convenience store, if there is a term indicating a convenience store in addition to a term indicating an air ticket. Output condition 1312 outputs "airline ticket (online)", i.e., an air ticket paid for online, if there is a term indicating a web-issued ticket in addition to a term indicating an air ticket. Output condition 1313 outputs "airline ticket (agency)", i.e., an air ticket paid for by an agency, if there is a term indicating an agency-issued ticket in addition to a term indicating an air ticket.

[0083] 6 and 8 based on the dictionary 1300 and the output condition 1310. The receipt 1200 contains the text search result 1201 "airline" searched in the dictionary 1301 and the text search result 1202 "convenience store" searched in the dictionary 1302. Therefore, the output value of "airline ticket (convenience store)" is obtained as the company name type (purchase type) matching the condition of the output condition 1311.

[0084] Table 1320 in FIG. 13(c) is a table listing conditions for determining the company name information to be output when the company name information is determined in S608. The company name information determination condition list 1320 has a column for a condition number, a column for a condition indicating whether the company name was acquired, a column for a condition indicating whether the company name type (purchase type) was acquired, a column for a condition indicating what the purchase type was, and a column for a condition indicating which output value to use as the company name information output when the conditions are met. The table is judged in order from the top to the bottom, and the company name information output of the matching row is used. Company name information determination condition 1321 indicates that if there is a "company name," there is a "company name type," and the "purchase type" is a convenience store, then the "company name type (purchase type)" is output as the company name information. Company name information determination condition 1322 indicates that if there is a "company name," there is a "company name type," and the "purchase type" is an agency, then the "company name type (purchase type)" is output as the company name information. Company name information determination condition 1323 indicates that if a "company name" is present and a company name type (purchase type) is not acquired, then "company name" is used as the company name information. Company name information determination condition 1324 indicates that if there is no "company name" and there is a "company name type," then "company name type (purchase type)" is output. Company name information determination condition 1325 indicates that if neither a "company name" nor a "company name type" is acquired, then there is no company name information.

[0085] When company name information is determined based on this table, the receipt 1200 matches the company name information determination condition 1321, and the purchase type "air ticket" is output as the company name information, instead of the company name.

[0086] As explained above, by applying this embodiment, not only is it possible to simply use a unique company name whenever a company name is obtained, but also to output values ​​required for expense settlement operations by prioritizing values ​​of the company name type under certain conditions.

[0087] <Other Examples> The present invention can also be realized by executing the following process: software (program) that realizes the functions of the above-described embodiments is supplied to a system or device via a network or a computer-readable storage medium, and the computer (or CPU, MPU, etc.) of the system or device executes the program.

Claims

1. a first identification means for identifying information about a company that issued the document image based on one or more character strings recognized from the document image by character recognition processing of the document image and on predefined conditions; a second identification means for identifying the name of the company that issued the document image, the name being included in the one or more character strings; an output means for outputting information about the company name; and The output means When the company name of the company is identified by the second identification means, the identified company name is output as information on the company name; If the company name of the company is not identified by the second identification means and information about the company is identified by the first identification means, the information about the identified company is output instead of the company name as information about the company name.

1. An image processing device comprising:

2. The image processing device according to claim 1 , wherein the predefined condition includes a condition based on whether a predetermined character string is present in the one or more character strings.

3. The image processing device according to claim 1 , wherein the predefined conditions include conditions based on logical operations.

4. The image processing device according to claim 1 , wherein the predefined conditions include a condition based on an image size.

5. 2. The image processing device according to claim 1, wherein the first identification means identifies information about the company that issued the document image and information about the supplier based on the one or more character strings and another predefined condition.

6. 2. The image processing device according to claim 1, wherein the predefined conditions include conditions based on a database search.

7. A program for causing a computer to function as each of the means of the image processing apparatus according to any one of claims 1 to 6.

8. 1. An image processing method, comprising: a first identification step of identifying information about the company that issued the document image based on one or more character strings recognized from the document image by character recognition processing of the document image and on predefined conditions; a second identification step of identifying the company name of the company that issued the document image, the company name being included in the one or more character strings; an output step of outputting information about the company name; and In the output step, If the company name of the company is identified in the second identification step, output the identified company name as the company name information; If the company name of the company is not identified in the second identification step, and information about the company is identified in the first identification step, output the information about the identified company instead of the company name as information about the company name. An image processing method that causes a computer to perform the following:

Images