Image processing device, image processing method, and program

Abstract

To enable identification of the classification type of a document image using predefined rule-based conditions.SOLUTION: An image processing device disclosed herein is configured to obtain a character recognition result by performing character recognition processing on a document image, and identify the classification type of the document image on the basis of character strings included in the character recognition result and predefined conditions.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to an image processing apparatus that specifies a classification type of a document image. [Background technology]

[0002] In recent years, it has become common to digitize documents by scanning them using an image scanner attached to an MFP (Multi Function Printer: a multifunction machine with functions such as printing, copying, and faxing). It has also become common to digitize documents by photographing them using a digital camera or a camera function of a mobile device such as a smartphone. In this way, it has become easy to obtain a document image (scanned document image) by optically scanning or photographing a document containing handwritten characters or type. Furthermore, by performing optical character recognition processing (OCR processing) on ​​this document image, the character image in the document image can be converted into a character code that can be used by a computer. By using such character recognition processing, it has become possible to automate the work of converting paper documents (receipts, invoices, etc.) into digital data and inputting the data into a system (for example, expense settlement work). This is expected to improve the productivity of data entry work.

[0003] Figure 1(a) shows the flow of information between the accounting system and various linked systems.

[0004] The accounting system 101 is a system that supports accounting and bookkeeping work. The accounting system 101 records transactions of cash, deposits, assets, products, etc. within a company while classifying them from a management or tax perspective by the accounting and bookkeeping staff, and is an electronic version of the various forms of ledgers that were previously used. The recorded contents are stored as accounting processing results 102, and are output as various ledgers 107 as necessary. In addition, the accounting system needs to classify expenses according to the contents and purpose of transactions, and uses expense item codes to identify each expense item.

[0005] There are also a cash flow management system 103 for managing cash and deposits, a budget management system 104 for managing the budgets of each department, an inventory management system 105 for managing product inventory, and an asset management system 106 for managing various assets. When transactions of cash, deposits, assets, products, etc. occur, the increase or decrease is recorded in each management system (103, 105, 106). Conventionally, accountants and bookkeepers refer to the information of each transaction recorded in each management system (103, 105, 106) and record it in the accounting system 102. Furthermore, the budget management system 104 uses a budget code to identify each budget when managing the budget.

[0006] The form document 110 in FIG. 1(b) is an example of a receipt (form document) issued when purchasing office supplies. This is evidence that proves that the office supplies have been purchased. Various information is recorded in each section of the form document 110. For example, the document title 111 is the section that becomes the title of the form document 110, and describes that this form document is a "receipt." The issue date 112 indicates the date that this form document was issued. In accounting and bookkeeping work, when the receipt for this form document is saved as evidence, the information on the issue date 302, "November 12, 2020," may be used to identify this form document.

[0007] Issuer 113 describes the company name, address, and telephone number as information about the company (issuer) that created and issued this form document. When receiving a form document and processing accounting or bookkeeping, it is necessary to clarify what was purchased and for what purpose in order to classify expenses, and the company name information of issuer 113, "AAA Office Machinery Co., Ltd.", may be used at that time. Also, when a receipt for this form document is saved as evidence, the company name information may also be used to identify this form document.

[0008] The recipient 114 lists the name of the company that made the purchase and payment. Additionally, the total amount of the purchase and payment is listed in the total amount 115. When accounting or bookkeeping is carried out for the document, this amount (and the tax amount, if necessary) will be used as the transaction amount. Details 116 lists the detailed information being billed, and information such as the unit price, quantity, and price for each item is listed. Additionally, summary 117 shows that the total amount 115 is calculated by adding information such as tax to the subtotal of each price.

[0009] Fig. 1(c) is an example of ledger information that is generally recorded in the accounting system 101 by an accountant or bookkeeper. The ledger transaction record 120 is a record of transactions used in accounting and bookkeeping, and is shown in the form of a table. Each line in this table represents one transaction, with the transaction date at the top and other information listed.

[0010] An example of recording transaction information in a certain department (division) will be described below. For example, in advance, the budget management system 104 budgets and records the purchase details and amounts of office supplies used in the business as the budget for that department. Then, in accordance with the purchase details, the department purchases office supplies from a company named "AAA Office Machines Co., Ltd." on "November 12, 2020" and pays in cash through the deposit and withdrawal management system 108. In this case, it is necessary to record that the purchase amount has been withdrawn. In addition, in order to prove that the purchase and payment have been made, the company named "AAA Office Machines Co., Ltd." has issued and obtained a receipt 110 as evidence. In this case, the following contents are recorded in the three columns (6th to 8th columns from the left) after "Credit" in the transaction record 120 in the ledger. That is, the following is entered in the three columns (second to fourth columns from the left) after the "debit" in the transaction record 120 in the ledger: "Cash" and the code "100" indicating the type of cash in the withdrawal record of the cash receipt management system, the department name "Kamata Branch" and the budget code "221" which are budget information, and the amount "7,700" yen that was paid. In addition, the following contents are entered in the three columns (second to fourth columns from the left) after the "debit" in the transaction record 120 in the ledger. In other words, it is entered that "Kamata Branch" used "7,700" yen as office supplies expenses. Therefore, the person in charge of accounting enters the expense item "Office supplies expenses" and the expense item code "300" in the "debit", the department name "Kamata Branch" and the budget code "221" in the "department", and "7,700" yen in the "amount". In addition, "AAA Office Machinery Co., Ltd." is entered in the "summary" as the basis for classifying the expense item, in order to facilitate linking to the form document 110. By recording in this way, it becomes possible to correlate and record individual facts, such as which department's budget was used to withdraw cash from assets and purchase office supplies, as transactions used as office supply expenses. When recording, the accountant or bookkeeper checks each fact, including supporting documents, classifies the expenses, and records them in the accounting system 101. The accumulated contents of these records are the accounting processing results 102.

[0011] When performing such work, if there was a function to automatically transcribe the contents (date, amount, company name, etc.) of the form document 110 into the accounting system 102, it would be possible to reduce the workload of the accounting and bookkeeping staff. For this reason, in recent years, a method has been considered in which documents such as evidence are read as electronic images using an image scanner, and the written information is extracted and transcribed by performing character recognition processing.

[0012] However, on documents such as receipts and invoices, the company name may be written in a logo or special font, or an unclear character image may be stamped on the document, which can result in erroneous recognition during character recognition processing.

[0013] In Patent Document 1, sorting elements including at least date, business partner, amount, application, and appearance such as size and color are extracted from a learning image, and accounting items for the sorting elements are machine-learned to create a journal entry AI. Then, when processing a document image, the sorting elements are extracted from the document image, and the sorting AI is used to select an accounting item. [Prior art documents] [Patent documents]

[0014] [Patent Document 1] JP 2018-097813 A Summary of the Invention [Problem to be solved by the invention]

[0015] The method of Patent Document 1 requires learning by machine learning in advance, and therefore requires a large amount of image data for learning. Furthermore, if the rules for journalizing expense items differ for each user, learning data must be prepared for each user. [Means for solving the problem]

[0016] In order to solve the above problem, the image processing device of the present invention is characterized by comprising: a character recognition means for obtaining a character recognition result by performing character recognition processing on a document image; and a first identification means for identifying a classification type based on a character string included in the character recognition result and predefined conditions. Effect of the Invention

[0017] In the present invention, rule-based conditions are defined in advance to identify classification types that serve as clues to expense items, rather than expense items that may have different definitions for different users, and the classification types of companies identified based on the conditions are used when processing document images. [Brief description of the drawings]

[0018] [Figure 1] This is a diagram showing an overview of processing in accounting and bookkeeping work, an example of a document image, and an example of input content for that processing. [Diagram 2] FIG. 1 is a diagram illustrating a system configuration of an image processing system. [Diagram 3] FIG. 2 is a diagram illustrating a hardware configuration of an image forming apparatus 101. [Figure 4] 2 is a diagram illustrating the hardware configuration of an image processing server 102 and a user terminal 103. FIG. [Diagram 5] FIG. 4 is a diagram illustrating an example of a receipt according to the first embodiment. [Figure 6] FIG. 2 is a diagram showing an overall flow of processing according to the first embodiment. [Figure 7] FIG. 4 is a diagram illustrating an example of a processing rule according to the first embodiment. [Figure 8] FIG. 11 is a diagram showing the flow of an item value output process according to the first embodiment. [Figure 9] FIG. 4 is a diagram illustrating an example of a receipt according to the first embodiment. [Figure 10] FIG. 11 is a diagram illustrating an example of input information according to the second embodiment. [Figure 11] FIG. 11 is a diagram showing the flow of an item value output process according to the second embodiment. [Figure 12]FIG. 13 is a diagram illustrating an example of a receipt according to the third embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of a rule according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the embodiment does not limit the present invention, and all of the configurations described in the embodiment are not necessarily essential means for solving the problems of the present invention.

[0020] <First embodiment> In this embodiment, a data input support device that extracts and displays item names and item values ​​from a document image will be described.

[0021] 2 is a diagram showing an example of the configuration of an image processing system 200 according to the first embodiment. The image processing system 200 includes an image forming apparatus 201, an image processing server 202, and a user terminal 203, which are interconnected via a network 204 and capable of communicating with each other.

[0022] The image forming apparatus 201 can receive a print request (print data) of image data from the user terminal 203 and print it, read image data with a scanner provided in the image forming apparatus 201, and print the image data read with the scanner. The image forming apparatus 201 can also store print data received from the user terminal 203 and transmit image data read with the scanner of the image forming apparatus 201 to the user terminal 203 or the image processing server 202. The image forming apparatus 201 can also realize functions of known image forming apparatuses such as MFPs (Multifunction Peripherals). The user terminal 203 can display the image processing results received from the image processing server 202 in an application with a user interface, and can also process the results interactively according to instructions from a user. The image processing server 202 may be located in a cloud, that is, on the Internet.

[0023] In this embodiment, the user terminal 203 is assumed to be a general PC equipped with a display, keyboard, and mouse, but may also be, for example, a mobile terminal equipped with a touch panel.

[0024] The following describes a series of data input support processes in which an image forming device 201 scans a document such as a receipt, an image processing server 202 extracts information from the image of the document, and a user confirms and corrects the extracted information on a user terminal 203.

[0025] 3 is a diagram showing an example of the configuration of the image forming apparatus 201. The image forming apparatus 201 has a controller 301, a printer 302, a scanner 303, and an operation unit 304. The controller 301 has a CPU 311, a RAM 312, a HDD 313, a network I / F 314, a printer I / F 315, a scanner I / F 316, an operation unit I / F 317, and an expansion I / F 318.

[0026] The CPU 311 controls the overall operation of the image forming apparatus 201. The CPU 311 can control data transmission and reception with the RAM 312, the HDD 313, the network I / F 314, the printer I / F 315, the scanner I / F 316, the operation unit I / F 317, and the expansion I / F 318. The CPU 311 also loads a control program (instructions) read from the HDD 313 into the RAM 312, and executes the instructions loaded into the RAM 312.

[0027] The HDD 313 stores control programs executable by the CPU 311, setting values ​​used by the image forming apparatus 201, data related to processing requested by a user, etc. The RAM 312 has an area for temporarily storing instructions read by the CPU 311 from the HDD 313. The RAM 312 can also store various data necessary for executing instructions. For example, in image processing, the CPU 311 can perform processing by expanding input data in the RAM 312.

[0028] The network I / F 314 is an interface for performing network communication with devices in the image processing system 200. The network I / F 314 can notify the CPU 311 of the completion of data reception and can transmit data on the RAM 312 to the network 204.

[0029] The printer I / F 315 can transmit print data sent from the CPU 311 to the printer 302, and can transfer printer status received from the printer 302 to the CPU 311. The scanner I / F 316 can transmit image reading instructions sent from the CPU 311 to the scanner 303, transfer image data received from the scanner 303 to the CPU 311, and transfer status received from the scanner 303 to the CPU 311.

[0030] The operation unit I / F 317 can transmit user instructions input from the operation unit 304 to the CPU 311 and can transmit screen information for user operation to the operation unit 304. The expansion I / F 318 is an interface that enables an external device to be connected to the image forming apparatus 201. The expansion I / F 318 includes, for example, a Universal Serial Bus (USB) type interface. When an external storage device such as a USB memory is connected to the expansion I / F 318, the image forming apparatus 201 can read data stored in the external storage device and write data to the external storage device.

[0031] The printer 302 can print image data received from the printer I / F 315 on paper and can transmit the status of the printer 302 to the printer I / F 315. The scanner 303 can read information displayed on a paper placed on the scanner 302 in accordance with an image reading instruction received from the scanner I / F 316, digitize the information, and transmit it to the scanner I / F 316. The scanner 303 can also transmit its own status to the scanner I / F 316.

[0032] The operation unit 304 is an interface that allows a user to perform operations for issuing various instructions to the image forming apparatus 201. For example, the operation unit 304 is provided with a liquid crystal display having a touch panel, and provides an operation screen to the user and accepts operations from the user. Details of the operation unit 304 will be described later with reference to FIG.

[0033] 4(a) is a diagram showing an example of the configuration of the image processing server 202. The image processing server 202 has a CPU 401, a RAM 402, a HDD 403, and a network I / F 404. The CPU 401 controls the entire image processing server 202. The CPU 401 can control the exchange of data with the RAM 402, the HDD 403, and the network I / F 404. The CPU 401 also loads a program (instructions) read from the HDD 403 into the RAM 402, and executes the instructions loaded into the RAM 402, thereby functioning as a processing unit that executes the processing of a flowchart described later.

[0034] FIG. 4B is a diagram showing an example of the configuration of the user terminal 203. The user terminal 203 includes a CPU 411, a RAM 412, a HDD 413, a network I / F 414, and an input / output I / F 415. The CPU 411 controls the entire user terminal 203. The CPU 411 can control data transmission and reception between the RAM 412, the HDD 413, the network I / F 414, and the input / output I / F 415. The display 420 is configured with a display device such as a liquid crystal display, and displays display information received from the input / output I / F 415. The input device 430 is configured with a pointing device such as a mouse or a touch panel, and a keyboard, and receives an operation from a user and transmits operation information to the input / output I / F 415. The HDD 413 can store image processing results received from the image processing server 202 via the network I / F 414. In this embodiment, the CPU 411 loads an application program read from the HDD 413 into the RAM 412 and executes the program, thereby displaying display information and accepting user operations on the operation unit I / F 415.

[0035] 5A and 5B are examples of document images 500 generated by scanning a document with an image forming device. The example of document image 500 in FIG. 5A is an image obtained by reading a taxi receipt with image forming device 201.

[0036] In this embodiment, a process of extracting company name information from a document image will be described. In a receipt such as that shown in FIG. 5, the company name information is the issuer information of the receipt, which is used to estimate the expense item such as transportation expenses in the accounting system or expense settlement process. The company name information usually indicates a unique company name itself, but in this embodiment, when a unique company name cannot be identified, information on the classification type of the company (information on the company name type), such as a railway company, an airline company, a taxi, or a toll road, is output. If it is possible to determine that the classification type is a receipt for a railway company, an airline company, a taxi, or the like, it is possible to infer that the payment is related to transportation expenses, making it easier to perform input operations into the accounting system. That is, when a unique company name can be identified in the summary column of the accounting system, the company name information is displayed, and when a unique company name cannot be identified, the company classification type information is displayed, thereby making it possible to support the user's input operations. Originally, it is desirable to display the unique company name information, since it is more detailed information, but when it is difficult to extract a unique company name for various reasons, the company classification type is identified and displayed using information other than the company name. Methods for extracting company names include searching the document recognition results using a dictionary of company names, or extracting patterns from the phone numbers in the document recognition results using character string rules and searching a dictionary that links company names and phone numbers. Either method will not be able to identify a unique company name if the characters corresponding to the company name or phone number cannot be recognized or are recognized incorrectly.

[0037] Fig. 6 is a flowchart illustrating a company name type extraction process (a process for determining a company classification type) according to the present embodiment. In the following, for example, a process in which the document image 500 in Fig. 5(a) is input and the company name type (a company classification type) is determined to be "taxi" will be described.

[0038] In S601, the image processing server 202 acquires the document image 500 obtained by reading it with a scanner from the image forming apparatus 201.

[0039] In S602, the CPU 401 of the image processing server 202 analyzes the document image 500 to detect a character area, and executes character recognition processing on the character area. As a result of the character recognition processing, the CPU 401 identifies the coordinates of the character area, the coordinates of each character in the character area, and the character code of the character recognition result. The array of character codes obtained here for each character area is called an OCR string.

[0040] In S603, the CPU 401 of the image processing server 202 loads the information extraction rules stored in the HDD 403 into the RAM 402. The information extraction rules include a dictionary for extracting item values, pattern information, and conditions for outputting item values.

[0041] FIG. 7 shows an example of an information extraction rule. Table 700 in FIG. 7(a) shows a list of dictionaries defined as information extraction rules. Each row shows a dictionary definition, and there is a column of numbers, a column of dictionary names, and a column showing a list of search strings. Dictionary 701 is named "total key" and is associated with a list of search strings that can be clues to the total amount item, such as "total amount," "payment amount," and "receipt amount." Dictionary 702 is named "telephone key" and is associated with a list of search strings that can be clues to the telephone number item, such as "TEL" and "telephone." Dictionary 703 is named "taxi term A" and is associated with a list of search strings that can be clues to taxi receipts, such as "taxi" and "hire." Dictionary 704 is named "taxi term B" and is associated with a list of search strings that can be clues to taxi receipts, such as "vehicle number," "vehicle number," and "radio number." The dictionary 705 is named "Taxi Term C" and is associated with a list of search character strings that may be clues to taxi receipts, such as "fare" and "meter charge." Note that the dictionaries 701 to 705 in this embodiment are merely examples for the purpose of explanation, and are not limited to these.

[0042] Table 710 in Fig. 7(b) is a diagram showing a list of patterns defined as information extraction rules in a table. Each row shows the definition of a pattern, and there are columns for the number, the pattern name, and the regular expression showing its search pattern. Pattern 711 has the name "amount", and extracts a character string pattern that matches the regular expression "￥?[￥d,]+円". The regular expression of this pattern 711 means a character string pattern where there are 0 or 1 "￥" symbols at the beginning, 1 or more consecutive digits and commas, and the character "円" exists at the end. For example, a character string such as "￥1,000円" matches this pattern. The regular expression for the amount is an example for explanation, and other patterns that extract character strings representing the value of the amount, patterns considering misrecognition and fluctuations in the character recognition results, or patterns having multiple search patterns may be used. Also, the method of expressing the search pattern is not limited to regular expressions. Pattern 712 has the name "phone number" and has a character string pattern that matches the regular expression "0￥d{1,3}[-(]￥d{2,4}[-)]￥d{4}". The regular expression of this pattern 712 means a character string pattern where the first character is "0", then 1 to 3 consecutive digits, then 1 character of "-" or "(", then 2 to 4 consecutive digits, then 1 character of "-" or ")", and finally 4 consecutive digits. For example, a character string representing a phone number such as "03(1234)5678" matches this pattern. The regular expression for the phone number is an example for explanation, and like the search pattern for the amount, it is not limited to this.

[0043] In S604, the CPU 401 of the image processing server 202 performs a text search for a character string that matches the search character string conditions of the dictionary list 700 loaded in S603 from among the OCR character strings of the character recognition results obtained in S602. FIG. 5(b) shows the result of a text search from the OCR character string. Search result 511 is the result of searching for "vehicle number" from taxi term B in dictionary 704. Search result 512 is the result of searching for "fare" from taxi term C in dictionary 705. Search result 513 is the result of searching for "total" from the total key in dictionary 701. Search result 514 is the result of searching for "TEL" from the telephone key in dictionary 702.

[0044] In S605, the CPU 401 of the image processing server 202 performs a text search for character strings matching the search patterns in the pattern list 710 loaded in S603 from among the OCR character strings resulting from the character recognition obtained in S602. Search results 515 and 516 in Fig. 5(b) are the results of searching for amounts matching the search pattern of pattern 711. Search result 517 in Fig. 5(b) is the result of searching for a phone number matching the search pattern of pattern 712.

[0045] Table 720 in FIG. 7(c) is an example of an output condition for item values ​​based on the text search results of S604 to S605. Table 720 has a column for output condition number, a column for item names to be output, a column for type, a column for output conditions, a column for text search results to be judged, and a column for output value. The output value indicates what is output as an item value when the output condition is met, and may be "value search result value (text search result value judged as value)" or a character string of an item value such as "taxi". "Search result position" written in the type column in table 720 indicates that the position of the text search result is used for condition judgment. Also, "search result logical operation" written in the type column indicates that the result is calculated by logical operation using the text search result.

[0046] When the type column in table 720 is "search result position", the condition column indicates under what conditions the positional relationship is output. For example, the condition column "value is to the right of key" indicates that the two text search result positions of the key and value satisfy the condition when there is a value to the right of the key. For example, the output condition 721 in FIG. 7(c) is an output condition in which the item name is "total amount", the text search result of "total key" is the key, the text search result of "amount" is the value, and the value is to the right of the key. This output condition 721 describes as a rule that the value of the amount of "amount pattern" 711 written to the right of the character string such as "total amount" and "payment amount" in the "total key" dictionary 701 on a receipt is highly likely to be the total amount. Similarly, the output condition 722 for phone number is an output condition in which the key is the text search result of "phone key", the value is the text search result of "phone number", and the value is to the right of the key when there is a value.

[0047] In addition, when the type column in table 720 is "search result logical operation", the condition column indicates what logical operation formula the search results satisfy to be output. "Logic operation result is true" indicates that the search results are output when the logical operation result of the specified logical expression is true. For example, the output condition 723 in FIG. 7(c) is an output condition that outputs a taxi if the logical expression "taxi term A | (taxi term B & taxi term C)" is true, using search result logical operation as the type. This output condition 723 describes a rule that if "taxi term A" in dictionary 703 exists, or if a character string belonging to "taxi term B" in dictionary 704 and "taxi term C" in dictionary 705 exists, the output of the item value of the company name type is a taxi. In other words, "taxi term A" in dictionary 703 is a strong term that can be identified as a taxi on its own, and "taxi term B" in dictionary 704 and "taxi term C" in dictionary 705 are terms that are weak evidence of being a taxi when used alone, but are highly likely to be a taxi when they appear together.

[0048] In S606, the CPU 401 of the image processing server 202 determines and outputs the item values ​​to be output from the output conditions of the rules loaded in S603, i.e., the output values ​​corresponding to the item names of the total amount, telephone number, and company name type. The item value output process in S606 will be described in detail with reference to the flowchart in FIG.

[0049] FIG. 8 is a flowchart illustrating details of the item value output process in this embodiment.

[0050] In S801, the CPU 401 of the image processing server 202 sequentially selects the output conditions of the rules loaded in S603, and proceeds to S802. For example, it is assumed that the output condition 721 in Fig. 7C is selected first.

[0051] In S802, the CPU 401 of the image processing server 202 acquires the text search results that are the target of the output condition from the text search results of S604 and S605. Since the output condition 721 targets the text search results of the "total key" that is the key and the text search results of the "amount" that is the value, the corresponding text search results 513 of the "total key" and the text search results 515, 516 of the amount are acquired.

[0052] In S803, the CPU 401 of the image processing server 202 determines the type of the output condition, and if it is "search result position", proceeds to S804, and if it is "search result logical operation", proceeds to S809. If the output condition 721 is selected, the type is "search result position", so proceeds to S804.

[0053] In S804, the CPU 401 of the image processing server 202 selects one text search result that can be a key and one text search result that can be a value, respectively, and creates a combination. Since the key in the output condition 721 is the text search result 513 and the values are the text search results 515 and 516, there are two combinations: the text search result 513 and the text search result 515, and the text search result 513 and the text search result 516. First, the text search result 513 and the text search result 515 are selected, and the process proceeds to S805.

[0054] In S805, the CPU 401 of the image processing server 202 determines whether the combination of the key and value selected in S805 matches the position condition. If it matches, the process proceeds to S806; if it does not match, the process proceeds to S807. Since the condition of the output condition 721 is that "the value is to the right of the key", it is determined whether the text search result 515 is located to the right of the text search result 513 using the coordinate values of the text search results. In this embodiment, the determination of whether "the value is to the right of the key" is as follows in the coordinate system with the upper left origin. When the rectangular coordinates of the text search result of the key are the upper left coordinates (KX1, KY1) - the lower right coordinates (KX2, KY2) and the rectangular coordinates of the value are the upper left coordinates (VX1, VY1) - the lower right coordinates (VX2, VY2), it is sufficient to satisfy "KX2 < VX1", "KY2 > VY1", and "KY1 < VY2". That is, the left end X coordinate of the text search result of the value is larger than the right end X coordinate of the text search result of the key, and the Y coordinate values of the rectangles of the text search results of the key and the value overlap. The method for determining whether it is to the right in this embodiment is an example for explanation, and other methods may also be used. Since the text search result 513 and the text search result 515 do not meet the condition, the process proceeds to S807.

[0055] In S807, the CPU 401 of the image processing server 202 determines whether there are any remaining key and value combinations, and if there are, proceeds to S804, and if not, proceeds to S808. After determining the combination of text search result 513 and text search result 515, there are still combinations remaining, so proceeds to S804, and the next combination, the combination of text search result 513 and text search result 517, is selected, and proceeds to S805. In S805, key text search result 513 and value text search result 516 are to the right, so proceeds to S806.

[0056] In S806, the CPU 401 of the image processing server 202 determines the output value because the text search result 513 and the value text search result 516 satisfy the output condition, and proceeds to S807. That is, the text search result 513 as the key and the text search result 516 as the value satisfy the conditions of the output condition 721, so the text search result 516 is determined to be the "total amount" as the output. In the subsequent S807, all combinations of the text search results for "total amount" have been processed, so the process proceeds to S808.

[0057] In S808, the CPU 401 of the image processing server 202 determines whether or not all output conditions have been processed. If there are unprocessed output conditions, the process proceeds to S801; if not, the process ends.

[0058] At the point where the determination of output condition 721 is finished, the processing of output conditions 722 and 723 remains, so the process proceeds to S801, and the next output condition 722 is processed, and the process proceeds to S802. Since output condition 722 is a search result position like output target 721, the processing is similarly performed from S802 to S808. The combination of text search result 514 as the key and text search result 517 as the value is evaluated, and as this combination matches the condition, text search result 517 is determined as the output value for "phone number".

[0059] After the determination process of output condition 722 is completed, the process proceeds to S802 with output condition 723 as the processing target in S801. In S802, the text search results of "taxi term A" in dictionary 703, "taxi term B" in dictionary 704, and "taxi term C" in dictionary 705 are obtained from the text search results of S604 and S605. As a result, no text search result is obtained for "taxi term A" in dictionary 703, text search result 511 is obtained for "taxi term B" in dictionary 704, and text search result 512 is obtained for "taxi term C" in dictionary 705, and the process proceeds to S803. In S803, the type of output condition 723 is a search result logical operation, so the process proceeds to S809.

[0060] In S809, the CPU 401 of the image processing server 202 determines whether the logical expression of the output condition 723 is satisfied, and if so, proceeds to S810, otherwise proceeds to S808. The output condition 723 is based on whether the logical expression "taxi term A | (taxi term B & taxi term C)" is true. When the number of text search results acquired in S802 is applied to this logical expression, it becomes "0 | (1 & 1)", which means that the logical expression is true, and proceeds to S810 as the condition is satisfied.

[0061] In S810, the CPU 401 of the image processing server 202 determines the output value since the output condition is satisfied, and proceeds to S808. In the case of the output condition 723, "taxi" is determined as the output value of "company name type (company classification type)". In S808 thereafter, since all output conditions have been processed, the process ends.

[0062] In this embodiment, for the sake of explanation, one output condition (output condition 723) is set for the item name of company name type (company classification type), but normally output conditions should be added in the same manner as the number of types of company name types to be distinguished.

[0063] Return to the description of the flowchart in FIG. 6. In S607, the CPU 401 of the image processing server 202 checks whether the telephone number output in S605 exists in the company name dictionary, and if it exists, identifies the company name from the search result and outputs the search result as the item name "company name". The company name dictionary is a database that links telephone numbers with company names, and it is also possible to identify the company name from the telephone number. However, it is difficult to cover the telephone numbers of all company names, and if there is an error in OCR recognition, the company name cannot be searched. For example, in the receipt 500, "03-1234-5678" is obtained as the telephone number, but if the company name cannot be obtained from the company name dictionary, the company name is left blank and the process proceeds to S608. Note that, although a method of searching for a company name based on a telephone number is used here, the method is not limited to this, and as described above, it is also possible to search for the company name itself from the character recognition result in addition to searching from the telephone number.

[0064] In S608, the CPU 401 of the image processing server 202 determines the final company name information based on the "company name" and "company name type (company classification type)" determined in S601 to S608. If there is an output value for "company name", that value is output; if there is no output value for "company name", and there is an output value for "company name type (company classification type)", the value of "company name type" is output; if there is neither, no output is made. Here, since there is no output for "company name" and "company name type" has been output, "taxi", which is the "company name type (company classification type)", is determined as the output of the final "company name information".

[0065] As a result of applying the processes in the flowcharts of Figures 6 and 8, in the example of receipt 500, three items and their values ​​are output: "Total amount" is text search result 516, "Telephone number" is text search result 517, and "Company name information" is "Taxi."

[0066] 9 is an example of a receipt from a gas station. A case where the processes of S601 to S608 are performed on this receipt 900 using the information extraction rule of FIG. 7 will be described. As a result of performing the text search of S603 and S604 on the document recognition result, the results obtained are that there is no result for "taxi term A" in dictionary 703, "vehicle number" in text search result 901 for "taxi term B" in dictionary 704, and there is no result for "taxi term C" in dictionary 705. When the number of text search results is applied to the logical expression of output condition 723, it becomes "0|(1&0)", which does not match the condition in S809, and therefore "company name type" is not output.

[0067] As described above, by applying this embodiment, it is possible to search for dictionaries and patterns from character strings in document recognition results, and output "taxi" as the company name type (company classification type) by performing logical operations on the results. Also, for receipts other than taxis that have similar terms, it is possible to not output the company name type without erroneous determination.

[0068] <Second embodiment> In the first embodiment, the company name type (company classification type) is determined from the appearance of terms in the document recognition results and the number of logical operation results thereof, but other conditions, for example, input image information such as the size of a receipt, may also be used.

[0069] FIG. 10(a) shows an example of an input document image, which is a railway receipt 1000. The receipt 1000 only has a logo 1001 as a clue to the name of the railway company and that it is a railway receipt. Normally, images decorated with logos or other ornaments are difficult to recognize using OCR, so it is difficult to directly identify the company name from the OCR character string. However, receipts issued by railway companies may be output in the same size as tickets, and the size of the receipt may be characteristic. Below, a method of using image size as an output condition type will be described.

[0070] Table 1010 in Fig. 10(b) shows the information extraction rules in this embodiment. Output conditions 721 to 723 are the same as the information extraction rules in Fig. 7(c). Output condition 1011 is a condition for identifying a company name type, and if the image size is 85 mm and the height is 58 mm, "railroad" is output. "Image size" is newly added to the type, and if the size information of the input image is the specified size, "railroad" is output.

[0071] 10C shows the image information acquired in S601 of the receipt 1000. The image information 1020 has a resolution of 300 dpi in both the width and height directions, and is 1000 pixels wide and 680 pixels high.

[0072] Fig. 11 is a flowchart for explaining the item value output process using the image processing size of this embodiment. The processes of steps (S801 to S810) indicated with the same numbers as those in the flowchart of Fig. 8 are the same as the corresponding steps in Fig. 8, so detailed explanations will be omitted.

[0073] In S1101, the CPU 401 of the image processing server 202 determines the type of output condition. If it is "search result position", the process proceeds to S804; if it is "search result logical operation", the process proceeds to S809; if it is "image size", the process proceeds to S1102.

[0074] In S1102, the CPU 401 of the image processing server 202 determines whether the size of the detected document image matches the conditions, and if it does match, proceeds to S1103, and if it does not match, proceeds to S808. Note that in this embodiment, the image to be processed is an image that has been cropped according to the outline of the document, but if the image to be processed is an uncropped image, the outline of the document may be detected to obtain the document size and determine whether it matches the conditions.

[0075] In S1103, the CPU 401 of the image processing server 202 determines an output value. When the receipt 1000 is processed in the flowchart of Fig. 11, in S1102, it is determined whether the size matches the output condition 1011. Since a width of 1000 pixels is 85 mm at a resolution of 300 dpi and a height of 680 pixels is 58 mm at a resolution of 300 dpi, which matches the image size condition, "railroad" is output as the company name type (company classification type).

[0076] In the example of output condition 1011, only image size is used as a condition, but this is not limited to this. A composite condition may be used that combines not only image size but also logical operation results from text search results from OCR character strings.

[0077] In addition to the image size, the characteristics of the document such as the color of the paper and the background pattern may also be used.

[0078] As described above, by applying this embodiment, even if the characteristics of the company name type cannot be obtained from the OCR character string, the company name type can be output by using image information such as image size.

[0079] <Third embodiment> In the first embodiment, if both the company name type (classification type of company) and the unique company name can be identified, the unique company name is output as information related to the final company name in S608. Normally, if there is a company name type and a unique company name, the unique company name is more detailed information, so in expense settlement purposes, the company name type is used as an aid when it is difficult to extract the company name. However, there are certain cases where it is preferable to output the company name type (classification type).

[0080] Receipt 1200 in FIG. 12 is an airline ticket receipt in this embodiment. Receipt 1200 is a receipt for an airline ticket paid at a convenience store, and the company name and telephone number listed are those of the convenience store. If a telephone number or company name dictionary is used to output the company name, the company name will be "CamonMart," the name of this convenience store. However, in expense settlement operations, outputting the type "air ticket" is more appropriate as support information for inputting the purpose of use than the specific company name of the convenience store.

[0081] FIG. 13 is an example of a rule in this embodiment. Table 1300 in FIG. 13(a) is a list of dictionaries having dictionaries 1301, 1302, 1303, and 1304. Dictionary 1301 is a dictionary of airline ticket terms having search strings of "AIRLINE" and "airline", which are clues that the company name type (purchase type) is an airline ticket. Dictionary 1302 is a dictionary of convenience store terms having search strings of "convenience store", which are clues that the receipt is issued at a convenience store. Dictionary 1303 is a dictionary of web terms having "display date", which are clues that the receipt is issued on the web. Dictionary 1304 is a dictionary of agency terms having "travel", "agency", and "agency", which are clues that the receipt is issued by a travel agency.

[0082] Table 1310 in FIG. 13(b) is a list of output conditions using these terms. Output condition 1311 has an item name of "company name type (purchase type)", a condition type of "search result logical operation", a condition of "airline ticket term & convenience store term", and outputs "airline ticket (convenience store)" as an output value. Output condition 1312 has an item name of "company name type (purchase type)", a condition type of "search result logical operation", a condition of "airline ticket term & web term", and outputs "airline ticket (online)" as an output value. Output condition 1313 has an item name of "company name type (purchase type)", a condition type of "search result logical operation", a condition of "airline ticket term & agency term", and outputs "airline ticket (agency)" as an output value. The item names of output conditions 1311, 1312, and 1313 are defined as company name type (purchase type) because, in addition to company name type information, purchaser information indicating where it was purchased, i.e., where payment was made, is additionally output. If, in addition to the terms indicating an airline ticket, there is a term indicating a convenience store, output condition 1311 outputs "air ticket (convenience store)", i.e., an airline ticket paid for at a convenience store. If, in addition to the terms indicating an airline ticket, there is a term indicating a web-issued ticket, output condition 1312 outputs "air ticket (online)", i.e., an airline ticket paid for online. If, in addition to the terms indicating an airline ticket, there is a term indicating an agency-issued ticket, output condition 1313 outputs "air ticket (agency)", i.e., an airline ticket paid for at an agency.

[0083] 6 and 8 based on the dictionary 1300 and output condition 1310. The receipt 1200 contains "airline" in the text search result 1201 searched in the dictionary 1301 and "convenience store" in the text search result 1202 searched in the dictionary 1302. This matches the condition of the output condition 1311, and an output value of "airline ticket (convenience store)" is obtained as the company name type (purchase type).

[0084] Table 1320 in FIG. 13(c) is a table that lists conditions for determining company name information to be output when the company name information is determined in S608. The company name information determination condition list 1320 has a column of a condition number, a column of conditions for whether the company name was acquired, a column of conditions for whether the company name type (purchase type) was acquired, a column of conditions for what the purchase type was, and a column of output values ​​to be used as the company name information output when the conditions are met. The table is judged in order from the top of the column, and the company name information output of the matching row is used. Company name information determination condition 1321 indicates that if there is a "company name", there is also a "company name type", and the "purchase type" is a convenience store, then the "company name type (purchase type)" is output as the company name information. Company name information determination condition 1322 indicates that if there is a "company name", there is also a "company name type", and the "purchase type" is an agency, then the "company name type (purchase type)" is output as the company name information. Company name information determination condition 1323 indicates that if there is a "company name" and the company name type (purchase type) is not acquired, then "company name" is used as the company name information. Company name information determination condition 1324 indicates that if there is no "company name" and there is a "company name type", then "company name type (purchase type)" is output. Company name information determination condition 1325 indicates that if neither "company name" nor "company name type" is acquired, then there is no company name information.

[0085] When company name information is determined based on this table, the receipt 1200 matches the company name information determination condition 1321, and the purchase type "air ticket" is output as the company name information, instead of the company name.

[0086] As described above, by applying this embodiment, not only is a unique company name always used when a company name is obtained, but under certain conditions, the company name type value is given priority, making it possible to output values ​​necessary for expense settlement operations.

[0087] <Other Examples> The present invention can also be realized by executing the following process, which involves supplying software (programs) that realize the functions of the above-described embodiments to a system or device via a network or a computer-readable storage medium, and having the computer (or CPU, MPU, etc.) of the system or device execute the programs.

Claims

1. a first identification means for identifying information about a company that has issued a document image based on one or more character strings recognized from the document image by a character recognition process for the document image and on predefined conditions; a second identification means for identifying a company name of a company that has issued the document image, the company name being included in the one or more character strings; An output means for outputting company name information; having The output means includes: When the company name of the company is identified by the second identification means, the identified company name is output as the information of the company name; When the second specifying means does not specify the company name of the company, and the first specifying means specifies information about the company, the information about the specified company is output instead of the company name as information about the company name.

13. An image processing device comprising:

2. The image processing device according to claim 1 , wherein the predefined condition includes a condition based on whether or not a predetermined character string is present in the one or more character strings.

3. The image processing apparatus according to claim 1 , wherein the predefined conditions include a condition based on a logical operation.

4. The image processing device according to claim 1 , wherein the predefined conditions include a condition based on an image size.

5. 2. The image processing device according to claim 1, wherein the first identification means identifies information about the company that issued the document image and information about the supplier based on the one or more character strings and another predefined condition.

6. The image processing device of claim 1, wherein the predefined conditions include conditions based on a database search.

7. A program for causing a computer to function as each of the means of the image processing apparatus according to any one of claims 1 to 6.

8. An image processing method, comprising: a first identification step of identifying information about a company that has issued the document image based on one or more character strings recognized from the document image by a character recognition process for the document image and on predefined conditions; a second identification step of identifying a company name of the company that issued the document image, the company name being included in the one or more character strings; an output step of outputting information on the company name; having In the output process, When the company name of the company is identified by the second identification means, the identified company name is output as the information of the company name; When the second specifying means does not specify the company name of the company, and the first specifying means specifies information about the company, the information about the specified company is output instead of the company name as information about the company name.

13. An image processing method comprising:

Images