Evidence information processing system, evidence information processing method, and evidence information processing program

Abstract

To provide a voucher information processing system, a voucher information processing method, and a voucher information processing program for obtaining information corresponding to user-specific usage items that have not been sufficiently learned in a learning model. [Solution] The supporting document information processing system 1 has a learning model 11 that has been machine-learned to learn general items and has been machine-learned to output information corresponding to the general items when supporting document data is input, a general item extraction unit 13 that inputs supporting document data of the supporting document to be extracted into the learning model and outputs information corresponding to the general items, and a language model 12 that outputs information corresponding to the item when an item, an extraction instruction for the item, and supporting document data are input, and an additional item extraction unit 14 that inputs additional items that are subject to extraction and have not been machine-learned by the learning model into the language model, extraction instructions for the additional items, and supporting document data of the supporting document to be extracted, and outputs information corresponding to the additional items.

Description

[Technical field]

[0001] The present invention relates to an evidence information processing system, an evidence information processing method, and an evidence information processing program. [Background technology]

[0002] A technology has been proposed for reading invoices using AI-OCR functions (Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] https: / / www.ricoh.co.jp / service / cloud-ocr / special / invoice Summary of the Invention [Problem to be solved by the invention]

[0004] A learning model is created that outputs information corresponding to the learned items when the supporting data is input by machine learning image data of supporting documents such as invoices and teacher data including a set of information corresponding to general items of supporting documents, and when supporting data is input into the learning model, the information corresponding to the items that have been machine-learned in advance can be output and estimated with high accuracy. For example, taking an invoice as an example, a learning model is constructed to output information corresponding to items such as "billing company name" and "billing amount" that are commonly used in invoice processing, so that information corresponding to these items can be output with high accuracy.

[0005] However, the learning model has not been fully trained on user-specific usage items, and cannot estimate them. For example, information corresponding to items such as "shipping name" and "shipper", which are specific to the transportation industry, cannot be obtained even if supporting data is input into the learning model.

[0006] Therefore, the present invention has been made in consideration of these points, and aims to provide a voucher information processing device, a voucher information processing method, and a voucher information processing program that can obtain information corresponding to user-specific usage items that have not been sufficiently learned in the learning model. [Means for solving the problem]

[0007] The information processing system of the first aspect of the present invention comprises a learning model in which general items have been machine-learned and which has been machine-learned to output information corresponding to the general items when supporting document data is input, and a language model which outputs information corresponding to the items when an item, an extraction instruction for the item, and supporting document data are input, and has a general item extraction unit which inputs supporting document data of the supporting document to be extracted into the learning model and outputs information corresponding to the general items, and an additional item extraction unit which inputs additional items that are to be extracted and have not been machine-learned by the learning model, extraction instructions for the additional items, and supporting document data of the supporting document to be extracted into the language model and outputs information corresponding to the additional items.

[0008] The learning model is a learning model that has been machine-learned to output information corresponding to general items when evidence image data is input, and the language model is a language model that outputs information corresponding to the items when an item, an instruction to extract the item, and evidence image data are input, and the general item extraction unit outputs information corresponding to general items when evidence image data of the evidence to be extracted is input into the learning model, and the additional item extraction unit outputs information corresponding to the additional items when an additional item that is to be extracted and has not been machine-learned in the learning model, an instruction to extract the additional item, and evidence image data to be extracted are input into the language model.

[0009] The learning model is a learning model that has been machine-learned to output information corresponding to general items when evidence image data is input, and the language model is a language model that outputs information corresponding to the items when an item, an instruction to extract the item, and evidence text data are input, and the general item extraction unit outputs information corresponding to general items when evidence image data of the evidence to be extracted is input into the learning model, and the additional item extraction unit outputs information corresponding to the additional items when an additional item that is to be extracted and has not been machine-learned in the learning model, an instruction to extract the additional item, and evidence text data to be extracted are input into the language model.

[0010] In the additional item extraction unit, the evidence text data to be input to the language model may be evidence text data obtained as a result of applying a character recognition technique to evidence image data of the evidence to be extracted.

[0011] The voucher may be a quotation, a purchase order, an invoice, a receipt, a delivery note, an inspection slip, a contract, or a bankbook.

[0012] The additional items to be extracted and input to the additional item extraction unit may be additional items designated by a user.

[0013] Specific examples of the additional items may be input to the additional item extraction unit.

[0014] The information processing method of the second aspect of the present invention includes a general item extraction step executed by a computer, in which general items have been machine-learned and the learning model has been machine-learned to output information corresponding to the general items when supporting data is input, and in which supporting data to be extracted is input into the learning model, and information corresponding to the general items is output; and an additional item extraction step, in which an additional item that has not been machine-learned in the learning model to be extracted, an extraction instruction for the additional item, and supporting data to be extracted are input into a language model that outputs information corresponding to the item when an item, an extraction instruction for the additional item, and supporting data are input, and information corresponding to the additional item is output.

[0015] In the third aspect of the evidential information processing program of the present invention, a computer is caused to execute a general item extraction step in which evidence image data or evidence text data to be extracted are input to the learning model, which has already learned general items by machine learning, and which has been trained to output information corresponding to the general items when evidence image data or evidence text data is input, and the computer executes an additional item extraction step in which an additional item that is subject to extraction and has not been machine-learned in the learning model, an extraction instruction for the additional item, and the evidence data to be extracted are input to a language model that outputs information corresponding to the item when an item, an extraction instruction for the additional item, and the evidential data are input, and the computer executes an additional item extraction step in which an additional item that is subject to extraction and has not been machine-learned in the learning model, an extraction instruction for the additional item, and the evidential data to be extracted are input to a language model that outputs information corresponding to the item when an item, an extraction instruction for the additional item, and the evidential data to be extracted, and the computer executes an additional item extraction step in which an additional item is ... has been trained by machine learning, and which has been trained by machine learning, and the extraction instruction for the additional item, and the evidential data to be extracted, and the computer executes an additional item extraction step. Effect of the Invention

[0016] According to the present invention, it is possible to provide a voucher information processing system, a voucher information processing method, and a voucher information processing program / voucher information processing device that can obtain information corresponding to user-specific usage items that have not been sufficiently learned in the learning model. [Brief description of the drawings]

[0017] [Figure 1] 1 is a diagram showing a configuration of an evidence information processing system according to an embodiment of the present invention. [Diagram 2] FIG. 1 is a diagram showing a configuration of a computer according to an embodiment of the present invention. [Diagram 3] FIG. 13 is a diagram showing an example of an invoice image. [Figure 4] FIG. 13 is a diagram showing an example of an order sheet image. [Diagram 5] FIG. 11 is a flowchart of evidential information processing in the first embodiment. [Figure 6] FIG. 11 is a flowchart of evidential information processing according to the second embodiment. [Figure 7] FIG. 11 is a diagram showing item-related information extracted according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, an embodiment of the present invention will be described. FIG. 1 is a diagram showing the configuration of a documentary evidence processing system 1 according to an embodiment of the present invention. The documentary evidence processing system 1 includes a learning model 11 that has been machine-learned to learn general items (sometimes called "existing items") and has been machine-learned to output information corresponding to the general items when documentary evidence data (documentary evidence image data and documentary evidence text data are collectively called "documentary evidence data") is input, and a language model 12 that outputs information corresponding to the items when an item, an extraction instruction for the item, and documentary evidence data are input. The system also includes a general item extraction unit 13 that inputs documentary evidence data 15 of a documentary evidence to be extracted to the learning model 11 and outputs information 18 corresponding to the general items, and an additional item extraction unit 14 that inputs additional items 16 that are to be extracted and have not been machine-learned by the learning model, an extraction instruction 17 for the additional items, and documentary evidence data 15 of a documentary evidence to be extracted to the language model and outputs information 19 corresponding to the additional items.

[0019] In this specification, "evidence" refers to documents related to the movement of money or changes in rights and obligations, such as contracts, approval documents, quotations, purchase orders, order forms, delivery notes, inspection slips, acceptance slips, invoices, bankbooks and other passbooks. In addition, in this specification, "accounting evidence" refers to evidence related to the payment of the price of goods or services, such as quotations, purchase orders, order forms, invoices, etc. The present invention is particularly effective when applied to accounting evidence, as it allows users to obtain the information they want with a high degree of accuracy.

[0020] In this specification, "general items" refer to typical items used by most users. For example, in the case of an invoice, these are items such as "billing company name" and "billing amount." "Additional items" refer to items that are used only by specific users and that have not been machine-learned in the learning model. For example, these are items such as "shipper" and "shipping name" in purchase orders used by transportation companies.

[0021] In this specification, "evidence image data" refers to image-formatted data of evidence. Evidence image data is expressed, for example, by the position coordinates of each pixel of the evidence image, the color of the pixel, the density, etc. of the pixel. "Evidence text data" refers to text data that represents the contents of the evidence. Evidence text data is typically expressed as a continuous line of text such as letters, numbers, symbols, etc., such as "AAA Co., Ltd. S1000224 Management Department July 1, 2024...". "Evidence data" refers to data related to evidence, and is a collective term for evidence image data and evidence text data. Evidence text data can be obtained by a person looking at the evidence image data and inputting it, or by applying character recognition technology such as OCR (Optical Character Recognition) to the evidence image data.

[0022] The learning model in this embodiment is constructed by storing a large amount of teacher data (correct answer data) that pairs supporting data and information corresponding to general items of the supporting data, and conducting machine learning. When supporting data is input into the learning model, information corresponding to the item is output.

[0023] The language model in this embodiment is a model of human language using the occurrence probability of words. When an instruction is given to the language model in language, an answer is output. In this embodiment, the instruction is, for example, "Extract information corresponding to item XX from the evidence data of the evidence to be extracted." The larger the model size (number of parameters) of the language model, the higher the answer accuracy, but when the number of parameters exceeds 10 million, the accuracy becomes practical. There are vision-LLM (Large language Models) that can process image data, and language models that can only process text data.

[0024] The function of the evidence information processing system 1 according to this embodiment is realized by a computer 2. FIG. 2 is a diagram showing the configuration of the computer. The input unit 21 is a component that inputs information to the storage unit, and includes, for example, a keyboard, a mouse, a digital camera, an Internet line, an intranet line, and the like. The storage unit 23 is a component that stores (records) information. Methods for inputting evidence data to the storage unit 23 by the input unit 21 include photographing the evidence with a digital camera, and uploading evidence data stored in a server from the server via an intranet line. The storage unit 23 stores a learning model 11 and a language model 12. The calculation unit 24 is a component that performs information processing. The calculation unit 24 outputs general item information by performing information processing based on the evidence data and learning model stored in the storage unit 23, and outputs additional item information based on the evidence data, additional items, and additional item extraction instructions stored in the storage unit 23. The output unit 22 is, for example, a display. The display displays information corresponding to the general items of the evidence to be extracted and information corresponding to the additional items output by the calculation unit 24.

[0025] Figure 3 is an example of a typical invoice image. Invoice image 301 contains the following items: "Document Title" (302), "Destination" (303), "Billing Source" (304), "Invoice Number" (305), "Billing Date" (306), "Billing Period" (307), "Item Name", "Quantity", "Unit Price", "Amount" and "Billed Amount" (308), and "Payment Account" (309). All of these items are typical items on an invoice and fall under the category of general items. Since the learning model has already learned about general items by machine learning, by inputting invoice image data 301 into the learning model, all of these items that the user needs can be obtained with high accuracy. FIG. 4 is an example of an order form image used by a transport company. 402 is a "destination" and is a general item, so if the order form image data 401 is input to the learning model, "AAA Transport Co., Ltd." can be extracted with high accuracy as information corresponding to the "destination". However, "shipper name" (403) and "ship name" (404) are items specific to order forms used by transport companies and are not machine-learned in the learning model, so even if the order image data 401 is input to the learning model, information corresponding to the "shipper name" and "ship name" cannot be extracted. In this embodiment, the order form image data 401, "shipper name" and "ship name" as additional items, and an instruction to extract additional items such as "extract the "shipper name" and "ship name" from the order image data 401" can be input to the language model to extract information corresponding to the additional items "shipper name" and "ship name".

[0026] By inputting specific examples such as "shipper name" and "ship name" into the learning model, it is possible to estimate the information corresponding to the additional items with even greater accuracy.

[0027] Fig. 7 shows the general item applicable information and additional item applicable information extracted from the purchase order image data 401. The extracted item applicable information can be displayed on a display as necessary, and can also be linked to a deadline management system to manage deadlines, display an alert when the port entry date is approaching, and search for required evidence by specifying an item or keyword.

[0028] As described above, according to this embodiment, even if there are additional items that are user-specific usage items that have not been sufficiently learned in the learning model, information corresponding to these items can be obtained. EXAMPLES

[0029] The evidence information processing method according to the first embodiment will be described below with reference to FIG.

[0030] First, the user inputs additional items, which are non-general items used specifically by the user, into the system (S50).

[0031] Next, the image data of the evidence to be extracted is obtained (S51). Examples of methods for obtaining the image data of evidence include taking a picture of the paper evidence with a digital camera, saving the image data of evidence attached to an e-mail in a specified folder, and downloading the image data of evidence saved in a server.

[0032] The acquired document image data is input to a learning model that has been trained by machine learning so that general items are output when the document image data is input (S52). Then, information corresponding to the general items is extracted and output (S54).

[0033] On the other hand, the document image data, the additional items, and the additional item extraction instructions are input to a language model capable of processing image information (S55). Then, information corresponding to the additional items is output from the language model (S56).

[0034] The output information corresponding to the general items and additional items is stored in a memory unit, and the information corresponding to the required items can be displayed on the display as necessary, or the required evidence can be found by searching using keywords.

[0035] In this embodiment, since the information corresponding to the additional items is extracted using a language model capable of processing image information, the information corresponding to the additional items can be extracted taking into account the layout of the items of the voucher, and the information corresponding to the additional items can be extracted with high accuracy. In particular, since the layout of items of accounting vouchers such as estimates, invoices, and receipts is standardized, according to this embodiment, the information corresponding to the additional items can be extracted with high accuracy. EXAMPLES

[0036] Hereinafter, the evidence information processing method according to the second embodiment will be described with reference to Fig. 6. Steps S60 to S64 are the same as those in the first embodiment, and therefore the description will be omitted.

[0037] This embodiment differs from embodiment 1 in steps S65 to S66. In this embodiment, the evidence image data is converted into text data by character recognition technology such as OCR (S65).

[0038] The converted evidence text data, additional items, and additional item extraction instructions are input to a language model that cannot process image data and can only process text data (S66).Then, the language model outputs the additional information corresponding to the information.

[0039] Generally, it takes a long time and a great deal of cost to build a language model capable of processing image information. According to this embodiment, since a language model capable of processing only text data can be used, the time and cost required to build a language model capable of processing images can be reduced.

[0040] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0041] 1. Evidence information processing system 11 Learning Model 12 Language Models 13 General item extraction part 14 Additional item extraction part 15 Evidence data 17 Extraction instructions for additional items 18 General information 19 Information applicable to additional items 21 Input section 22 Output section 23 Memory section 24 Arithmetic section 301 Invoice Image 401 Purchase Order Image

Claims

1. A learning model that has been machine-learned to learn general items and that outputs information corresponding to the general items when supporting document data is input; a language model that outputs information corresponding to an item when an item, an instruction to extract the item, and evidence data are input; a general item extraction unit that inputs evidence data of the evidence to be extracted into the learning model and outputs information corresponding to the general items; and an additional item extraction unit that inputs additional items to be extracted in the language model and not machine-learned in the learning model, an instruction to extract the additional items, and evidence data of the evidence to be extracted, and outputs information corresponding to the additional items. Evidence information processing system.

2. The learning model is a learning model that has been machine-trained to output information corresponding to general items when evidence image data is input, the language model is a language model that outputs information corresponding to an item when an item, an instruction to extract the item, and document image data are input; the general item extraction unit outputs information corresponding to general items when the document image data of the document to be extracted is input to the learning model; The additional item extraction unit outputs information corresponding to an additional item that is to be extracted in the language model and has not been machine-learned in the learning model, an instruction to extract the additional item, and evidence image data to be extracted when the additional item extraction unit receives the additional item. The evidence information processing system according to claim 1.

3. The learning model is a learning model that has been machine-trained to output information corresponding to general items when evidence image data is input, the language model is a language model that outputs information corresponding to an item when an item, an extraction instruction for the item, and evidence text data are input; the general item extraction unit outputs information corresponding to general items when the document image data of the document to be extracted is input to the learning model; the additional item extraction unit, when receiving an additional item to be extracted in the language model and not machine-learned in the learning model, an instruction to extract the additional item, and evidence text data to be extracted, outputs information corresponding to the additional item; The evidence information processing system according to claim 1.

4. In the additional item extraction unit, the evidence text data to be input to the language model is evidence text data obtained as a result of applying a character recognition technique to evidence image data of the evidence to be extracted. The evidence information processing system according to claim 3.

5. The voucher is a quotation, purchase order, invoice, receipt, delivery note, inspection slip, contract, or bankbook.

4. The evidence information processing system according to claim 1.

6. The additional items to be extracted and input to the additional item extraction unit are additional items designated by a user.

4. The evidence information processing system according to claim 1.

7. inputting specific examples of the additional items into the additional item extraction unit; 4. The evidence information processing system according to claim 1.

8. The computer executes A general item extraction step in which the general items have been machine-learned and the learning model has been machine-learned to output information corresponding to the general items when supporting data is input, and the supporting data to be extracted is input to the learning model to output information corresponding to the general items; and an additional item extraction step of inputting an additional item that has not been machine-trained in the learning model to be extracted into a language model that outputs information corresponding to the item when an item, an extraction instruction for the additional item, and supporting data to be extracted are input, and outputting information corresponding to the additional item. Method for processing evidentiary information.

9. On the computer, a general item extraction step of inputting the voucher image data or the voucher text data to be extracted into a learning model that has been machine-learned to learn general items and that outputs information corresponding to the general items when the voucher image data or the voucher text data is input; and an additional item extraction step of inputting an additional item that is to be extracted in a language model that outputs information corresponding to the item when an item, an extraction instruction for the additional item, and supporting data are input, the additional item that is not machine-learned in the learning model, the extraction instruction for the additional item, and supporting data to be extracted are input, and outputting information corresponding to the additional item; A document information processing program that executes the above.

Images