Information processing systems, information processing methods, programs

Abstract

To provide an information processing system, information processing device, information processing method, and program that efficiently utilize AI. [Solution] The information processing device comprises: a first processing means S302 that performs a correspondence process between items and values ​​contained in a document to be analyzed; a determination means 304 that determines items to be performed by a second processing means based on the results of the correspondence process by the first processing means; an instruction output means S306 that instructs the second processing means to perform a correspondence process for the items determined by the determination means; and an acquisition means S310 that acquires the results of the correspondence process performed by the second processing means S308 based on the instructions output by the instruction output means.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] In recent years, so-called AI (trained models) has been utilized in various fields, and AI is also used in the field of OCR. Patent Document 1 discloses a technique for performing character recognition using AI.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Disclosure of the Invention

Problems to be Solved by the Invention

[0004] In the use of AI and generative AI, there are many cases where it is a pay-per-use system according to the number of tokens used, and it is required to suppress unnecessary tokens as much as possible from the perspective of usage costs.

[0005] Therefore, an object of the present invention is to provide a mechanism for efficiently using AI.

Means for Solving the Problems

[0006] A first processing means for performing an association process between items and values included in a document to be analyzed, Determination means for determining items for which an association process by a second processing means is to be performed based on the result of the association process by the first processing means, Instruction output means for instructing the second processing means to execute an association process for the items determined by the determination means, An acquisition means for acquiring the result of the correspondence processing performed by the second processing means based on the instruction output by the instruction output means, An information processing device characterized by comprising: [Effects of the Invention]

[0007] According to the present invention, it becomes possible to provide a mechanism for efficiently using AI. [Brief explanation of the drawing]

[0008] [Figure 1] A diagram showing an example of the configuration of an information processing system. [Figure 2] Diagram showing the hardware configuration of the information processing device. [Figure 3] A flowchart illustrating an example of the processing details of the present invention. [Figure 4] Flowchart showing details of the process in step S304 [Figure 5] Flowchart showing details of the process in step S402 [Figure 6] Flowchart showing details of the process in step S403 [Figure 7] Flowchart showing details of the process in step S305 [Figure 8] Flowchart showing details of the process in step S311 [Figure 9] A diagram showing an example of an invoice to be processed. [Figure 10] Figure showing a specific example of the analysis results for S302. [Figure 11] A diagram showing an example of an item / character type table. [Figure 12] A diagram showing a concrete example of an item list. [Figure 13] A diagram showing a specific example of the process in step S503. [Figure 14] A diagram showing an example of a list of target items. [Figure 15] Figure 6 is a diagram illustrating the process of the flowchart. [Figure 16] A diagram showing an example of a list of target items. [Figure 17] Figure showing an example of a base prompt [Figure 18] Figure showing an example of a prompt completed in the process of step S703 [Figure 19] Figure showing an example of the result of association by a generative AI [Figure 20] Figure showing an example of the result of association by a generative AI [Figure 21] Figure showing an example of the association result after the process of the flowchart in FIG. 8 is executed [Figure 22] Figure showing an example of the screen displayed in step S312

Mode for Carrying Out the Invention

[0009] FIG. 1 is a diagram showing a configuration example of the information processing system of the present invention

[0010] As shown in FIG. 1, an information processing apparatus 101, a scanner 102, and a server apparatus 103 are communicably connected

[0011] The information processing apparatus 101 performs association of items and values by analyzing the image data acquired by the scanner 102. Further, it has a function of instructing the generative AI to execute processing, accepting various operations from the user, and notifying (displaying) information to the user

[0012] The scanner 102 has a function of acquiring documents such as forms to be processed as images. The acquired image data is transmitted to the information processing apparatus 101

[0013] The server apparatus 103 is a so-called generative AI (large language model, LLM)

[0014] FIG. 2 is a block diagram showing an example of the hardware configuration of the information processing apparatus of the present invention

[0015] As shown in Figure 2, the information processing device is connected via a system bus 200 to a CPU (Central Processing Unit) 201, ROM (Read Only Memory) 202, RAM (Random Access Memory) 203, storage device 204, input controller 205, audio controller 206, video controller 207, memory controller 208, and communication I / F controller 209.

[0016] CPU201 provides comprehensive control over all devices and controllers connected to system bus 200.

[0017] ROM202 or external memory213 holds the BIOS (Basic Input / Output System) and OS (Operating System), which are control programs executed by the CPU201, as well as computer-readable and executable programs and various necessary data (including data tables) for realizing this information processing method.

[0018] RAM203 functions as the main memory, work area, etc., of the CPU201. The CPU201 loads the necessary programs, etc., from ROM202 or external memory 213 into RAM203, and then executes the loaded programs to perform various operations.

[0019] The input controller 205 controls input from input devices such as the keyboard 210 and pointing devices such as a mouse (not shown). If the input device is a touch panel, the user can give various instructions by pressing (touching with a finger, etc.) icons, cursors, or buttons displayed on the touch panel.

[0020] Furthermore, the touch panel may be a multi-touch screen or other touch panel capable of detecting the positions of multiple fingers touching it.

[0021] The video controller 207 controls the display to an external output device such as the display 212. The display may include the display of a notebook computer integrated with the main unit. The external output device is not limited to a display; for example, it may be a projector. Furthermore, for the aforementioned touch-enabled device, an input device is also provided.

[0022] The video controller 207 can control the video memory (VRAM) used for display control. It can utilize a portion of the RAM 203 as the video memory area, or it can provide a separate, dedicated video memory.

[0023] The memory controller 208 controls access to the external memory 213. The external memory can include an external storage device (hard disk), a flexible disk (FD), or a CompactFlash® memory connected to a PCMCIA card slot via an adapter, which stores boot programs, various applications, font data, user files, editing files, and other data.

[0024] The communication interface controller 209 connects to and communicates with external devices via a network and performs communication control processing over the network. For example, it can handle communication using TCP / IP, telephone lines such as ISDN, and mobile phone 4G and 5G lines.

[0025] Furthermore, the CPU 201 enables display on the display 212 by, for example, performing the process of expanding (rasterizing) outline fonts into the display information area in RAM 203. The CPU 201 also enables user input via a mouse cursor (not shown) on the display 212.

[0026] Next, an example of the processing content of the present invention will be explained using the flowchart in Figure 3. The processes shown in the flowchart in Figure 3 (except for S307 to S309) are processes in which the CPU 201 of the information processing device 101 reads and executes a predetermined control program. The processes in S307 to S309 are processes executed by the large-scale language model (generative AI).

[0027] In step S301, an image of the document to be processed is obtained. In this embodiment, an invoice (an example is shown in Figure 9) is used as an example, but the document to be processed is not limited to an invoice; it may also be a form such as a supplier's form or a purchase order, or any document that requires the correspondence between items and their contents.

[0028] The acquisition method may involve acquiring an image read by scanner 102, or an image captured by an imaging device (not shown).

[0029] In step S302, the image acquired in step S301 is analyzed. Specifically, OCR processing (character detection and recognition, table detection, etc.) is performed to associate items with values. Known technologies will be used for the OCR processing. Known technologies will also be used for the item-value association process, but this will be rule-based processing by the processing module (first processing means) of the information processing device 101, rather than processing using the generative AI (large-scale language model) described later.

[0030] In step S303, the analysis results from step S302 are obtained. The analysis results include information related to the items and values ​​obtained from the invoice to be processed. A specific example of the analysis results is shown in Figure 10. As shown in Figure 10, the strings read from the invoice are output in the form of items and values. For example, the item "Invoice No." is associated with the value "¥10,050,000".

[0031] Step S304 performs item restriction processing. This process determines which items the generating AI will use to create the item-value mapping. The details will be explained using the flowchart in Figure 4.

[0032] Figure 4 shows a flowchart detailing the process in step S304.

[0033] In step S401, the item-character type table (an example is shown in Figure 11) is obtained. The item-character type table is a table in which items are associated with the appropriate character types for the values ​​associated with those items. For example, in the example in Figure 11, it is shown that the value for the item "Invoice No." can be one of the character types "0123456789-". In other words, it is appropriate that the value for the item "Invoice No." be represented by a "number".

[0034] Step S402 performs the target item selection process 1. The details of the process in step S402 will be explained using the flowchart in Figure 5.

[0035] Figure 5 shows a flowchart detailing the process in step S402.

[0036] In step S501, an item list is created. Specifically, the information related to "items" from the analysis results obtained in step S303 is listed (extracted) to create the list. A concrete example is shown in Figure 12. As shown in Figure 12, only items such as "Invoice No.," "Invoice Date," and "Issuer" are extracted.

[0037] Then, for each item in the item / character type table obtained in step S401, the processing in steps S503 and S504 is performed (step S502).

[0038] In step S503, it is determined whether the first item in the item / character type table is included in the item list created in S501. If it is included (S503: YES), that is, if the item in the item / character type table to be processed is obtained from the invoice to be processed, the process moves on to processing the next item.

[0039] On the other hand, if it is not included (S503:NO), that is, if the item in the item / character type table to be processed is not obtained from the invoice to be processed, then the item (the item registered in the item / character type table) is added to the target item list.

[0040] A concrete example of the process will be explained using Figure 13. Figure 13a is the item list created in S501, and Figure 13b is the items registered in the Item / Character Type Table. The first item registered in the Item / Character Type Table, "Invoice No.," also exists in the item list created in S501. Similarly, "Invoice Date" also exists. The third item, "Responsible Person," does not exist in the item list created in S501, so "Responsible Person" is added to the target item list.

[0041] By performing the above process for all items registered in the Item / Character Type table, it becomes possible to list the items registered in the Item / Character Type table that have not been retrieved from the invoice being processed as the target item list.

[0042] An example of a list of target items created in this way is shown in Figure 14. In the example shown in Figure 14, only "Responsible Person" is registered in the Item / Character Type table but is not retrieved from the invoice.

[0043] Once processing is complete for all items registered in the item / character type table, the flowchart in Figure 5 terminates, and the process moves to step S403.

[0044] The details of the process in step S403 will be explained using the flowchart in Figure 6.

[0045] Figure 6 shows a flowchart detailing the process in step S403.

[0046] In the flowchart of Figure 6, the processes S602 to S605 are executed on the analysis results obtained in step S303 (one set of information, which associates the items and values ​​obtained in S303, is treated as one record, and the processes S602 to S605 are executed on all obtained records) (S601).

[0047] In step S602, the string value of the record to be processed is obtained. Specifically, for the first record, "¥10,050,000" is obtained as the value corresponding to the "Invoice No." field (see Figures 10 and 15).

[0048] In step S603, the character type corresponding to the field in the record to be processed is obtained from the field / character type table. Specifically, "0123456789-" is obtained as the character type corresponding to the field "Invoice No." (See Figures 11 and 15).

[0049] Step S604 determines whether all characters in the value obtained in step S602 are included in the character type obtained in step S603. If all characters are included (S604: YES), processing proceeds to the next record. If there are any characters that are not included in the character type obtained in step S603, processing proceeds to step S605. Looking at the example of the first record, "¥" and "," (comma) are not registered as character types related to the "Invoice No." item in the Item / Character Type table, so it is determined to be S604: NO (see Figure 15).

[0050] In step S605, the fields related to the record to be processed are added to the target field list.

[0051] By performing the above process for all records obtained in step S303, it becomes possible to list the items that were incorrectly (mistakenly) mapped to values ​​in the process of step S303 as a list of target items.

[0052] Figure 16 shows an example of the list of target items after processing related to the flowchart in Figure 6. As shown in Figure 16, in addition to the "person in charge" registered in the flowchart process in Figure 5, "invoice number" and "total amount" have been added.

[0053] Once the process shown in Figure 6 is complete, the process moves to step S305 in Figure 3. The details of the process in step S305 will be explained using the flowchart in Figure 7.

[0054] Figure 7 is a flowchart detailing the process in step S305.

[0055] In step S701, the list of target items (Figure 16) created in the process of step S304 is obtained.

[0056] In step S702, the base prompt is obtained. The base prompt is a template for instructions (prompts) to the generating AI, and as shown in Figure 17, for example, it contains common instructions for the generating AI. In the example in Figure 17, it consists of three parts: "Description," "Target Item," and "Rules." The "Description" contains the processing content to be executed by the generating AI ("Extract key-value pairs from the text input read from the invoice sent by the business partner"). The "Rules" contain rules for when the generating AI associates items with values, and rules regarding the format of the response. You can freely set any desired rules, but in this embodiment, for the processing in step S311, the condition "If the item is not found, set the value to an empty string" is included.

[0057] The "Target Items" field is blank in the template (base prompt) shown in Figure 17, as this is where you enter the items from which the generating AI will extract values ​​to complete the prompt.

[0058] In step S703, the prompt is completed by adding the items related to the target item list obtained in step S701 to the "Target Item" field in the base prompt.

[0059] Figure 18 shows an example of a prompt completed in step S703. As shown in Figure 18, the base prompt (Figure 17) has three items entered in the target item column: "Responsible Person," "Invoice No.," and "Total Amount."

[0060] Once the prompt is complete, the process moves to step S306.

[0061] In step S306, the prompt generated in step S305 is sent to the large-scale language model (generating AI) (second processing means) along with the image of the invoice acquired in S301. (Instruction output means)

[0062] The generating AI then receives the prompt and image sent in S306 (S307), and based on the content of the prompt, it associates values ​​with the items included in the invoice (S308). Finally, it sends the result of the association to the information processing device 101 (S309).

[0063] In step S309, the results of the processing by the generating AI are received. An example of the mapping results by the generating AI is shown in Figure 19. As shown in Figure 19, values ​​have been mapped for the target items "Invoice No." and "Total Amount". The target item "Responsible Person" is an empty string, which means that "the item was not found" as described in the rules above.

[0064] Step S311 involves post-processing. The details of the process in step S311 will be explained using the flowchart in Figure 8.

[0065] Figure 8 is a flowchart detailing the process in step S311.

[0066] In step S801, the processing result from the generating AI received in step S309 is obtained.

[0067] In step S802, the results obtained in step S801 are used to retrieve the mapping results. Specifically, the value of "result" in the example result in Figure 19 (Invoice No: 20190501-01, Total Amount: ¥10,854,000, Person in Charge: ) is retrieved (see Figure 20). Then, for each record of items and values ​​obtained in S802, the processes in steps S804 and S805 are executed (S803).

[0068] Step S804 determines whether the value of the record to be processed is an empty string. If it is an empty string (S804: YES), the process proceeds to step S805. If it is not an empty string (S804: NO), the process proceeds to the next record.

[0069] Step S805 deletes records with empty string values ​​from the mapping results.

[0070] To illustrate with the example in Figure 19, for "Invoice No." and "Total Amount," the corresponding values ​​are not empty strings, so the S805 process is not executed. However, for "Responsible Person," the corresponding value is an empty string, so the S805 process is executed, and the record with the field "Responsible Person" is deleted from the mapping results.

[0071] Figure 21 shows an example of the mapping result after the flowchart in Figure 8 has been processed. Compared to the example in Figure 20, the record related to "Responsible Person" has been deleted, while "Invoice No." and "Total Amount" remain.

[0072] Once the processing shown in the flowchart in Figure 8 is complete, the process moves to step S312.

[0073] Step S312 displays the result of integrating the results of processing S303 and S304-S311. Specifically, for items that are present in S311 but not in S303, the results from S311 are added to the results of S303. For items that are present in both S303 and S311, the results from S311 overwrite the results from S303.

[0074] This makes it possible to use the generation AI to supplement items that could not be obtained from the invoice in step S303 (items that were missed). In addition, it becomes possible to reflect (correct) items that were incorrectly (mistakenly) mapped to values ​​in the processing of step S303 using the results of the mapping performed by the generation AI.

[0075] Figure 22 shows an example of the screen displayed in step S312.

[0076] As shown in Figure 22, the image of the document to be analyzed acquired in S301 is displayed, along with the results of associating the items and values ​​obtained by OCR.

[0077] As mentioned above, the results of the item-value mapping show that for "Invoice No." and "Total Amount," the values ​​displayed are the results of processing by the generation AI. Furthermore, the values ​​related to "Invoice No." and "Total Amount" are highlighted in bold and underlined to make it recognizable that they are the result of processing by the generation AI (and not the result of processing by the processing module of the information processing device 101). Note that while Figure 22 uses bold and underlined text for identification, the identification method is not limited to this. Any method that allows for the distinction between the results of processing by the generation AI and the results of processing by the processing module of the information processing device 101 is acceptable, such as adding an icon to indicate that it is processing by the generation AI (or processing by the processing module of the information processing device), changing the font size, changing the font, italicizing the text, or changing the text color.

[0078] Alternatively, the system may display both the results of processing by the generating AI and the results of processing by the processing module of the information processing device 101, allowing the user to choose which result to use.

[0079] As explained above, in this invention, the information processing device 101 first associates items with values, and then processes items that could not be properly associated or that were not detected using a generation AI, thereby minimizing the amount of data (number of tokens) exchanged with the generation AI. As a result, it becomes possible to reduce costs in systems where costs are incurred according to the amount of data exchanged, such as pay-per-use billing.

[0080] Although embodiments have been described above, the present invention can take the form of, for example, a system, apparatus, method, program, or recording medium. Specifically, it may be applied to a system consisting of multiple devices, or to an apparatus consisting of a single device.

[0081] Furthermore, the program in this invention is a program that a computer can execute using the processing methods shown in the flowcharts in Figures 3 to 8, and the storage medium of this invention stores a program that a computer can execute using the processing methods in Figures 3 to 8. Note that the program in this invention may also be a separate program for each processing method of each device in Figures 3 to 8.

[0082] As described above, it goes without saying that the object of the present invention can also be achieved by supplying a recording medium containing a program that realizes the functions of the embodiments described above to a system or device, and by having the computer (or CPU or MPU) of that system or device read and execute the program stored on the recording medium.

[0083] In this case, the program read from the recording medium itself realizes the novel function of the present invention, and the recording medium on which that program is recorded constitutes the present invention.

[0084] For recording media used to supply programs, examples include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, silicon disks, and the like.

[0085] Furthermore, it goes without saying that the functions of the aforementioned embodiments are realized not only by the computer executing the program it has read, but also by the operating system (OS) running on the computer performing some or all of the actual processing based on the instructions of that program, thereby realizing the functions of the aforementioned embodiments.

[0086] Furthermore, it goes without saying that this also includes cases where, after a program read from a recording medium is written to the memory of a function expansion board inserted into a computer or a function expansion unit connected to a computer, the CPU or other components of the function expansion board or function expansion unit perform some or all of the actual processing based on the instructions of the program code, and the functions of the aforementioned embodiments are realized through that processing.

[0087] Furthermore, the present invention may be applied to a system consisting of multiple devices or to a device consisting of a single device. It goes without saying that the present invention can also be applied when the results are achieved by supplying a program to a system or device. In this case, by reading a recording medium containing a program for achieving the present invention into the system or device, the system or device can enjoy the effects of the present invention.

[0088] Furthermore, by downloading and reading the program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or device can enjoy the effects of the present invention. It should be noted that configurations combining the above-described embodiments and their variations are all included in the present invention. [Explanation of symbols]

[0089] 101 Information Processing Device 102 Scanners 103 Server equipment

Claims

1. A first processing means that performs the process of matching items and values ​​contained in the document to be analyzed, A determination means that determines the items to be subjected to the correspondence processing by the second processing means based on the results of the correspondence processing by the first processing means, An instruction output means that instructs the second processing means to perform an association process for the items determined by the determination means, An acquisition means for acquiring the result of the correspondence processing performed by the second processing means based on the instruction output by the instruction output means, An information processing device characterized by comprising:

2. The information processing apparatus according to claim 1, characterized in that the second processing means is a means provided by an external device.

3. The information processing apparatus according to claim 2, characterized in that the second processing means is a generated AI.

4. It comprises a storage means that stores an item and the character type related to the value of that item, The information processing apparatus according to claim 1, characterized in that the determination means determines an item to be subjected to association processing by the second processing means if the character type of the value associated with the item by the first processing means is not the character type of the value associated with the item stored in the storage means.

5. The system includes a storage means for storing items to be extracted from the aforementioned documents, The information processing apparatus according to claim 1, characterized in that the determination means determines, among the items stored in the storage means, items for which the first processing means has not performed a correspondence with a value, as items for which the second processing means will perform the correspondence.

6. The information processing apparatus according to claim 1, characterized in that the instruction output means outputs a prompt including an image relating to the document to be analyzed and an item determined by the determination means.

7. The information processing apparatus according to claim 1, further comprising a display control means for controlling the display of the results of the mapping process acquired by the acquisition means.

8. The information processing apparatus according to claim 7, characterized in that the display control means controls the display of the result of the correspondence processing acquired by the acquisition means and the result of the correspondence processing by the first processing means.

9. The first processing means of the information processing device performs a first processing step of matching items and values ​​contained in the document to be analyzed, The determination means of the information processing device includes a determination step in which it determines items to be subjected to correspondence processing by the second processing means based on the results of the correspondence processing by the first processing step, The instruction output means of the information processing device provides an instruction output step that instructs the second processing means to perform an association process with respect to the items determined by the determination step, The acquisition means of the information processing device includes an acquisition step that acquires the result of the correspondence processing performed by the second processing step based on the instruction output by the instruction output step, An information processing method characterized by comprising:

10. A program for causing a computer to function as one of the means described in any one of claims 1 to 8.

Images