Information processing device, information processing method, and program

Abstract

An information processing device 1 comprises: a storage unit 12 that stores a first learning model having undergone machine training to output a feature amount which corresponds to a layout of document data obtained by digitizing a document and which indicates a feature corresponding to the content of said document data and a second learning model having undergone machine training to execute a natural language processing task; a first acquisition unit 131 that acquires document data to be input into the first learning model; a second acquisition unit 133 that acquires the content of a task to be executed by the second learning model; a first output unit 132 that inputs, into the first learning model, the document data acquired by the first acquisition unit 131 and that outputs a feature amount output by the first learning model; and a second output unit 134 that inputs, into the second learning model, the feature amount output by the first output unit 132 and the content of the task acquired by the second acquisition unit 133 and that outputs the task execution result output by the second learning model.

Description

Information processing device, information processing method, and program

[0001] The present invention relates to an information processing device, an information processing method, and a program.

[0002] A system that executes natural language processing tasks using a trained model generated by machine learning is known (see, for example, Patent Document 1).

[0003] JP 2023-91514 A

[0004] Natural language processing systems using existing technology have had problems in improving accuracy when it comes to tasks such as processing the content of documents that have a layout structure and where the location of information is meaningful, such as invoices.

[0005] The present invention has been made in view of these points, and aims to improve the accuracy of tasks that require recognizing the meaning of documents having layout structures.

[0006] An information processing device of a first aspect of the present invention has a memory unit that stores a first learning model that has been machine-trained to output features that correspond to the layout of document data, which is data obtained by digitizing a document, and that indicate features that correspond to the content of the document data, and a second learning model that has been machine-trained to perform a natural language processing task; a first acquisition unit that acquires document data to be input to the first learning model; a second acquisition unit that acquires the content of a task to be performed by the second learning model; a first output unit that inputs the document data acquired by the first acquisition unit into the first learning model and outputs the features output by the first learning model; and a second output unit that inputs the features output by the first output unit and the content of the task acquired by the second acquisition unit into the second learning model and outputs the execution result of the task output by the second learning model.

[0007] The second learning model may be (1) trained to be able to execute a task based on the content of the task indicating that it is a general-purpose task, which is a natural language processing task that does not depend on document data, and (2) trained to be able to execute a task based on the content of the task indicating that it is a document task, which is a natural language processing task that depends on document data, and the features.

[0008] The first output unit may (1) not output the features when the first acquisition unit does not acquire document data, and the second output unit inputs the content of the task acquired by the second acquisition unit into the second learning model and outputs the execution result of the task output by the second learning model, and (2) when the first acquisition unit acquires document data, output the features output by the first learning model, and the second output unit inputs the features output by the first output unit and the content of the task acquired by the second acquisition unit into the second learning model and outputs the execution result of the task output by the second learning model.

[0009] The first output unit may (1) output a predetermined feature indicating the absence of document data when the first acquisition unit does not acquire document data, and (2) output the feature output by the first learning model when the first acquisition unit acquires document data.

[0010] The system may further include a third output unit that (1) indicates that the content of the task is a document task, which is a natural language processing task that depends on document data, and (2) outputs predetermined information indicating that document data is not input if the first acquisition unit does not acquire document data.

[0011] The system may further include a third output unit that outputs predetermined information indicating that (1) the content of the task is a document task, which is a natural language processing task that depends on document data, and (2) the required document is not acquired if the document data acquired by the first acquisition unit differs from the document required in the content of the task.

[0012] An information processing method of a second aspect of the present invention includes a first acquisition step executed by a computer to acquire document data to be input into a first learning model that has been machine-learned to output features that indicate characteristics corresponding to the content of the document data, which is data that has been digitized from a document; a second acquisition step to acquire the content of a task to be executed by a second learning model that has been machine-learned to perform a natural language processing task; a first output step to input the document data acquired in the first acquisition step into the first learning model stored in a memory unit and output the features corresponding to the content of the document data output by the first learning model; and a second output step to input the features output in the first output step and the content of the task acquired in the second acquisition step into the second learning model stored in a memory unit and output the execution result of the task output by the second learning model.

[0013] In a third aspect of the program of the present invention, a computer is caused to execute the following steps: a first acquisition step of acquiring document data to be input into a first learning model that has been machine-learned to output features that indicate characteristics corresponding to the content of the document data, which is data obtained by digitizing a document; a second acquisition step of acquiring the content of a task to be executed by a second learning model that has been machine-learned to perform a natural language processing task; a first output step of inputting the document data acquired in the first acquisition step into the first learning model stored in a memory unit and causing the first learning model to output the features corresponding to the content of the document data; and a second output step of inputting the features output in the first output step and the content of the task acquired in the second acquisition step into the second learning model stored in a memory unit and causing the second learning model to output the execution results of the task output by the second learning model.

[0014] According to the present invention, it is possible to improve the accuracy of a task that requires recognizing the meaning of a document having a layout structure.

[0015] FIG. 1 is a diagram for explaining an overview of an information system S. FIG. 2 is a block diagram showing the configuration of an information processing device 1. FIG. 3 is a diagram showing an example of document data acquired by a first acquisition unit 131. FIG. 4 is a diagram showing an example of a reception screen displayed by a second acquisition unit 133. FIG. 5 is a flowchart showing the flow of processing in the information processing device 1.

[0016] [Overview of Information Processing System S] Fig. 1 is a diagram illustrating an overview of the information processing system S. The information processing system S is a system for executing processing for natural language using a machine learning model. The information processing system S includes an information processing device 1 and an information terminal 2.

[0017] The information processing device 1 is a device that causes an AI (Artificial Intelligence) to execute a natural language processing task and output the result of the execution. The information processing device 1 is, for example, a server.

[0018] The information terminal 2 is a terminal device used by a user of the information processing system S. The information terminal 2 is, for example, a personal computer, a tablet, or a smartphone.

[0019] The processing in the information processing system S will be described. The information processing device 1 acquires document data D1 from the information terminal 2. The document data D1 is electronic data of a document. The document data D1 may be image data of the document. The information processing device 1 acquires a task D2. The task D2 is text data indicating the content of a task to be executed by the information processing device 1.

[0020] The information processing device 1 inputs document data D1 into a first learning model M1 and outputs feature values ​​D3. The first learning model M1 is trained to output feature values ​​D3 that indicate the content of the document data D1, taking into account the arrangement of character strings within the document indicated by the input document data D1. As an example, the first learning model M1 extracts feature values ​​D3 that indicate the content of the document data D1 using the known Layout LM (Language Model) method.

[0021] 2 is a diagram illustrating an example of document data D1 acquired by the information processing device 1. When the document data D1 illustrated in FIG. 2 is input, the first learning model M1 outputs, based on the layout of the document, feature quantities D3 including: (1) "AAA Corporation, Administration Department" described in item I1 is the invoice destination; (2) "S1000224" described in item I2 is the invoice number; (3) "January 31, 2023" described in item I3 is the invoice issue date; (4) the information described in item I4 is the name, address, telephone number, and fax number of the invoice issuer; (5) the information described in item I5 indicates the transaction content covered by the invoice; and (6) the information described in item I6 is the transfer destination information.

[0022] The information processing device 1 inputs the feature quantity D3 and the task D2 into the second learning model M2. The second learning model M2 is a trained model that has been trained to be able to execute natural language processing tasks such as sentence generation, translation, information extraction, and question answering. The information processing device 1 displays the task execution results output by the second learning model M2 on the information terminal 2.

[0023] With this configuration, the information processing system S can improve the accuracy of tasks that require recognizing the meaning of documents having a layout structure.

[0024] 3 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 has a communication unit 11, a storage unit 12, and a control unit 13. The control unit 13 has a first acquisition unit 131, a first output unit 132, a second acquisition unit 133, a second output unit 134, and a third output unit 135.

[0025] The communication unit 11 is a communication interface for transmitting and receiving data to and from other devices via a network. The storage unit 12 is a storage medium including a read-only memory (ROM), a random access memory (RAM), a solid-state drive (SSD), a hard disk drive, etc. The storage unit 12 stores in advance a program to be executed by the control unit 13.

[0026] The storage unit 12 stores a first learning model M1 and a second learning model M2. The first learning model M1 is a learning model that has been machine-trained to output a feature D3 that indicates a feature corresponding to the content of document data D1, the feature D3 being a feature corresponding to the layout of document data D1, which is data obtained by digitizing a document.

[0027] The second learning model M2 is a learning model trained by machine learning to execute a natural language processing task. The second learning model M2 may be configured using known large language models (LLMs). When text data indicating the content of a task is input, the second learning model M2 outputs text corresponding to the task indicated by the acquired text data.

[0028] The control unit 13 is a processor such as a CPU (Central Processing Unit), etc. The control unit 13 executes a program stored in the storage unit 12, thereby functioning as a first acquisition unit 131, a first output unit 132, a second acquisition unit 133, a second output unit 134, and a third output unit 135.

[0029] The first acquisition unit 131 acquires document data D1 to be input to the first learning model M1. The first acquisition unit 131 acquires the document data D1 from the information terminal 2. As will be described later, the first acquisition unit 131 may acquire the document data D1 via a screen displayed on the information terminal 2 by the second acquisition unit 133.

[0030] The first output unit 132 inputs the document data D1 acquired by the first acquisition unit 131 to the first learning model M1 and outputs the feature D3 output by the first learning model M1. The first output unit 132 outputs the feature D3 output by the first trained model M1 to the second output unit 134.

[0031] The second acquisition unit 133 acquires the content of the task to be executed by the second learning model M2. The second acquisition unit 133 acquires text data indicating the content of the task described in natural language. As an example, the second acquisition unit 133 displays an input screen for accepting input data on the information terminal 2 and acquires the task D2 input on the reception screen. FIG. 4 is a diagram showing an example of the reception screen displayed by the second acquisition unit 133. The reception screen includes an object O1 for inputting document data D1, an object O2 for inputting the task D2 to be executed by the information processing device 1, an execution button O3 for instructing the execution of the input task, and an object O4 for displaying the execution result of the task.

[0032] Object O1 is a drop zone for inputting a file. The user selects a document file by dragging the document file to be input onto object O1. Object O2 is a text area for inputting text. Execute button O3 is a button for sending the document file selected in object O1 and the text input into object O2 to the information processing device 1. Object O4 is a display field for displaying information output by the second output unit 134.

[0033] The second output unit 134 inputs the feature D3 output by the first output unit 132 and the task content acquired by the second acquisition unit 133 into the second learning model M2 and outputs the task execution result output by the second learning model M2. As an example, the second output unit 134 breaks down the text data indicating the task acquired by the second acquisition unit 133 into words and generates an array in which vector data (features) representing each word are stored in the order in which the words appear. The second output unit 134 then inputs an array in which the feature D3 output by the first output unit 132 is stored first and the vector data included in the array generated from the text data is stored after that into the second learning model M2, and outputs the task execution result. The second output unit 134 displays the information output by the second learning model M2 on the information terminal 2. More specifically, the second output unit 134 displays the text output by the second learning model M2 in the object O4 on the display screen shown in FIG. 4.

[0034] This configuration of the information processing device 1 can improve the accuracy of tasks that require recognizing the meaning of documents with layout structures. Furthermore, because there is no need to perform transfer learning of the first trained model for each task that requires reading layout-dependent documents, multiple tasks can be performed, reducing the training burden and the resource usage of the model.

[0035] The information processing device 1 may be configured to execute a task that does not require input of document data D1. Hereinafter, a task that does not depend on document data D1 is referred to as a "general-purpose task," and a task that depends on document data D1 is referred to as a "document task." Examples of general-purpose tasks include summarizing an input document or machine translation. In the case of a general-purpose task, the task D2 acquired by the second acquisition unit 133 may include the text to be processed. Examples of document tasks include extracting information described in a document file or reading a graph.

[0036] The second learning model M2 is trained to be able to execute the task D2 based on the content of the task D2, which indicates that the task is a general-purpose task. The second learning model M2 is also trained to be able to execute the task based on the content of the task D2, which indicates that the task is a document task, and the feature D3.

[0037] If the first acquisition unit 131 does not acquire document data D1, the first output unit 132 does not output feature D3, and the second output unit 134 inputs the content of task D2 acquired by the second acquisition unit 133 into the second learning model M2 and outputs the execution result of task D2 output by the second learning model M2. That is, if an operation to press object O3 is performed without selecting a document file on the screen shown in FIG. 4 , the first acquisition unit 131 does not acquire document data D1, and the second output unit 134 inputs text indicating task D2 input into object O2 into the second learning model M2. In this case, the first output unit 132 does not output feature D3, and therefore the second output unit 134 does not input feature D3 output by the first output unit 132 into the second learning model M2.

[0038] As mentioned above, when the first acquisition unit 131 acquires document data D1, the first output unit 132 outputs the feature D3 output by the first learning model M1, and the second output unit 134 inputs the feature D3 output by the first output unit 132 and the content of task D2 acquired by the second acquisition unit 133 into the second learning model M2, and outputs the execution result of task D2 output by the second learning model M2.

[0039] By configuring the information processing device 1 in this manner, both general-purpose tasks and document tasks can be executed using a single learning model, making it possible to reduce the effort required for collecting learning data and for learning processing.

[0040] When the task to be executed is a general-purpose task, the information processing device 1 may be configured to input predetermined features into the second learning model.

[0041] When the first acquisition unit 131 does not acquire document data D1, the first output unit 132 outputs a predetermined feature. The predetermined feature is a feature including a value predetermined to be input to the second learning model M2 when document data D1 is not acquired. The predetermined feature is, for example, "0." On the screen shown in FIG. 4 , when document data D1 is not selected and the execute button O3 is pressed, the first acquisition unit 131 does not acquire document data D1, and the second acquisition unit 133 accepts task D2, the first output unit 132 outputs the predetermined feature. In this case, the second learning model M2 is trained to execute task D2, which indicates a general-purpose task, using task D2, which indicates a general-purpose task, and the predetermined feature as input. When the first acquisition unit 131 acquires document data D1, the first output unit 132 outputs feature D3 output by the first learning model M1 based on the document data.

[0042] By configuring the information processing device 1 in this way, both general-purpose tasks and document tasks can be performed using a single learning model.

[0043] The information processing device 1 may be configured to output an alert when execution of a task that is a document task is instructed and the input document data differs from the document data required to execute the task.

[0044] The third output unit 135 outputs predetermined information indicating that the document data will not be input when the following two conditions are met: When the conditions described below are met, the third output unit 135 causes the information terminal 2 to display a screen warning the user that the document data D1 will not be output.

[0045] The first condition is that the content of task D2 indicates that it is a document task. The third output unit 135 determines whether it is a document task or a general-purpose task. As an example, the storage unit 12 may store a deterministic learning model, which is a trained model that, when task D2 is input, outputs whether the input task D2 is a document task or a general-purpose task. In this case, the third output unit 135 inputs task D2 into the deterministic learning model to determine whether task D2 acquired by the second acquisition unit 133 is a document task or a general-purpose task.

[0046] The second condition is that the first acquisition unit 131 does not acquire the document data D1. For example, the first acquisition unit 131 does not acquire the document data D1 when the execute button O3 is operated without selecting the document data D1 on the screen shown in Fig. 4. In other words, when the third output unit 135 determines that the input task D2 is a document task and the first acquisition unit 131 does not acquire the document data D1, the third output unit 135 outputs predetermined information to the information terminal 2.

[0047] By configuring the information processing device 1 in this way, it is possible to alert the user when necessary information is missing.

[0048] The information processing device 1 may be configured to output an alert when the input document data does not match the content of the input task.

[0049] The third output unit 135 outputs predetermined information indicating that the required document cannot be acquired when the following two conditions are satisfied: When the conditions described below are satisfied, the third output unit 135 causes the information terminal 2 to display a screen prompting the user to confirm that the input document data D1 is different from the file required to execute the task.

[0050] The first condition is that the content of task D2 is a document task. The second condition is that the document data D1 acquired by the first acquisition unit 131 is different from the document required to execute task D2. As an example, the second condition is satisfied when the content of the task acquired by the second acquisition unit 133 is "extract the invoice amount" and the document data acquired by the first acquisition unit 131 is a delivery note. As an example, the storage unit 12 stores a trained model that receives the feature amount D3 output by the first output unit 132 and the task D2 acquired by the second acquisition unit 133 as input and is trained to output whether the document data D1 matches the document required to execute task D2. The third output unit 135 inputs the feature amount D3 output by the first output unit 132 and the task D2 acquired by the second acquisition unit 133 into the trained model and determines whether the document data D1 matches the document required to execute task D2 based on the output information.

[0051] By configuring the information processing device 1 in this way, it is possible to alert the user when the input document data does not match the task.

[0052] 5 is a flowchart showing the flow of processing in the information processing device 1. The processing shown in FIG.

[0053] The second acquisition unit 133 displays a reception screen (S01). The first acquisition unit 131 and the second acquisition unit 133 acquire information entered on the reception screen (S02). The second acquisition unit 133 acquires task D2 entered on the reception screen. Furthermore, if document file D2 is selected on the reception screen, the first acquisition unit 131 acquires the entered document file D1. The third output unit 135 determines whether the content of the acquired task D2 is a document task (S03).

[0054] If the acquired task D2 is not a document task (NO in S03), the second output unit 134 inputs the task D2 into the second learning model M2 and outputs the execution result (S04).Then, the information processing device 1 proceeds to S09.

[0055] If the acquired task D2 is a document task (YES in S03), the first acquisition unit 131 determines whether or not document data D1 has been acquired (S05).

[0056] If the document data D1 is not acquired (NO in S05), an alert is output to warn that the document data D1 has not been input (S06), and the information processing device 1 ends the process.

[0057] When the document data D1 is acquired (YES in S05), the first output unit 132 inputs the document data D1 to the first learning model M1, which outputs the feature D3 (S07).The first output unit 132 inputs the feature D3 and the task D2 to the second learning model M2, which outputs the execution result of the task (S08).

[0058] The second output unit 134 displays the execution result output by the second learning model M2 on the reception screen (S09). Then, the information processing device 1 ends the process.

[0059] The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by functionally or physically distributing or integrating in any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combination also have the effects of the original embodiments.

[0060] REFERENCE SIGNS LIST 1 Information processing device 2 Information terminal 11 Communication unit 12 Storage unit 13 Control unit 131 First acquisition unit 132 First output unit 133 Second acquisition unit 134 Second output unit 135 Third output unit

Claims

1. An information processing device having: a memory unit that stores a first learning model that has been machine-trained to output features that correspond to the layout of document data, which is data obtained by digitizing a document, and that indicates features corresponding to the content of the document data; and a second learning model that has been machine-trained to perform a natural language processing task; a first acquisition unit that acquires document data to be input to the first learning model; a second acquisition unit that acquires content of a task to be executed by the second learning model; a first output unit that inputs the document data acquired by the first acquisition unit into the first learning model and outputs the features output by the first learning model; and a second output unit that inputs the features output by the first output unit and the content of the task acquired by the second acquisition unit into the second learning model and outputs the execution result of the task output by the second learning model.

2. The information processing device of claim 1, wherein the second learning model is (1) trained to be able to execute a task based on the content of the task indicating that it is a general-purpose task, which is a natural language processing task independent of document data, and (2) trained to be able to execute the task based on the content of the task indicating that it is a document task, which is a natural language processing task dependent on document data, and the features.

3. The information processing device of claim 1 or 2, wherein the first output unit: (1) does not output the feature when the first acquisition unit does not acquire document data; the second output unit inputs the content of the task acquired by the second acquisition unit to the second learning model and outputs the execution result of the task output by the second learning model; (2) outputs the feature output by the first learning model when the first acquisition unit acquires document data; and the second output unit inputs the feature output by the first output unit and the content of the task acquired by the second acquisition unit to the second learning model and outputs the execution result of the task output by the second learning model.

4. The information processing device of claim 1 or 2, wherein the first output unit: (1) outputs a predetermined feature indicating the absence of document data when the first acquisition unit does not acquire document data; and (2) outputs the feature output by the first learning model when the first acquisition unit acquires document data.

5. An information processing device as described in claim 1 or 2, further comprising a third output unit that (1) indicates that the content of the task is a document task, which is a natural language processing task that depends on document data, and (2) outputs specified information indicating that document data is not input if the first acquisition unit does not acquire document data.

6. An information processing device as described in claim 1 or 2, further comprising a third output unit that (1) indicates that the content of the task is a document task, which is a natural language processing task that depends on document data, and (2) outputs specified information indicating that the required document has not been acquired when the document data acquired by the first acquisition unit is different from the document required in the content of the task.

7. An information processing method executed by a computer, comprising: a first acquisition step of acquiring document data to be input into a first learning model trained by machine learning to output features indicating features corresponding to the content of document data, which is data obtained by digitizing a document; a second acquisition step of acquiring content of a task to be executed by a second learning model trained by machine learning to execute a natural language processing task; a first output step of inputting the document data acquired in the first acquisition step into the first learning model stored in a memory unit, and causing the first learning model to output the features corresponding to the content of the document data; and a second output step of inputting the features output in the first output step and the content of the task acquired in the second acquisition step into the second learning model stored in a memory unit, and causing the second learning model to output the execution result of the task output by the second learning model.

8. A program for causing a computer to execute the following steps: a first acquisition step of acquiring document data to be input into a first learning model that has been machine-learned to output features indicating features corresponding to the content of the document data, which is data obtained by digitizing a document; a second acquisition step of acquiring content of a task to be executed by a second learning model that has been machine-learned to execute a natural language processing task; a first output step of inputting the document data acquired in the first acquisition step into the first learning model stored in a memory unit, and causing the first learning model to output the features corresponding to the content of the document data; and a second output step of inputting the features output in the first output step and the content of the task acquired in the second acquisition step into the second learning model stored in a memory unit, and causing the second learning model to output the execution result of the task output by the second learning model.

Images