Information processing device, information processing method, and program

Abstract

The present invention provides an information processing device that performs high-precision reading of evidence data. [Solution] The information processing device includes a first acquisition unit that acquires document data relating to a document, a text information generation unit that generates text information including text data relating to text contained in the document data and coordinate data relating to the coordinates of the text, a prompt generation unit that generates prompts to instruct a generation model to extract payment information necessary for processing payment data based on the document data and to extract journal entry information necessary for processing journal entries, a second acquisition unit that inputs the prompts to the generation model and acquires payment information and journal entry information, and a display control unit that controls an identification unit and a display unit that identify the position of text data in the document data corresponding to the payment information and journal entry information, and displays the document data, payment information and journal entry information, highlighting the text data in the document data corresponding to the payment information and journal entry information based on the position identified by the identification unit.

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Conventionally, an information processing system for performing payment processing using a communication network such as the Internet has been known. When a payee such as a buyer performs payment processing based on an invoice issued by a creditor such as a supplier using such an information processing system, it is necessary to manually input the payee and the payment amount described in the invoice. Therefore, as the number of invoices increases, the labor and burden of the payer's input work and input information check become heavier.

[0003] In recent years, an optical character recognition technology (OCR) for recognizing a document described in a paper output form such as a form using an optical reading device such as a camera or a scanner has been known. Also, a technology for classifying form image data, which is a form image, by image analysis has been known. By using these technologies, the labor and burden of the payer's input work and input information check can be reduced. However, in order to reduce the labor and burden of the payer's input work and input information check using these technologies, it is necessary to accurately read the invoice data.

[0004] Moreover, not only when reading invoice data, but also when reading voucher data such as receipt data and receipt data, accurate reading is required.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

[0006] The present invention aims to provide an information processing device, an information processing method, and a program capable of reading document data with high accuracy. [Means for solving the problem]

[0007] The information processing device according to the present invention includes a first acquisition unit that acquires evidence data relating to evidence, A text information generation unit generates text information that includes text data relating to the text contained in the evidence data and coordinate data relating to the coordinates of the text, based on the aforementioned evidence data. A prompt generation unit generates prompts to instruct the generation model to extract payment information necessary for processing payment based on the aforementioned text data and to extract journal entry information necessary for processing journal entries based on the aforementioned proof data, A second acquisition unit inputs the aforementioned prompt to the generation model and acquires the payment information and the journal entry information from the generation model. An identification unit that identifies the location of the text data in the document data corresponding to the payment information and the journal entry information based on the text information, payment information and journal entry information, A display control unit that controls a display unit to display the document data, payment information, and journal entry information, the display control unit that highlights the text data in the document data corresponding to the payment information and journal entry information based on a position identified by the identification unit, It is equipped with.

[0008] The information processing method according to the present invention includes the step of a first acquisition unit acquiring evidence data relating to evidence, The text information generation unit generates text information based on the document data, including text data relating to the text contained in the document data and coordinate data relating to the coordinates of the text. The prompt generation unit generates prompts to instruct the generation model to extract payment information necessary for processing payment based on the document data and to extract journal entry information necessary for processing journal entries based on the document data, based on the text data. The second acquisition unit inputs the prompt to the generation model and acquires the payment information and the journal entry information, The identifying unit identifies the location of the text data in the document data corresponding to the payment information and the journal entry information based on the text information, the payment information and the journal entry information, The display control unit controls the display unit to display the document data, payment information, and journal entry information, and the step of highlighting the text data in the document data corresponding to the payment information and journal entry information based on a specified position. It is equipped with.

[0009] The program according to the present invention is configured to cause a computer to execute the information processing method described above. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide an information processing device, an information processing method, and a program that can read evidence data with high accuracy. [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows a schematic configuration of an information processing system according to one embodiment. [Figure 2] This block diagram shows an example of the configuration of an information processing device according to one embodiment. [Figure 3] This block diagram shows an example of the configuration of a server device according to one embodiment. [Figure 4] This is a flowchart illustrating the display process according to one embodiment. [Figure 5] This figure shows an example of evidence data according to one embodiment. [Figure 6] It is a diagram showing an example of a first prompt template according to an embodiment. [Figure 7] It is a diagram showing an example of a second prompt template according to an embodiment. [Figure 8] It is a diagram showing an example of a display screen for displaying voucher data, payment information, and journal information according to an embodiment.

Embodiment for Carrying Out the Invention

[0012] Hereinafter, an information processing apparatus, an information processing method, and a program according to an embodiment will be described with reference to the drawings. In the following description, components having substantially the same functions and configurations will be denoted by the same reference numerals, and duplicate descriptions will be made only when necessary.

[0013] FIG. 1 is a diagram showing a schematic configuration of an information processing system according to an embodiment. The information processing system 1 is a system for reading voucher data and performing payment processing based on the voucher data. As shown in FIG. 1, the information processing system 1 includes an information processing apparatus 10, a server apparatus 30, and a terminal apparatus 50. The information processing apparatus 10, the server apparatus 30, and the terminal apparatus 50 are communicably connected to each other via a network NW.

[0014] The information processing apparatus 10, which will be described in detail later, is an apparatus that generates text information including text data and coordinate data based on voucher data, generates a prompt from the text data, acquires payment information and journal information, and displays voucher data, payment information, and journal information. Note that the information processing apparatus 10 is not limited to a form composed of a single apparatus, and may be composed of a plurality of apparatuses.

[0015] The server device 30 is, for example, a cloud server on a network. In the present embodiment, the server device 30 is an LLM server having a large language model (LLM) which is a generative model, and uses the large language model to generate payment information and journal information. Note that the large language model can also be rephrased as a generative model. Further, the server device 30 is not limited to a cloud server, and may be a server installed within a facility. Further, the server device 30 is not limited to a form composed of a single device, and may be composed of a plurality of devices.

[0016] The terminal device 50 is an information processing terminal used by a user, and is, for example, a desktop personal computer, a notebook personal computer, a tablet terminal, a smartphone, or the like. The user logs in to the information processing device 10 from the terminal device 50, and performs operations such as uploading credential data and confirming the content of the uploaded credential data. In the example shown in FIG. 1, one terminal device 50 is connected to the network NW, but the number of terminal devices 50 connected to the network NW is not limited to one. That is, the number of terminal devices 50 connected to the network NW is arbitrary, and two or more terminal devices 50 may be connected to the network NW.

[0017] The network NW means the entire information communication network using telecommunication technology. The network NW includes, for example, a wireless / wired LAN (Local Area Network), the Internet network, as well as a telecommunication circuit network, an optical fiber communication network, a cable communication network, and a satellite communication network.

[0018] <L Note that the information processing system 1 may have a configuration other than those described above. For example, it may include accounting software / cloud with payment-processed credential data. In this case, the information processing device 10 and the accounting software / cloud may be API (Application Programming Interface) -linked.

[0019] Figure 2 is a block diagram showing an example of the configuration of an information processing device according to an embodiment. As shown in Figure 2, the information processing device 10 is configured to include a communication unit 11, a storage unit 13, and a control unit 15. Multiple elements for configuring this information processing device 10 may be realized by the cooperation of multiple servers that can communicate with each other.

[0020] The communication unit 11 implements various communication protocols according to the configuration of the network NW. The communication unit 11 enables communication with other devices via the network NW according to the various communication protocols. In this embodiment, the information processing device 10 communicates with the server device 30 and the terminal device 50 via the communication unit 11.

[0021] The memory unit 13 is a non-transient memory device that stores various types of information and can be implemented using semiconductor memory elements such as RAM (Random Access Memory) or flash memory, a hard disk, or an optical disc. For example, the memory unit 13 receives and stores various types of data acquired by the control unit 15. The memory unit 13 also stores programs corresponding to various functions executed by circuits included in the information processing device 10. The memory unit 13 may also be implemented using a group of servers (cloud) connected to the information processing device 10 via a network NW.

[0022] The control unit 15 is a control circuit that performs overall control of the information processing device 10, and is also an arithmetic circuit that performs various calculations. In this embodiment, the control unit 15 comprises a first acquisition unit 151, a text information generation unit 152, a prompt generation unit 153, a second acquisition unit 154, a specification unit 155, and a display control unit 156.

[0023] In Figure 2, the control unit 15 is shown to implement the first acquisition unit 151, the text information generation unit 152, the prompt generation unit 153, the second acquisition unit 154, the identification unit 155, and the display control unit 156, but the embodiment is not limited to this. For example, the control unit 15 may be configured by combining multiple independent processors, with each processor executing its respective program to realize these functions. Furthermore, the processing functions of the control unit 35 may be implemented by appropriately distributing or integrating them across one or more control units.

[0024] The first acquisition unit 151 acquires document data related to the document. Specifically, the first acquisition unit 151 acquires document data transmitted from the terminal device 50. The document is, for example, an invoice, receipt, or receipt. In other words, the document data is, for example, invoice data, receipt data, or receipt data. In this embodiment, the document data is data (image data, PDF, etc.) obtained by scanning a paper document received by mail, facsimile, etc. Note that the document is not limited to paper documents, and may also be electronic document data received by email, or document data in a unique format output from a core business system or order / receiver system.

[0025] The text information generation unit 152 generates text information, including text data and coordinate data, based on the document data. The text data is data relating to the text contained in the document data. For example, the text data includes text such as the title, business partner, payment amount, payment date, payment method, bank transfer destination, and claimant. The coordinate data is data relating to the coordinates of the text. For example, the coordinate data is two-dimensional coordinate data of each text contained in the text data, with a predetermined position in the document data as the origin.

[0026] The prompt generation unit 153 generates prompts to instruct the generation model to extract payment information and journal entry information based on the text data. Payment information is the information necessary for processing payments based on the supporting document data. For example, payment information includes the business partner, payment amount, payment date, payment method, recipient, branch, account type, account number, and account holder name. Payment information may also include information such as the invoice number and qualified billing registration number.

[0027] Journal entry information is information necessary for performing journal entry processing based on document data. Journal entry information includes, for example, one or more accounts selected from the account information included in the second prompt described later, the amount corresponding to the account, and the tax classification corresponding to the account. In this embodiment, journal entry information includes, along with one or more accounts, the amount corresponding to the account, and the tax classification corresponding to the account, billing source information and summary information. Billing source information is information about the billing source of the document in the document data. Billing source information is, for example, the name of the billing source. Summary information is information based on transaction information regarding the content of the transaction or items in the document data.

[0028] The second acquisition unit 154 inputs a prompt to the generation model and acquires payment information and journal entry information from the generation model. The identification unit 155 identifies the location of the text data in the supporting document data corresponding to the payment information and journal entry information based on the text information and the payment information and journal entry information.

[0029] The display control unit 156 controls the display unit to display the document data, payment information, and journal entry information. The display control unit 156 also highlights the text data in the document data corresponding to the payment information and journal entry information based on the position identified by the identification unit 155.

[0030] Next, the server device 30 will be described in detail with reference to Figure 3. Figure 3 is a block diagram showing an example of the configuration of a server device according to this embodiment. As shown in Figure 3, the server device 30 is configured to include a communication unit 31, a storage unit 33, and a control unit 35.

[0031] The communication unit 31 implements various communication protocols according to the configuration of the network NW. The communication unit 31 enables communication with other devices via the network NW according to the various communication protocols. In this embodiment, the server device 30 communicates with the information processing device 10 and the terminal device 50 via the communication unit 31.

[0032] The memory unit 33 is a non-transient memory device that stores various types of information and can be implemented using, for example, semiconductor memory elements such as RAM and flash memory, a hard disk, or an optical disc. For example, the memory unit 33 receives and stores various types of data acquired by the control unit 35. The memory unit 33 also stores programs corresponding to various functions executed by circuits included in the server device 30. The memory unit 33 may also be implemented using a group of servers (cloud) connected to the server device 30 via a network NW.

[0033] As shown in Figure 3, in this embodiment, the memory unit 33 includes a large-scale language model 331. The large-scale language model 331 is a language model trained to process natural language and outputs a response in text format or the like in response to an input prompt. Examples of large-scale language models 331 include GPT-3 and GPT-4 developed by OpenAI, and BERT developed by Google.

[0034] The control unit 35 is a control circuit that performs overall control of the server device 30, and also an arithmetic circuit that performs various calculations. In this embodiment, the control unit 35 includes an information acquisition unit 351, an information input unit 352, and an information output unit 353.

[0035] In Figure 3, the control unit 35 is shown to implement the information acquisition unit 351, the information input unit 352, and the information output unit 353, but the embodiment is not limited to this. For example, the control unit 35 may be configured by combining multiple independent processors, with each processor executing its own program to realize these functions. Furthermore, the processing functions of the control unit 35 may be implemented by appropriately distributing or integrating them across one or more control units.

[0036] The information acquisition unit 351 acquires information from the information processing device 10 via the communication unit 31. The information acquired by the information acquisition unit 351 is, for example, a prompt generated by the prompt generation unit 153 and input to the server device 30 by the second acquisition unit 154.

[0037] The information input unit 352 inputs the prompts acquired by the information acquisition unit 351 into the large-scale language model 331. The information input unit 352 also acquires payment information and journal entry information as output from the large-scale language model 331.

[0038] The information output unit 353 outputs information to the information processing device 10 via the communication unit 31. The information output by the information output unit 353 is, for example, payment information and journal entry information.

[0039] Next, the display process according to the embodiment will be described with reference to Figure 4. Figure 4 is a flowchart for explaining the display process according to the embodiment. This display process involves acquiring document data, generating text information, generating prompts, inputting prompts into a large-scale language model, acquiring payment information and journal entry information, identifying the location of payment information and journal entry information in the document data, and displaying the document data, payment information, and journal entry information. This display process is executed when document data is transmitted from the terminal device 50.

[0040] As shown in Figure 4, the first acquisition unit 151 in the control unit 15 of the information processing device 10 acquires the verification data (step S11). Specifically, the first acquisition unit 151 acquires the verification data transmitted from the terminal device 50.

[0041] Figure 5 shows an example of document data according to the embodiment. As shown in Figure 5, the document data ED1 is data that includes the following items: title A1, name of the billing party A2, name of the billing company A3, address of the billing company A4, contact information of the billing company A5, qualified billing company registration number A6, billing date A7, billing number A8, subject A9, billing amount A10, payment deadline A11, item A12, quantity A13, unit price A14, billing amount by item A15, subtotal A16, consumption tax amount A17, withholding tax amount A18, total amount A19, and bank transfer destination A20. In the example shown in Figure 5, the document data is invoice data that was scanned using a copier or camera from a paper invoice sent by mail or fax from the supplier. The data format (file type) is, for example, an image file, but is not particularly limited.

[0042] Next, as shown in Figure 4, the text information generation unit 152 in the control unit 15 of the information processing device 10 generates text information (step S13). Specifically, the text information generation unit 152 generates text information including text data and coordinate data based on the evidence data acquired in step S11.

[0043] More specifically, the text information generation unit 152 generates text data by extracting (transcribes) the text of the entire document data. In other words, in step S13, the text information generation unit 152 extracts the text of the entire document data, rather than extracting the text of a specific item from the document data. Furthermore, when extracting text data, the text information generation unit 152 generates coordinate data for each text, using one of the four corners of the document data as the origin. If the document data is obtained by scanning a paper document, the text information generation unit 152 may generate text data by performing text extraction processing using AI-based optical character recognition (AI-OCR) to improve the character recognition rate. In the above-mentioned generation of text coordinate data, one of the four corners of the document data was used as the origin, but the position of the origin is not limited to this. That is, the position of the origin is arbitrary, and for example, the center of the document data may be used as the origin.

[0044] Next, as shown in Figure 4, the prompt generation unit 153 in the control unit 15 of the information processing device 10 generates a prompt (step S15). Specifically, the prompt generation unit 153 generates a prompt to instruct the extraction of payment information and journal entry information based on text data. More specifically, in this embodiment, the prompt is composed of multiple prompts, and the prompt generation unit 153 generates at least a first prompt to instruct the generation model to extract payment information necessary for processing payment data based on the document data, and a second prompt to instruct the generation model to extract journal entry information in order to process journal entries based on the document data. In other words, in this embodiment, the prompt to instruct the extraction of payment information and the prompt to instruct the extraction of journal entry information are generated separately. By separating the prompts in this way, the processing time by the generation model can be shortened.

[0045] In step S15, the prompt generation unit 153 generates a first prompt and a second prompt based on a first prompt template for generating a first prompt and a second prompt template for generating a second prompt, along with text information. The first prompt template and the second prompt template are stored, for example, in the storage unit 13.

[0046] Figure 6 shows an example of a first prompt template according to the embodiment. In the example shown in Figure 6, the first prompt template P1 is used when the supporting data is invoice data. In other words, in the example shown in Figure 6, the supporting data is invoice data, and the prompt generation unit 153 generates a first prompt to instruct the extraction of payment information necessary for processing payment for expense claims based on the invoice data. As shown in Figure 6, the prompt generation unit 153 generates the first prompt by substituting the text data contained in the text information generated in step S13 into the variable "content" in condition B1 of the first prompt template P1.

[0047] Furthermore, in the example shown in Figure 6, the prompt generation unit 153 generates the first prompt so that the first prompt includes information about the recipient of the document in the document data. Here, the recipient of the document is the billing party if the document is an invoice. In other words, the information about the recipient of the document is, for example, the name of the billing party. As shown in Figure 6, the first prompt template P1 includes condition B21, "The following is text information read by OCR from an invoice received by {[User Company Name]}," and condition B22, "*The company that receives the invoice will not be the billing party," so as to include condition B2 regarding the recipient of the document. The prompt generation unit 153 then generates the first prompt by inputting the name of the billing party into the variable {[User Company Name]} in condition B21.

[0048] Furthermore, in the example shown in Figure 6, the prompt generation unit 153 generates a first prompt that includes instructions to extract one or more items from the text data. The one or more items extracted from the text data may include, for example, the identity of the billing party, the type of company of the billing party, the qualified business registration number, the email address, the billing amount, the billing date, the payment deadline, the invoice number, the payment method, the bank to which the payment will be made, the bank branch, the account type, the account number, the account holder's name, and the amount of withholding tax.

[0049] The legal entity of the claimant is either a corporation or an individual. For this reason, for example, the first prompt template P1 includes instructions for inferring whether the claimant is a corporation or an individual from the text data, as instructions for extracting the legal entity of the claimant from the text data. In other words, the prompt generation unit 153 generates the first prompt so that the first prompt includes instructions for inferring the legal entity of the claimant in the document data.

[0050] The claimant's company structure is, for example, a stock company, a limited liability company, a limited partnership, or a general partnership. For this reason, for example, the first prompt template P1 includes instructions for extracting the claimant's company structure from the text data, as well as instructions for inferring the company structure when the claimant is a legal entity. In other words, the first prompt generation unit 153 generates the first prompt so as to include instructions for inferring the company structure. Note that the claimant's company structure is not limited to a stock company, a limited liability company, a limited partnership, or a general partnership. The claimant's company structure may be a medical corporation or other form other than a stock company, a limited liability company, a limited partnership, or a general partnership. Also, as shown in Figure 6, the first prompt template P1 may include instructions regarding the claimant's company structure, such as "If the claimant's company structure is a stock company, it will always contain a legal entity abbreviation such as "(Ka)" or "Ka)", and if it is a medical corporation, it will always contain a legal entity abbreviation such as "(I)" or "I)".

[0051] The qualified business registration number is, for example, a 13-digit number starting with T. Therefore, for example, the first prompt template P1 includes an instruction to extract a 13-digit number starting with T as an instruction to extract the qualified business registration number from text data.

[0052] For example, the first prompt template P1 includes an instruction to extract the invoice amount from text data, which extracts only numbers with commas every three digits from the text data. For example, the first prompt template P1 includes an instruction to extract the invoice date from text data, which extracts the invoice date in "yyyy-mm-dd" format from the text data. Similarly, the first prompt template P1 includes an instruction to extract the payment due date from text data, which extracts the payment due date in "yyyy-mm-dd" format from the text data.

[0053] The invoice number is a number that uniquely identifies an invoice. For this reason, for example, the first prompt template P1 includes an instruction to extract a number that uniquely identifies an invoice, as an instruction to extract the invoice number from text data.

[0054] Payment methods include, for example, bank transfer, direct debit, cash, and credit card. For this reason, the first prompt template P1, for example, includes instructions to infer and select bank transfer, direct debit, cash, and credit card from the text data as instructions to extract the payment method from the text data.

[0055] Account types include, for example, savings accounts and current accounts. For this reason, the first prompt template P1, for example, includes an instruction to infer either a savings account or a current account from the text data as an instruction to extract the account type from the text data.

[0056] The account number is, for example, a 7-digit number. Therefore, for example, the first prompt template P1 includes an instruction to extract a 7-digit number from the text data as an instruction to extract the account number from the text data.

[0057] Furthermore, for example, the first prompt template P1 includes an instruction to extract only full-width katakana characters representing the account holder from the text data, as an instruction to extract the account holder's name from the text data. As shown in Figure 6, the first prompt template P1 may also include a condition for extracting the account holder's name, such as "Only katakana characters are allowed for the account holder's name, so please pay special attention if kanji or English characters are included, as this indicates an error." This reduces the possibility that the large-scale language model 331 may incorrectly extract the account holder's name from the text data.

[0058] Furthermore, for example, the first prompt template P1 includes an instruction to extract only numbers with commas every three digits from the text data, as an instruction to extract the withholding tax amount from the text data. As shown in Figure 6, the first prompt template P1 may also include a condition for extracting the withholding tax amount, such as "Please note that the withholding tax amount is withholding tax, not consumption tax." This reduces the possibility that the large-scale language model 331 may incorrectly extract the withholding tax amount from the text data.

[0059] In this embodiment, the first prompt template P1 is defined as extracting one or more items from text data, including the identity of the claimant, the type of company of the claimant, the qualified business registration number, the email address, the amount claimed, the date of claim, the payment deadline, the invoice number, the payment method, the bank to which the payment will be made, the bank branch, the account type, the account number, the account holder's name, and the withholding tax amount, but is not limited to these. In other words, the one or more items extracted from the text data are arbitrary, and the first prompt template P1 may include one or more items from among the following as the one or more items extracted from the text data: the identity of the billing party, the type of company of the billing party, the qualified business registration number, the email address, the billing amount, the billing date, the payment deadline, the invoice number, the payment method, the bank to which the payment is to be made, the bank branch, the account type, the account number, the account holder's name, and the withholding tax amount, or it may include items other than the identity of the billing party, the type of company of the billing party, the qualified business registration number, the email address, the billing amount, the billing date, the payment deadline, the invoice number, the payment method, the bank to which the payment is to be made, the bank branch, the account type, the account number, the account holder's name, and the withholding tax amount.

[0060] Figure 7 shows an example of a second prompt template according to the embodiment. In the example shown in Figure 7, the second prompt template P2 is used when the supporting data is invoice data. In the example shown in Figure 7, the prompt generation unit 153 generates a second prompt to instruct the extraction of journal entry information necessary for processing the journal entry of the invoice data.

[0061] Furthermore, as shown in Figure 7, the prompt generation unit 153 generates the second prompt such that the second prompt includes account information relating to the account titles used in the user's business. Specifically, as shown in Figure 7, the second prompt template P2 includes instruction B3 so that it can include account information relating to the account titles used in the user's business. The prompt generation unit 153 generates the second prompt by inputting account information relating to the account titles used in the user's business into instruction B3.

[0062] Furthermore, as shown in Figure 7, the prompt generation unit 153 generates a second prompt such that the second prompt includes instructions for inferring summary information based on transaction information in the document data. Specifically, the prompt generation unit 153 includes instruction B4, shown in Figure 7, "Read the body of the invoice, create a short sentence to be used as the summary of the journal entry, and output it to "journal_summary"" in the second prompt template P2 as instructions for inferring summary information. In other words, the inferred summary information is output to the variable "journal_summary". Note that although the second prompt template P2 in the embodiment described above includes instructions for inferring summary information, it does not have to include instructions for inferring summary information.

[0063] Furthermore, as shown in Figure 7, instruction B4 in the second prompt template P2 includes information to be included in the summary, information to be excluded from the summary, the length of the summary, notes on expression, and the output format. Note that instruction B4 is not limited to information to be included in the summary, information to be excluded from the summary, the length of the summary, notes on expression, and the output format, but may also include instructions other than information to be included in the summary, information to be excluded from the summary, the length of the summary, notes on expression, and the output format.

[0064] The information to be included in the summary is the information necessary for creating the summary. This includes, for example, the billing party, the details and items of the transaction, and supplementary information such as the period. However, supplementary information such as the period is not required to be included in the summary. In other words, supplementary information such as the period should be included in the summary only if necessary.

[0065] Information to be excluded from the summary is information that is not included when creating the summary. Examples of information to be excluded from the summary include the invoice number, amount, and date. However, the date may be included in the summary to the extent necessary, such as for a period or month.

[0066] The length of the summary refers to information about the number of characters included in the summary. For example, the second prompt template P2 may include instructions such as, "Please keep the summary length to a maximum of approximately 80 characters and summarize it concisely so that the main points are conveyed."

[0067] The formatting notes concern important points regarding the wording of the summary. For example, the second prompt template P2 may include formatting notes such as, "Please omit formal language such as 'desu' and 'masu' and use clear language suitable for accounting summaries," and "Please be as specific as possible, including quantities, breakdowns, and monthly amounts, to the extent that they can be read from the invoice."

[0068] The output format is information regarding the order in which the information to be included in the summary should be listed. For example, the second prompt template P2 may include instructions for the output format, specifying the order in which "supplementary information such as billing source / period, transaction details / items" should be listed.

[0069] Next, as shown in Figure 4, the second acquisition unit 154 in the control unit 15 of the information processing device 10 inputs a prompt to the large-scale language model 331 (step S17). Specifically, the second acquisition unit 154 transmits the prompt generated by the prompt generation unit 153 to the server device 30. The information acquisition unit 351 of the server device 30 receives the prompt transmitted from the information processing device 10. Subsequently, the information input unit 352 inputs the prompt to the large-scale language model 331. More specifically, the second acquisition unit 154 transmits the first prompt and the second prompt, respectively, to the server device 30. The information acquisition unit 351 of the server device 30 receives the first prompt and the second prompt, respectively, transmitted from the information processing device 10. Subsequently, the information input unit 352 inputs the first prompt and the second prompt, respectively, to the large-scale language model 331.

[0070] Next, as shown in Figure 4, the second acquisition unit 154 acquires payment information and journal entry information (step S19). Specifically, first, the information output unit 353 of the server device 30 acquires payment information and journal entry information from the large-scale language model 331. Then, the information output unit 353 transmits the payment information and journal entry information to the information processing device 10. Subsequently, the second acquisition unit 154 acquires payment information and journal entry information by receiving the payment information and journal entry information from the server device 30. Specifically, the second acquisition unit 154 acquires payment information generated using the first prompt and journal entry information extracted using the second prompt.

[0071] Next, as shown in Figure 4, the identification unit 155 in the control unit 15 of the information processing device 10 identifies the location of payment information and journal entry information in the document data (step S21). Specifically, the identification unit 155 identifies the location of text data in the document data corresponding to payment information and journal entry information based on text information and payment information and journal entry information. More specifically, the identification unit 155 compares the items included in the payment information and journal entry information with the text data included in the text information to identify the text data corresponding to the payment information and journal entry information, and identifies the location of the text data in the document data corresponding to the payment information and journal entry information based on the coordinate data corresponding to the text data. For example, if the payment information includes "AAA", the identification unit 155 identifies the location of the text data corresponding to "AAA" in the payment information based on the coordinate data of "AAA" in the document data.

[0072] Next, as shown in Figure 4, the display control unit 156 in the control unit 5 of the information processing device 10 displays the document data, payment information, and journal entry information (step S23). Specifically, the display control unit 156 displays the document data acquired in step S11 and the payment information and journal entry information acquired in step S19 on the terminal device 50. In addition, the display control unit 156 highlights the text data in the document data corresponding to the payment information and journal entry information based on the position identified by the identification unit 155 in step S21.

[0073] Figure 8 shows an example of a display screen that displays document data, payment information, and journal entry information according to the embodiment. As shown in Figure 8, the display screen SC1 displays the following items from the document data as payment information: name of the billing party C1, total amount C2, payment deadline C3, payment method C4, bank transfer destination C5, invoice number C6, billing date C7, and qualified billing registration number C8. The display screen SC1 also displays the following as journal entry information: account title C9 corresponding to item A12, item amount C10, tax classification C11, and description C12. Furthermore, because the document data ED1 includes withholding tax, that is, because the payment information includes withholding tax, the display control unit 156 generates journal entry data using a predetermined account title corresponding to the withholding tax, which is "deposits received". Therefore, the display screen SC1 shows the account C13, tax category C14, and amount C15 corresponding to the journal entry for withholding tax as journal entry information. Furthermore, the display control unit 156 highlights the items in the supporting document data ED1 corresponding to the name of the billing party C1, the total amount C2, the payment deadline C3, the payment method C4, the bank transfer destination C5, the invoice number C6, the billing date C7, and the qualified billing party registration number C8.

[0074] In step S23, the display process according to the embodiment is terminated by displaying the supporting document data, payment information, and journal entry information.

[0075] As described above, in the information processing device 10 of the information processing system 1 according to the embodiment, the information processing device 10 acquires document data relating to the document, generates text information including text data and coordinate data based on the document data, generates a prompt based on the text data, inputs the prompt to the generation model, acquires payment information and journal entry information, identifies the position of the text data in the document data corresponding to the payment information and journal entry information based on the text information, payment information and journal entry information, and highlights the text data in the document data corresponding to the payment information and journal entry information identified by the identification unit, thereby enabling high-precision reading of the document data.

[0076] Furthermore, in the information processing device 10 according to the embodiment, text information including text data and coordinate data is generated based on the document data, and a prompt including text data is input to the generation model to obtain items necessary for payment processing and journal entry processing in the document data, and the position of the text data in the document data corresponding to the payment information and journal entry information is identified. As a result, it is possible to extract items included in the document data without depending on the layout of the document data, while performing coordinate identification, which is difficult with the generation model. In other words, in the information processing device 10 according to the embodiment, coordinates can be identified while extracting items included in the document data without depending on the layout of the document data. For this reason, the information processing device can read the document data with high accuracy.

[0077] Furthermore, in the information processing device 10 according to this embodiment, the second acquisition unit 154 acquires payment information and journal entry information from the large-scale language model 331, that is, it generates payment information and journal entry information using the large-scale language model 331, so the cost of reading the supporting documents can be kept low.

[0078] Furthermore, in the information processing device 10 according to this embodiment, the prompt generation unit 153 is configured to input the name of the billing party into instruction B21, thereby reducing the possibility that the generation model may misidentify the billing source (invoice issuer) and the billing recipient (invoice recipient). Note that information regarding the recipient of the document does not necessarily need to be included in the prompt template.

[0079] Furthermore, in the information processing device 10 according to this embodiment, account information regarding the account titles used in the user's business is input into the second prompt template P2. Therefore, even if different account title settings are used for each user, the account titles can be accurately inferred.

[0080] Furthermore, in the information processing device 10 according to this embodiment, the prompt includes instructions for inferring the identity and company type of the claimant, which can be used for validating the account holder's name and reduce input errors in the account holder's name.

[0081] Furthermore, in the information processing device 10 according to this embodiment, instructions for inferring summary information are included in the prompt, so that a summary is suggested on the display screen SC1, reducing the effort required of the user to generate a summary that is mandatory to input under tax law.

[0082] Furthermore, in the information processing device 10 according to the above embodiment, when the document data reads the item for withholding tax amount, it proposes a deposit journal entry as journal entry information, thereby reducing the effort required for the user to generate a journal entry that takes the withholding tax amount item into consideration.

[0083] [Variation] In the embodiments described above, a prompt is composed of multiple prompts, but this is not limited to this. A prompt may be composed of one prompt. For example, the prompt generation unit 153 may generate one prompt by combining a first prompt and a second prompt.

[0084] Furthermore, although the above-described embodiment explained the case where the supporting data is invoice data, the supporting data may also be receipt data or invoice data. In this case, the prompt generation unit 153 may generate a first prompt to instruct the extraction of payment information necessary for processing payment for expense reimbursements based on the receipt data or invoice data.

[0085] Furthermore, in the embodiment described above, the display control unit 156 is configured to create a single journal entry, but it is not limited to this. The display control unit 156 can also be configured to create multiple journal entries. In this case, the journal entry information may include the account titles of the multiple journal entries and a breakdown of the amounts for each. Specifically, if the supporting data includes items related to consulting and items related to goods, the journal entry information may include, as account titles for the multiple journal entries, outsourcing expenses and purchases as debit accounts, accounts payable corresponding to outsourcing expenses and accounts payable corresponding to purchases as credit accounts, and a breakdown of the amounts corresponding to the account titles of the multiple journal entries. [Explanation of Symbols]

[0086] 1... Information processing system, 10... Information processing device, 11... Communication unit, 13... Storage unit, 15... Control unit, 30... Server device, 31... Communication unit, 33... Storage unit, 35... Control unit, 50... Terminal device

Claims

1. A first acquisition unit that acquires document data related to the document, A text information generation unit generates text information that includes text data relating to the text contained in the evidence data and coordinate data relating to the coordinates of the text, based on the aforementioned evidence data. A prompt generation unit generates prompts to instruct the generation model to extract payment information necessary for processing payment based on the aforementioned text data and to extract journal entry information necessary for processing journal entries based on the aforementioned proof data, A second acquisition unit inputs the aforementioned prompt to the generation model and acquires the payment information and the journal entry information from the generation model. An identification unit that identifies the location of the text data in the document data corresponding to the payment information and the journal entry information based on the text information, payment information and journal entry information, A display control unit that controls a display unit to display the document data, payment information, and journal entry information, the display control unit that highlights the text data in the document data corresponding to the payment information and journal entry information based on a position identified by the identification unit, An information processing device equipped with the following features.

2. The aforementioned prompt is composed of multiple prompts, The information processing apparatus according to claim 1, wherein the prompt generation unit generates at least a first prompt for instructing the generation model to extract payment information necessary for processing payment based on the document data, and a second prompt for instructing the generation model to extract journal entry information necessary for processing journal entry based on the document data, as the plurality of prompts.

3. The system includes a storage unit that stores a first prompt template for generating the first prompt and a second prompt template for generating the second prompt, The information processing apparatus according to claim 2, wherein the prompt generation unit generates the first prompt and the second prompt based on the first prompt template and the second prompt template together with the text information.

4. The aforementioned supporting document data is invoice data, The information processing apparatus according to claim 2, wherein the prompt generation unit generates a first prompt for instructing the extraction of payment information necessary for processing payment for expense claims based on the invoice data.

5. The aforementioned supporting document data is receipt data or invoice data. The information processing apparatus according to claim 2, wherein the prompt generation unit generates a first prompt for instructing the extraction of payment information necessary for processing payment for expense settlements based on receipt data or invoice data.

6. The information processing apparatus according to claim 2, wherein the prompt generation unit generates the first prompt such that the first prompt includes information about the recipient of the document in the document data acquired by the first acquisition unit.

7. The information processing apparatus according to claim 2, wherein the prompt generation unit generates the second prompt such that the second prompt includes account information relating to account titles for journal entries used in the user's business.

8. The information processing apparatus according to claim 7, wherein the journal entry information includes at least one or more accounts selected from the accounts in the account information included in the second prompt, an amount corresponding to the account, and a tax category corresponding to the account.

9. The information processing device according to claim 8, wherein the journal entry information includes the one or more account titles, the amount, the tax classification, and summary information based on the billing source information relating to the billing source of the document in the document data and transaction information relating to the content or items of the transaction in the document data.

10. The information processing apparatus according to claim 2, wherein the prompt generation unit generates the first prompt such that the first prompt includes instructions for inferring the personality and company type of the claimant of the document in the document data.

11. The information processing apparatus according to claim 1, wherein the display control unit generates journal entry data using a predetermined account corresponding to the withholding tax amount when the payment information includes a withholding tax amount.

12. The first acquisition unit takes the step of acquiring document data related to the document, The text information generation unit generates text information based on the document data, including text data relating to the text contained in the document data and coordinate data relating to the coordinates of the text. The prompt generation unit generates prompts to instruct the generation model to extract payment information necessary for processing payment based on the document data and to extract journal entry information necessary for processing journal entries based on the document data, based on the text data. The second acquisition unit inputs the prompt to the generation model and acquires the payment information and the journal entry information, The identifying unit identifies the location of the text data in the document data corresponding to the payment information and the journal entry information based on the text information, the payment information and the journal entry information, The display control unit controls the display unit to display the document data, payment information, and journal entry information, and the step of highlighting the text data in the document data corresponding to the payment information and journal entry information based on a specified position. An information processing method comprising:

13. A program configured to cause a computer to execute the information processing method described in claim 12.

Images