Review work document creation support device, review work document creation support method and review work document creation support program

Abstract

To provide a review work document creation support device for supporting financial institution personnel for creating a review work document.SOLUTION: A review work document creation support device in a financial institution includes: question creation means for creating question information for causing a language model to respond to the question information for input items provided in a format of a review work document; question transmission means for transmitting the question information to the language model; response receiving means for receiving response information corresponding to the question information from the language model; and response input means for inputting input information based on the response information, into the input items.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an examination work document creation support device, an examination work document creation support method, and an examination work document creation support program. [Background technology]

[0002] A business feasibility assessment is a process in which financial institutions evaluate a company when deciding whether to provide loans to a client company, taking into account not only quantitative information such as the company's financial statements, but also qualitative information such as industry trends, the company's strengths and weaknesses, future vision, and management issues. Generally, when conducting a business feasibility assessment, a document called a "business feasibility assessment sheet" is prepared as a sheet format for financial institutions to use for internal information communication. A business feasibility assessment sheet is a sheet format / format that has been uniquely devised to enable financial institutions to appropriately evaluate companies. The staff at the branch office fill in the sheet items by collecting quantitative information such as the company information and financial statements of the company they are in charge of, and summarizing qualitative information based on interviews with the management, such as industry trends, the company's strengths and weaknesses, future vision, and management issues. The completed business feasibility assessment sheet is mainly used, for example, as an attachment when drafting a loan request document.

[0003] As a related technique, for example, Patent Document 1 describes a method in which a business feasibility assessment sheet uses standardized processing results based on answers to specific questions from the customer, and the results are revised to reflect the actual situation to identify the customer's business issues and provide information on a support menu for dealing with those issues. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-091678 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, although the invention described in the above Patent Document 1 supports the creation of a business feasibility evaluation sheet, it is made to specialize in supporting the creation of a sheet item called "management issues and support needs." In fact, there are many other sheet items. In addition, when creating the sheet items, a process of obtaining answers to certain questions from the customer is essential.

[0006] The present invention has been proposed in view of the above points, and has an object in one aspect to support financial institution personnel in creating examination work documents. [Means for solving the problem]

[0007] In order to solve the above problems, the review work document creation support device of the present invention is a review work document creation support device for a financial institution, and has a question generation means for generating question information to allow a language model to answer input information for input fields provided in the format of a review work document, a question sending means for sending the question information to the language model, an answer receiving means for receiving answer information corresponding to the question information from the language model, and an answer input means for inputting input information based on the answer information into the input field. Effect of the Invention

[0008] According to one aspect of the embodiment of the present invention, it is possible to support financial institution personnel in creating examination work documents. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram showing an example of the configuration of an examination work document preparation support system 100 according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing an example of knowledge data in a knowledge base 30 according to the present embodiment. [Diagram 3] FIG. 2 is a diagram showing an example of labeled training data for the fine-tuned large-scale language model 40 according to the present embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a functional configuration of an examination work document creation support server 20 according to the present embodiment. [Diagram 5] 1 is a diagram showing an outline of the operation of the review work document preparation support system 100 according to the present embodiment. [Figure 6] FIG. 2 is a diagram showing an examination work document creation process 1 according to the present embodiment. [Figure 7] FIG. 2 is a diagram showing an examination work document creation process 2 according to the present embodiment. [Figure 8A] 13 is a diagram showing example questions and answers for the sheet item "1. Corporate attribute information" according to the embodiment. FIG. [Figure 8B] FIG. 13 is a diagram showing example questions and answers for the sheet item “2. Business details” according to the present embodiment. [Figure 8C] FIG. 13 is a diagram showing example questions and answers for the sheet item “3. Shareholder / Investor Information” according to the present embodiment. [Figure 8D] FIG. 13 is a diagram showing example questions and answers for the sheet item “4. Management Information” according to the present embodiment. [Figure 8E] FIG. 13 is a diagram showing example questions and answers for the sheet item “5. Industry trends” according to the present embodiment. [Figure 8F] FIG. 13 is a diagram showing example questions and answers for the sheet item “6. SWOT analysis” according to the present embodiment. [Figure 8G] FIG. 13 is a diagram showing example questions and answers for the sheet item “break-even sales analysis” according to the embodiment. [Figure 8H] FIG. 13 is a diagram showing example questions and answers for a sheet item "cash flow analysis" according to the present embodiment. [Figure 9] FIG. 2 is a diagram showing an example 1 of sheet items of a business feasibility evaluation sheet according to the present embodiment. [Figure 10] FIG. 2 is a diagram showing example 2 of sheet items of a business feasibility evaluation sheet according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to the drawings. <System configuration> Fig. 1 is a diagram showing an example of the configuration of a screening work document preparation support system 100 according to this embodiment. The screening work document preparation support system 100 in Fig. 1 includes a financial institution system 10, a screening work document preparation support server 20, a knowledge base (vector DB) 30, a fine-tuned large-scale language model (LLM) 40, and a terminal 50, which are connected via a network 70.

[0011] The financial institution system 10 includes various systems and DBs (databases) owned by a financial institution such as a bank. The financial institution system 10 includes, for example, a core system that manages transactions such as various deposits and loans, a sales management system that manages sales records of financial products, a front compliance system for appropriately managing transaction details in accordance with laws and regulations, an SFM (Sales Force Management System) and CRM (Customer Relationship Management) that manage customer information including loan history and negotiation records with customers (sales records), and each DB associated with each system. Each DB of the financial institution system 10 includes, for example, DBs for customer information, transaction data, financial data, negotiation record data, etc. of client companies at the financial institution.

[0012] The screening business document creation support server (hereinafter also simply referred to as the support server) 20 is a server device that supports the creation of screening business documents to be created by financial institution personnel. When the support server 20 receives a request to create, for example, a business feasibility evaluation sheet as a screening business document from the terminal 50 of the personnel, the support server 20 creates a business feasibility evaluation sheet by asking questions to a fine-tuned large-scale language model (hereinafter also simply referred to as the language model) 40 about predetermined input items (hereinafter also referred to as sheet items) in the business feasibility evaluation sheet and filling in the sheet items based on the answers obtained. A specific creation example will be described later.

[0013] The knowledge base 30 is a knowledge database in which information on client companies of financial institutions (called customer relationship information) is stored and constructed in advance, for example, by vector conversion. The customer relationship information includes not only public information about client companies that is publicly available, but also private information such as in-bank information. Since the language model 40 cannot answer questions about contents other than those included in the learning data, even if an attempt is made to answer a question specific to the client company, for example, an answer that the question is not known or an answer based on different knowledge may be obtained. As a method for solving this problem, in this embodiment, customer relationship information that is likely to be necessary to answer the question is stored (stored) in the knowledge base 30 in advance. The support server 20 first searches the customer relationship information stored in the knowledge base 30 for question-related information that is likely to be necessary to answer the question, and passes the searched question-related information to the language model 40 in addition to a prompt, which is an instruction / input sentence. This makes it possible to have the language model 40 think of an appropriate answer even if the content is not learned by the language model 40.

[0014] The fine-tuned large-scale language model 40 is a language model in which knowledge of a specific domain is further supervised and learned from a general-purpose natural language processing model (LLM: Large Language Models), which has been pre-trained in an unsupervised manner using a large amount of text data to learn grammar, word meanings, and the like. The language model 40 is made capable of generating highly specialized answer sentences required for review work documents by fine-tuning a set of appropriate "questions" and desirable "answers" in the general-purpose natural language processing model. For example, the general-purpose natural language processing model may be BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pre-trained Transformer)-3.5, GPT-4, etc. Note that instead of a network system configuration in which the language model 40 is stored in a storage device within the bank's network 70, it is also possible to access a server of a fine-tuned large-scale language model provided as an external service and read it out.

[0015] The terminal 50 is, for example, a PC (personal computer), a smartphone, a tablet terminal, or the like, and is, for example, a user terminal used by a person in charge of a sales office. A predetermined application program for accessing the support server 20, a general-purpose web browser, and the like are pre-installed in the terminal 50. The person in charge uses the terminal 50 to access the support server 20, obtains the business feasibility evaluation sheet created by the support server 20, and displays it on the screen. Note that, depending on the sheet items in the business feasibility evaluation sheet, there may be desirable items that the person in charge has filled in himself / herself, so the support server 20 does not necessarily complete the business feasibility evaluation sheet by filling in all the sheet items in the business feasibility evaluation sheet. The person in charge can complete the final business feasibility evaluation sheet by reviewing and supplementing the business feasibility evaluation sheet created primarily by the support server 20.

[0016] The network 70 is a communication network of the financial institution, and is preferably an in-bank network from the viewpoint of security. However, the network 70 may include, for example, the Internet, a public line network, WiFi (registered trademark), etc., provided that sufficient security is ensured.

[0017] (Knowledge Base 30) FIG. 2 is a diagram showing an example of knowledge data in the knowledge base 30 according to this embodiment. The knowledge base 30 is a knowledge database in which customer relationship information in a financial institution is stored and constructed in advance after, for example, being vector-converted so as to be easily searched. The knowledge data in the knowledge base 30 according to this embodiment includes, as customer relationship information of client companies in the financial institution, for example, customer information, transaction data, financial data, and negotiation record data of Company A, Company B, Company C, etc. Furthermore, the knowledge data in the knowledge base 30 according to this embodiment includes, for example, industry-specific examination dictionary information and corporate enterprise statistics information, which are stored as external reference materials.

[0018] The customer information is customer information of a customer company that does business with a financial institution. In addition to general company information such as trade name or name, location, telephone number, and representative name, the information includes information required to be submitted when opening an account, such as industry and business type. The customer information can be acquired, for example, from the core system of the financial institution system 10. The customer information can be further supplemented and enriched with information such as the president's greeting, our strengths, future vision, business content, products and services, bases, sales areas, history, major clients, and shareholders, based on information such as the customer company's own website posted by scraping or other means, or a corporate information database provided by an external credit investigation company. The business feasibility evaluation sheet includes sheet items such as "corporate attribute information," "business content," "shareholder and investor information," "management information," "industry trends," and "SWOT analysis," and the customer information can be used as one of the question-related information for these questions when answering them using the language model 40.

[0019] The transaction data includes, for example, information on each account held by each customer, such as ordinary savings accounts, current accounts, fixed term deposit accounts, and foreign currency deposit accounts, credit card information, and transaction details including foreign exchange transactions (liquidity transaction details information), as well as information on loans made in the past. All transaction details from the present to the past are recorded. The transaction data can be obtained, for example, from the core system of the financial institution system 10. The business feasibility evaluation sheet includes, for example, a sheet item called "corporate attribute information," and the transaction data can be used as one piece of question-related information for these questions when the language model 40 responds.

[0020] The financial data includes not only financial statements (balance sheets, profit and loss statements, cash flow statements, etc.) for each business year of the client company, but also data related to the finances of the client company, such as trial balance data for the period showing the progress of profit and loss for the year. The financial data can be acquired, for example, from the core system of the financial institution system 10. The business feasibility evaluation sheet includes sheet items such as "shareholder / investor information," "management information," "industry trends," "SWOT analysis," "break-even sales analysis," and "cash flow analysis," and the financial data can be used as one piece of question-related information for these questions when the language model 40 answers the questions.

[0021] The negotiation record data is a record history of negotiations that have been conducted in the past between the financial institution side, such as a person in charge of a branch office, and the client company side, such as the manager or a person in charge of the financial department. It includes not only documents and emails exchanged between the two parties, but also conversation items (such as the manager's future vision, management issues, succession issues, etc.) discussed in dialogues such as interviews (visits, visits to the branch), online conferences, and telephone calls, as long as they are recorded in the form of text, audio, images, videos, etc. The negotiation record data can be obtained, for example, from SFM or CRM that manages customer information including the loan history of the financial institution system 10 and negotiation records with customers (sales records). The business feasibility evaluation sheet has sheet items such as "industry trends" and "SWOT analysis," and the negotiation record data can be used as a piece of question-related information for these questions when the language model 40 answers them.

[0022] The industry-specific screening dictionary information is information from an industry information dictionary that covers all industries and business types in Japan and compiles industry trends, information analysis, screening points, etc. for each industry. It is used as a reference by financial institution personnel when conducting loan screening to understand the type of industry of the borrower. A business feasibility evaluation sheet may have sheet items such as "industry details," "industry trends," and "SWOT analysis," and the industry-specific screening dictionary information can be used as a piece of question-related information for these questions when the language model 40 responds. This is expected to enable the language model 40 to provide responses that are more in line with the perspective of the financial institution's loan screening.

[0023] The Corporate Enterprise Statistics Survey information is information from a core statistical survey conducted as a sample survey based on the Statistics Act in order to grasp the actual state of business activities of profit-making corporations and the like in Japan. Survey results include financial indicators such as sales, capital investment, and current profits of profit-making corporations and the like, as well as the status of business activities by industry, size, and region. For example, the business feasibility evaluation sheet has sheet items such as "industry trends" and "SWOT analysis," and the Corporate Enterprise Statistics Survey information can be used as a piece of question-related information for these questions when the language model 40 responds. This is expected to enable the language model 40 to provide responses that are more in line with the perspective of financial institutions' loan screening.

[0024] (40 fine-tuned large-scale language models) 3 is a diagram showing an example of labeled learning data of the fine-tuned large-scale language model 40 according to this embodiment. As described above, the fine-tuned large-scale language model 40 is a language model in which knowledge of a specific domain is further supervised-learned on a general-purpose natural language processing model (LLM) that has been pre-trained in an unsupervised manner on grammar, word meanings, and the like using a large amount of text data. A model capable of generating highly specialized answer sentences required for review work documents is created by learning (fine-tuning) a sentence set of "questions" and exemplary "answers" to elicit appropriate answers to the sheet items on a business feasibility evaluation sheet.

[0025] For example, the business feasibility evaluation sheet has a sheet item called "History," and as a sample set of questions and model answers to obtain appropriate answers to this sheet item "History," multiple sets of "Question: Summarize the history of Company A in chronological order using XXX characters" and "Answer: In 2020, XXX founded Company A in XX ward, Tokyo. In 2021..." (learning data a-1)... are trained in advance (fine-tuned).

[0026] In addition, the business feasibility evaluation sheet has a sheet item called "SWOT analysis," for example. As a sample set of questions and model answers to obtain an appropriate answer to this sheet item "SWOT analysis," the following are provided: "Question: You are a banker. As a banker conducting a loan review, please conduct a SWOT analysis from the perspective of the strengths and weaknesses of Company A's internal environment, and the opportunities and threats in the external environment. The analysis should take into account the company's financial data as well as the future vision and management issues as envisioned by the management." and "Answer: S (Strengths): ··, W (Weaknesses): ··, O (Opportunities): ··, T (Threats): ··" (learning data a-6) ··. Multiple sets of these are trained (fine-tuned) in advance.

[0027] Furthermore, for a client company whose business feasibility evaluation sheet has been output in the past by the support server 20 through the processes of S1 to S5 in Fig. 6 and S21 to S23 in Fig. 7 described below, the language model 40 may be trained (fine-tuned) on the questions (S3) and question-related information (S5) generated for each sheet item in the past, and the answers (S22) in the language model 40 input corresponding to the sheet items. Furthermore, instead of the answers (S22) in the language model 40, if the answers have been further revised by the person in charge to be more desirable, the revised answers obtained from the final business feasibility evaluation sheet may be trained.

[0028] (Functional configuration) 4 is a diagram showing an example of the functional configuration of the review work document creation support server 20 according to this embodiment. The support server 20 has, as main functional units, a document creation request receiving unit 201, an item identifying unit 202, a question generating unit 203, a search unit 204, a question sending unit 205, an answer receiving unit 206, an answer input unit 207, a document output unit 208, and a memory unit 209.

[0029] The document creation request receiving unit 201 has a function of receiving, from the terminal 50 of the person in charge, a request to create an examination business document (for example, a business feasibility evaluation sheet) specifying a client company for which the document is to be created.

[0030] The item specification unit 202 has a function of specifying a predetermined answer input item according to the type of the review business document from among the input items provided in the format of the review business document.

[0031] The question generation unit 203 has a function of generating question information for causing the language model to answer input information for a predetermined answer input item.

[0032] The search unit 204 has a function of searching for question-related information related to the generated question information from a knowledge base in which customer relationship information on customers is accumulated.

[0033] The question sending unit 205 has a function of sending question information and customer relationship information to the language model 40 .

[0034] The answer receiving unit 206 has a function of receiving answer information corresponding to question information from the language model 40.

[0035] The answer input unit 207 has a function of inputting answer information into predetermined answer input items.

[0036] The document output unit 208 has the function of outputting the review business document (e.g., a business feasibility evaluation sheet) to the terminal 50 once answer information based on answers from the language model 40 has been input into all specified answer input items in the review business document.

[0037] The memory unit 209 has a function of storing, in a storage device, information on the format of examination work documents, standard questions for each sheet item, and the like.

[0038] The review work document creation support server 20 can be realized by a general-purpose computer. Specifically, the support server 20 includes hardware such as a processing unit (CPU), a memory, an input / output interface, and a communication interface. The functions of the support server 20 are realized by the processing unit executing processing according to a computer program stored in the memory. That is, each functional unit is realized by a computer program executed on the hardware resources such as the processing unit and memory of the computer constituting the support server 20. The review work document creation support server 20 may be called a review work document creation support device or a computing machine for supporting review work document creation. These functional units may also be read as "means," "module," "unit," or "circuit." Each DB may be placed in the memory of the support server 20 or an external storage device on the network 70. Each functional unit of the support server 20 may not only be realized by a single server device, but may also be realized as a system consisting of multiple devices with distributed functions. The computer program may also be stored in a computer-readable storage medium.

[0039] <Support for creating review documents> Fig. 5 is a diagram showing an outline of the operation of the review work document creation support system 100 according to this embodiment. Fig. 6 is a diagram showing review work document creation processing 1 according to this embodiment. The arithmetic processing unit of the support server 20 reads and executes a program capable of realizing the processing, thereby realizing each of the following steps (hereinafter abbreviated as "S").

[0040] S1: The support server 20 receives a request to create an examination work document (e.g., a business feasibility evaluation sheet) specifying a client company for which the document is to be created, from the person in charge's terminal 50. The creation request includes at least an identifier (e.g., a client number, a client company name, etc.) for identifying the target client company.

[0041] S2: The support server 20 identifies sheet items (predetermined answer input items) to be answered by the language model 40 from among the sheet items (input items) predefined in the format of the review work document stored in the storage unit 209. Since some sheet items in the review work document sheet are desired items filled in by the person in charge himself / herself, the sheet items to be filled in by the support server 20 are predefined according to the type of review work document. In the case of a business feasibility evaluation sheet, for example, "corporate attribute information", "business content", "shareholder / investor information", "management information", "industry trends", "SWOT analysis", "break-even sales analysis", and "cash flow analysis" can be identified as sheet items to be answered by the language model 40.

[0042] S3: The support server 20 generates a question (problem) for answering the sheet item specified in S2 for each sheet item. Note that the sentences of the questions (problems) generated for each sheet item are determined for each sheet item and stored in a standard format in the storage unit 209, and in order to obtain an exemplary answer from the language model 40, the question sentences are in the same format as the sample questions for obtaining an appropriate answer to the sheet item in question in the fine-tuning situation (FIG. 3).

[0043] Specifically, the support server 20 generates a prompt such as "Summarize the history of Company A in chronological order in XXX characters" as a question to answer "History" in one sheet item "Corporate attribute information" identified in S2. The support server 20 also generates a prompt such as "You are a banker. As a banker conducting loan screening, please perform a SWOT analysis of Company A from the perspective of the strengths and weaknesses of the internal environment and the opportunities and threats in the external environment. The analysis should take into account the company's financial data as well as the future vision and management issues as envisioned by the management." as a question to answer "SWOT analysis" in one sheet item identified in S2.

[0044] In addition, questions to answer all of the sheet items identified in S2 may be created as one question, such as, "In order to create a business feasibility evaluation sheet for Company A, please summarize the following in a maximum of XXX characters or less."

[0045] S4: The support server 20 searches and acquires question-related information similar to the question generated in S3 from the knowledge base 30. Specifically, the support server 20 converts the sentence of the question generated in S3 directly into a vector as a query, searches the knowledge base 30 (knowledge vector DB), and acquires similar sentences with close vector distance as question-related information.

[0046] Specifically, as a similar sentence close to the question generated in S3, "Summarize the history of Company A in chronological order in XXX characters," for example, corporate information about Company A (≒ "Company A" and "History") can be obtained as question-related information. In addition, as a similar sentence to the question generated in S3, "You are a bank employee. As a bank employee conducting a loan screening, please conduct a SWOT analysis of Company A from the perspective of the strengths and weaknesses of the internal environment and the opportunities and threats of the external environment. The analysis should take into account the company's financial data as well as the future vision and management issues as envisioned by the management.", for example, corporate information of Company A (≒ "Company A", "Management", "Strengths and Weaknesses", "Future Vision"), financial data of Company A (≒ "Company A", "Financial Data", negotiation record data of Company A (≒ "Company A", "Strengths and Weaknesses", "Future Vision", "Management Issues"), screening dictionary information corresponding to Company A's industry (≒ "Company A", "Loan Screening", "Banker"), and corporate statistics information of Company A's industry (≒ "External Environment") can be obtained as question-related information.

[0047] In addition, in order to answer the question with a SWOT analysis based on the perspective of a financial institution, which is unique to screening work documents, words such as "loan screening" and "bank employee" can be included in the question, making it possible to search for similar sentences in the screening dictionary information corresponding to Company A's industry. As described above, the industry-specific screening dictionary information covers all industries and business types in Japan, and is information from an industry information dictionary that summarizes industry trends, information analysis, screening points, etc. for each business type. It is used by financial institution personnel to understand the type of business of the borrower when conducting loan screening. For this reason, when searching and acquiring screening dictionary information as a knowledge base, it is extremely important to identify the type of business that Company A belongs to. When searching and acquiring question-related information similar to the question (question) generated in S3 from the knowledge base 30, the support server 20 can explicitly enter Company A's business type in the query, thereby making it possible to search and acquire screening dictionary information corresponding to Company A's business type as a similar sentence.

[0048] In this case, it is advisable for the support server 20 to specify the industry of company A in advance at least before S4. A specific method of specification may be, for example, acquisition from the core system of the financial institution system 10, and if it is not possible to acquire the information from the core system or it is difficult to do so, acquisition may be based on information collected by scraping or the like from the customer company's own website or from a company information database provided by an external credit investigation company or the like.

[0049] S5: The support server 20 transmits the question generated in S3 and the question-related information searched and acquired in S4 to the language model 40. Specifically, the support server 20 embeds the generated question and the question-related information searched and acquired from the knowledge base 30 in a prompt, and inputs them to the language model 40 as an inquiry.

[0050] FIG. 7 is a diagram showing the examination work document creation process 2 according to this embodiment. S21: The support server 20 receives an answer from the language model 40 as a response to the inquiry in S5. For example, as a response to the question "Summarize the history of Company A in XXX characters starting from the oldest." and the question-related information "Company information on Company A", the answer text "In 2020, XX founded Company A in XX ward, Tokyo. In 2021, the head office moved to XX ward, Tokyo..." can be received. Note that the answer from the language model 40 is an answer text in the same format as the desirable sample answer based on fine tuning (Figure 3).

[0051] S22: The support server 20 inputs the answer received in S21 into the sheet item (predetermined answer input item) identified in S2 in the review work document. For example, into one sheet item "History" identified in S2, the answer text "In 2020, XX founded company A in XX ward, Tokyo. In 2021, the head office moved to XX ward, Tokyo..." is input. The answer text may be directly transcribed into the sheet item, or the format, etc. may be processed according to the form of the sheet item.

[0052] S23: When the support server 20 has finished inputting all sheet items in the document that should be answered by the language model 40, it outputs the examination work document to the terminal 50. In the business feasibility evaluation sheet according to this embodiment, the sheet items that the language model 40 should answer are, for example, "corporate attribute information," "business content," "shareholder / investor information," "management information," "industry trends," "SWOT analysis," "break-even sales analysis," and "cash flow analysis." Therefore, when all answers from the language model 40 corresponding to these sheet items have been inputted into the business feasibility evaluation sheet, the business feasibility evaluation sheet for the designated client company is output to the terminal 50.

[0053] <Examples of questions and answers for sheet items> Among the sheet items provided in the format of the business feasibility evaluation sheet, examples of sheet items that the support server 20 should fill in using the answers from the language model 40 include "corporate attribute information," "business content," "shareholder / investor information," "management information," "industry trends," "SWOT analysis," "break-even sales analysis," and "cash flow analysis." Specific examples are given below.

[0054] FIG. 8A is a diagram showing an example of a question and answer for the sheet item "1. Corporate attribute information" according to this embodiment. Question Q1 is a question (prompt) generated to have the language model 40 answer the sheet item "1. Corporate attribute information." Furthermore, question Q1 is input as a question to the language model 40 with the corporate information of Company A searched and acquired from the knowledge base 30 and transaction data of Company A embedded in the prompt. Answer A1 is an answer received from the language model 40 as an answer to question Q1. The support server 20 inputs (transcribes) the text of answer A1 received from the language model 40 into sheet item "1. Corporate attribute information" IN1 of the business feasibility evaluation sheet.

[0055] FIG. 8B is a diagram showing an example of a question and answer for the sheet item "2. Business content" according to this embodiment. Question Q2 is a question (prompt) generated to cause the language model 40 to answer the sheet item "2. Business content". Furthermore, question Q2 is input as a question to the language model 40 with the corporate information of Company A searched and acquired from the knowledge base 30 and the examination dictionary information for Company A's industry embedded in the prompt. Answer A2 is an answer received from the language model 40 as an answer to question Q2. The support server 20 inputs (transcribes) the text of answer A2 received from the language model 40 into sheet item "2. Business content" IN2 of the business feasibility evaluation sheet.

[0056] It should be noted that the sheet item “2. Business Content” IN2 includes an item “(1) Industry.” However, since the customer company’s “(1) Industry” can be obtained and input from the customer information DB or the like held by the financial institution system 10 without having to have the language model 40 answer the question, in this embodiment, question Q2 does not include a question regarding “(1) Industry.”

[0057] FIG. 8C is a diagram showing an example of a question and answer for the sheet item "3. Shareholder / Investor Information" according to this embodiment. Question Q3 is a question (prompt) generated to cause the language model 40 to answer the sheet item "3. Shareholder / Investor Information". Furthermore, question Q3 is input as a question to the language model 40 with the corporate information of Company A and financial data of Company A searched and acquired from the knowledge base 30 embedded in the prompt. Answer A3 is an answer received from the language model 40 as an answer to question Q3. The support server 20 inputs (transcribes) the text of answer A3 received from the language model 40 into the sheet item "3. Shareholder / Investor Information" IN3 of the business feasibility evaluation sheet.

[0058] FIG. 8D is a diagram showing an example of a question and answer for the sheet item "4. Management Information" according to this embodiment. Question Q4 is a question (prompt) generated to cause the language model 40 to answer the sheet item "4. Management Information". Furthermore, question Q4 is input as a question to the language model 40 with the corporate information of Company A searched and acquired from the knowledge base 30 and financial data of Company A embedded in the prompt. Answer A4 is an answer received from the language model 40 as an answer to question Q4. The support server 20 inputs (transcribes) the text of answer A4 received from the language model 40 into sheet item "4. Management Information" IN4 of the business feasibility evaluation sheet.

[0059] FIG. 8E is a diagram showing an example of a question and an answer for the sheet item "5. Industry trends" according to this embodiment. Question Q5 is a question (prompt) generated to make the language model 40 answer the sheet item "5. Industry trends". In addition, the question Q5 is input to the language model 40 as a question, with the corporate information of Company A, financial data of Company A, examination dictionary information for Company A's industry, and corporate statistical information for Company A's industry searched and acquired from the knowledge base 30 embedded in the prompt. Answer A5 is an answer received from the language model 40 as an answer to question Q5. The support server 20 inputs (transcribes) the text of answer A5 received from the language model 40 into sheet item "5. Industry trends" IN5 of the business feasibility evaluation sheet.

[0060] FIG. 8F is a diagram showing an example of a question and an answer for the sheet item "6. SWOT analysis" according to this embodiment. The question Q6 is a question (prompt) generated to make the language model 40 answer the sheet item "6. SWOT analysis". In addition, the question Q6 is input to the language model 40 as a question by embedding the corporate information of the company A searched and acquired from the knowledge base 30, the financial data of the company A, the negotiation record data of the company A, the examination dictionary information of the industry of the company A, and the corporate statistical information of the industry of the company A in the prompt. The answer A6 is an answer received from the language model 40 as an answer to the question Q6. The support server 20 inputs (transcribes) the sentence of the answer A6 received from the language model 40 into the sheet item "6. SWOT analysis" IN6 of the business feasibility evaluation sheet.

[0061] Here, for items such as SWOT analysis in a business feasibility evaluation sheet, quantitative information such as general information about the client's industry and business sector, or financial data is insufficient; qualitative information such as the client's daily thoughts and perspective is essential for analysis. For this reason, in the past, when creating a business feasibility evaluation sheet, sales office staff would fill in the sheet items by collecting quantitative information such as corporate information and financial statements of the company they were in charge of, or by analyzing and summarizing qualitative information based on interviews with the client's management.

[0062] For example, the answer A6 according to this embodiment is an answer to the question Q6 in which the negotiation record data of Company A searched and acquired from the knowledge base 30 is embedded in the prompt, and the answer from the language model 40 is an answer based on the information recorded in the negotiation record data. As described above, the negotiation record data is a record history of negotiations that were previously conducted between the financial institution side, such as a sales office staff member, and the client company side, such as the manager or a financial department staff member, which was originally acquired from SFM or CRM. It includes not only documents and emails exchanged between the two parties, but also conversation matters (the manager's future vision, management issues, succession issues, etc.) discussed in dialogues such as interviews (visits, visits to the branch), online conferences, and telephone calls, as long as they are recorded. Therefore, for example, the sentences in the answer A6, "The manager aims to acquire the top share in the industry within three years, and it is expected that this will be achieved." and "Improving development capabilities is a management issue," are answer sentences that reflect the goal (to acquire the top share in the industry within three years), the likelihood of achieving this (high), and the management issue (improving development capabilities) spoken by the manager of Company A, which were recorded as the negotiation record history.

[0063] FIG. 8G is a diagram showing an example of a question and an answer for the sheet item "Break-even sales analysis" according to this embodiment. Question Q7 is a question (prompt) generated to have the language model 40 answer the sheet item "x. Break-even sales analysis." Furthermore, the financial data (profit and loss statement) of company A searched and acquired from the knowledge base 30 is embedded in the prompt for question Q7, and the question is input to the language model 40 as a question. Answer A7 is an answer received from the language model 40 as an answer to question Q7. The support server 20 inputs (transcribes) a graph of answer A7 received from the language model 40 into the sheet item "x. Break-even sales analysis" IN7 of the business feasibility evaluation sheet.

[0064] FIG. 8H is a diagram showing an example of a question and an answer for the sheet item "cash flow analysis" according to this embodiment. Question Q8 is a question (prompt) generated to cause the language model 40 to answer the sheet item "y. cash flow analysis". Furthermore, the financial data (cash flow statement) of company A searched and acquired from the knowledge base 30 is embedded in the prompt for question Q8, and the question is input to the language model 40 as a question. Answer A7 is an answer received from the language model 40 as an answer to question Q8. The support server 20 inputs (transcribes) the graph of answer A8 received from the language model 40 into the sheet item "y. cash flow analysis" IN8 of the business feasibility evaluation sheet.

[0065] In the business feasibility evaluation sheet, for sheet items (answer input items) where graphs or other figures should be input, such as "break-even sales analysis" and "cash flow analysis," the language model 40 can be used to answer questions by providing figures, and the figures based on the answers can be input (transcribed) into the sheet items.

[0066] (Business feasibility evaluation sheet) Fig. 9 is a diagram showing a sheet item example 1 of a business feasibility evaluation sheet according to this embodiment. Fig. 10 is a diagram showing a sheet item example 2 of a business feasibility evaluation sheet according to this embodiment. The person in charge accesses the support server 20 using the terminal 50, acquires the business feasibility evaluation sheet created by the support server 20, and displays it on the screen.

[0067] Of the sheet items provided in advance in the format of the business feasibility evaluation sheet according to this embodiment, it can be seen that sentences and graphs based on the answers of the language model 40 are input into sheet items IN1-IN8.

[0068] Meanwhile, for example, the title of sheet item IN0 and "(1) Industry" of sheet item IN2-2 are input with information acquired / extracted from the customer information DB etc. of the financial institution system 10. Depending on the sheet item, there are also input items for which information can be simply acquired / extracted from various DBs or the knowledge base 30 etc. of the financial institution system 10. For such sheet items, the support server 20 does not need to generate questions and have the language model 40 answer them, which can improve the processing speed etc. up to the output of the business feasibility evaluation sheet.

[0069] Also, for example, sheet item IN10 "sales office opinion" is left blank. This is because some sheet items in the business feasibility evaluation sheet are desirable items that the person in charge has filled in themselves. The person in charge can complete the final business feasibility evaluation sheet by filling in the blank sheet items themselves in the business feasibility evaluation sheet that was primarily created by the support server 20. Of course, the person in charge may also make corrections to the text or graphs based on the answers of the language model 40 for sheet items IN1-IN8 in the primarily created business feasibility evaluation sheet before completing the final business feasibility evaluation sheet.

[0070] (Processing type) The processing types executed by the support server 20 according to this embodiment will be described below. The processing types shown below are categorized from the viewpoint of how the support server 20 acquires input information to be input into the input items of the examination work document. Note that which processing type is applied to each input item is determined in advance by the system administrator of the support server 20, etc., as the processing type that is optimal for each input item according to its subject matter, properties, etc.

[0071] ·Search and generation type (30 knowledge bases + 40 language models) Search and obtain question-related information similar to the generated question from the knowledge base 30, and embed the generated question, the question-related information searched and obtained from the knowledge base 30, and the prompt, and input them as a query to the language model 40. The search and generation type corresponds to the above-mentioned sheet items IN1-IN8, for example.

[0072] Extraction type (only various DBs and knowledge bases 30 of financial institution systems 10) Information is simply acquired and extracted from various DBs or the knowledge base 30 of the financial institution system 10. The extraction type corresponds to, for example, the title of sheet item IN0 or "(1) Industry" of sheet item IN2-2 described above.

[0073] ·Generative (Language Model 40 only) This type does not search or obtain question-related information similar to the generated question from the knowledge base 30, but inputs the generated question as an inquiry to the language model 40. This is applicable to input items for which a sufficient answer can be obtained from the language model 40 even if there is no question-related information.

[0074] <Summary> As described above, the creation of screening work documents such as a business feasibility evaluation sheet is an important process for screening companies, and it is necessary to properly evaluate the market environment, future prospects, management issues, etc. of the target company. According to the screening work document creation support system of this embodiment, by using an appropriately tuned language model and specialized knowledge base, it is possible to quickly and efficiently create detailed screening work documents such as industry trends, SWOT analysis, and management issues from the perspective of the management. In other words, the screening work document creation support system of this embodiment can support the creation of screening work documents by financial institution personnel.

[0075] Although the present invention has been described by showing specific examples according to the preferred embodiment of the present invention, it is apparent that various modifications and changes can be made to these examples without departing from the broad spirit and scope of the present invention as defined in the claims. In other words, the details of the examples and the accompanying drawings should not be construed as limiting the present invention.

[0076] The business feasibility evaluation sheet according to this embodiment is an example of an examination work document. The examination work document created by the support server 20 may be a loan approval document for loan examination. In addition, each financial institution may call the business feasibility evaluation sheet by various names such as "company overview", "examination table", "management issue sharing sheet", "hearing sheet", etc.

[0077] (Additional Note) An examination business document creation support device (examination business document creation support server 20) in a financial institution (bank, etc.), An item specifying means for specifying a predetermined input item (answer input item) among the input items provided in the format of the examination business document (business feasibility evaluation sheet) (item specifying unit 202, S2); A question generation unit (203, S3) that generates question information (questions) for causing a language model (fine-tuned large-scale language model 40) to answer the input information for the predetermined input items. a search unit (204, S4) for searching a knowledge base (knowledge base 30) in which customer relationship information (customer information DB, transaction data DB, financial data DB, negotiation record data DB, industry-specific audit dictionary information, corporate enterprise statistics information) is accumulated to search for question-related information related to the generated question information; a question sending means for sending the question information and the question-related information to the language model (question sending unit 205, S5); an answer receiving means for receiving answer information (answer) corresponding to the question information from the language model (answer receiving unit 206, S21); An answer input unit for inputting input information based on the answer information into the predetermined input item (answer input unit 207, S22); An examination document creation support device comprising: [Explanation of symbols]

[0078] 10 Financial Institutions System 20 Review document creation support server 30 Knowledge Base 40 Fine-tuned Large-Scale Language Models 50 terminals 70 Network 100 Examination Document Creation Support System 201 Document Creation Request Reception Unit 202 Item Specification Section 203 Question generation section 204 Search Department 205 Question Sending Department 206 Response Receiving Department 207 Answer input section 208 Document Output Unit 209 Memory section

Claims

1. A document preparation support device for examination operations in a financial institution, comprising: A question generation means for generating question information for causing a language model to respond to input information for input items provided in a format of an examination task document; a question sending means for sending the question information to the language model; an answer receiving means for receiving answer information corresponding to the question information from the language model; an answer input means for inputting input information based on the answer information into the input items; An examination document creation support device comprising:

2. A document preparation support device for financial institutions A question generation step of generating question information for causing a language model to respond to input information for input items provided in a format of a review business document; a question transmission step of transmitting the question information to the language model; an answer receiving step of receiving answer information corresponding to the question information from the language model; a response input step of inputting input information based on the response information into the input item; A method for supporting the preparation of review work documents to carry out the above.

3. Computer, A question generation means for generating question information for causing a language model to respond to input information for input items provided in a format of an examination task document; a question sending means for sending the question information to the language model; an answer receiving means for receiving answer information corresponding to the question information from the language model; an answer input means for inputting input information based on the answer information into the input items; This is a program to assist in the creation of review documents to make the system function as a review process.

Images