Examination work document creation support apparatus, examination work document creation support method, and examination work document creation support program

Abstract

To further enrich contents or to improve accuracy in the preparation of an examination task document.SOLUTION: An examination business document creation support apparatus in a financial institution according to the present invention includes UI providing means for providing an interactive UI screen, question input means for inputting a question sentence related to an input item provided in a format of an examination business document via the UI screen, question generation means for generating question information for causing a language model to answer the question sentence, question transmission means for transmitting the question information to the language model, answer reception means for receiving answer information corresponding to the question information from the language model, and answer output means for outputting an answer sentence corresponding to the question sentence on the basis of the answer information via the UI screen. This system is provided with an instruction input means for inputting the input instruction of an answer sentence to an input item and an answer input means for inputting input information based on the answer sentence to the input item corresponding to the input instruction.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] The present invention relates to an examination business document creation support device, an examination business document creation support method, and an examination business document creation support program. [Background technology]

[0002] A business feasibility assessment is a process by which financial institutions evaluate a client company when deciding whether to provide financing. The assessment takes into account not only quantitative information such as the company's financial statements, but also qualitative information such as industry trends, the company's strengths and weaknesses, future vision, and management challenges. Typically, when conducting a business feasibility assessment, financial institutions prepare a document known as a "business feasibility assessment sheet" for internal communication. The business feasibility assessment sheet is a uniquely designed format designed to enable financial institutions to appropriately conduct business evaluations. Branch staff fill out the sheet by collecting quantitative information such as corporate information and financial statements for the company they are responsible for, and compiling qualitative information based on interviews with management regarding industry trends, the company's strengths and weaknesses, future vision, and management challenges. The completed business feasibility assessment sheet is primarily used, for example, as an attachment when drafting a loan request.

[0003] As an example of related technology, Patent Document 1 describes a device for supporting the creation of screening work documents in financial institutions. By using an appropriately tuned language model and specialized knowledge base, financial institution personnel can quickly and efficiently create detailed screening work documents that include information on industry trends, SWOT analysis, and management issues from the perspective of management. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7396582 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the language model and knowledge base described in Patent Document 1 above have the problem that the content of the generated AI answers may sometimes be insufficient or inaccurate, or may not match the perspective desired by financial institution personnel.

[0006] The present invention has been proposed in view of the above points, and in one aspect, aims to further enrich the content and improve the accuracy in the preparation of examination work documents. [Means for solving the problem]

[0007] In order to solve the above problems, the screening work document creation support device of the present invention is a screening work document creation support device for a financial institution, and comprises a UI providing means for providing an interactive UI screen, a question input means for inputting a question regarding an input field provided in the format of a screening work document via the UI screen, a question generating means for generating question information for a language model to answer the question, a question sending means for sending the question information to the language model, an answer receiving means for receiving answer information corresponding to the question information from the language model, an answer output means for outputting an answer corresponding to the question based on the answer information via the UI screen, an instruction input means for inputting an input instruction for the answer text for the input field via the UI screen, and an answer input means for inputting input information based on the answer text into the input field in accordance with the input instruction. [Effects of the Invention]

[0008] According to one aspect of the embodiment of the present invention, it is possible to further enhance the content and improve the accuracy of the preparation of examination work documents. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram showing an example of the configuration of an examination work document creation support system 100 according to the present embodiment. [Figure 2]FIG. 2 is a diagram showing an example of knowledge data in a knowledge base 30 according to the present embodiment. [Figure 3] FIG. 2 is a diagram showing an example of labeled training data for a fine-tuned large-scale language model 40 according to the present embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a functional configuration of an examination business document creation support server 20 according to the present embodiment. [Figure 5] 1 is a diagram showing an outline of the operation of an examination work document preparation support system 100 according to the present embodiment. [Figure 6] FIG. 1 is a diagram showing an examination business document creation process 1 according to the present embodiment. [Figure 7] FIG. 10 is a diagram showing an examination business document creation process 2 according to the present embodiment. [Figure 8A] FIG. 10 is a diagram showing example questions and answers for the sheet item "1. Corporate attribute information" according to the present embodiment. [Figure 8B] FIG. 10 is a diagram showing example questions and answers for the sheet item "2. Business content" according to the present embodiment. [Figure 8C] 10 is a diagram showing example questions and answers for the sheet item "3. Shareholder / Investor Information" according to the present embodiment. FIG. [Figure 8D] FIG. 10 is a diagram showing example questions and answers for the sheet item "4. Management Information" according to the present embodiment. [Figure 8E] FIG. 10 is a diagram showing example questions and answers for the sheet item "5. Industry trends" according to the present embodiment. [Figure 8F] FIG. 10 is a diagram showing example questions and answers for the sheet item "6. SWOT analysis" according to the present embodiment. [Figure 8G] FIG. 10 is a diagram showing example questions and answers for the sheet item "Break-even sales analysis" according to the present embodiment. [Figure 8H] FIG. 10 is a diagram showing example questions and answers for the sheet item "cash flow analysis" according to the present embodiment. [Figure 9] FIG. 1 is a diagram showing an example 1 of sheet items of a business feasibility evaluation sheet according to the present embodiment. [Figure 10] FIG. 10 is a diagram showing an example 2 of sheet items of a business feasibility evaluation sheet according to this embodiment. [Figure 11] 1 is a diagram showing an example of the configuration of an examination work document creation support system 100 according to the present embodiment. [Figure 12] 2 is a diagram illustrating an example of the functional configuration of an examination business document creation support server 20 according to the present embodiment. FIG. [Figure 13] FIG. 10 is a diagram showing an example of a UI screen for a sheet item “2. Business content” according to the present embodiment. [Figure 14A] FIG. 10 is a diagram showing a UI screen example 1 for a sheet item "6. SWOT analysis" according to the present embodiment. [Figure 14B] FIG. 10 is a diagram showing a UI screen example 2 for the sheet item "6. SWOT analysis" according to the present embodiment. [Figure 15] FIG. 10 is a diagram showing a UI screen example 3 for the sheet item "6. SWOT analysis" according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to the drawings. [Embodiment 1] <System configuration> Fig. 1 is a diagram showing an example of the configuration of a screening work document preparation support system 100 according to this embodiment. The screening work document preparation support system 100 in Fig. 1 includes a financial institution system 10, a screening work document preparation support server 20, a knowledge base (vector DB) 30, a fine-tuned large-scale language model (LLM) 40, and a terminal 50, all of which are connected via a network 70.

[0011] The financial institution system 10 is made up of various systems and databases (DBs) owned by a financial institution such as a bank. The financial institution system 10 includes, for example, a core system that manages transactions such as various deposits and loans, a sales management system that manages sales performance of financial products, a front compliance system that appropriately manages transaction details in accordance with laws and regulations, an SFM (Sales Force Management System) and CRM (Customer Relationship Management) that manage customer information including loan history and negotiation records with customers (sales records), and each DB associated with each system. Each DB of the financial institution system 10 includes, for example, DBs for customer information, transaction data, financial data, negotiation record data, etc. of client companies of the financial institution.

[0012] The screening business document creation support server (hereinafter simply referred to as the support server) 20 is a server device that supports the creation of screening business documents to be created by financial institution personnel. When the support server 20 receives a request to create a business feasibility evaluation sheet, for example, as a screening business document from the terminal 50 of the personnel, the support server 20 creates a business feasibility evaluation sheet by asking questions to a fine-tuned large-scale language model (hereinafter simply referred to as the language model) 40 about specified input items (hereinafter also referred to as sheet items) in the business feasibility evaluation sheet, and filling in the sheet items based on the answers obtained. A specific creation example will be described later.

[0013] The knowledge base 30 is a knowledge database in which information about client companies (referred to as customer relationship information) of a financial institution is pre-stored and constructed, for example, by vector conversion. The customer relationship information includes not only publicly available information about client companies but also private information such as internal bank information. Because the language model 40 cannot answer questions about content other than that contained in the training data, even when attempting to answer a specific question about a client company, it may receive a response that the question is unknown or an answer based on different knowledge. To solve this problem, in this embodiment, customer relationship information likely to be necessary to answer the question is pre-stored (stored) in the knowledge base 30. The assistance server 20 first searches the customer relationship information stored in the knowledge base 30 for question-related information likely to be necessary to answer the question, and then passes the searched question-related information to the language model 40 along with a prompt, which is an instruction / input sentence. This allows the language model 40 to come up with an appropriate answer even for content that it has not learned.

[0014] The fine-tuned large-scale language model 40 is a language model that further performs supervised learning of specific domain knowledge on a general-purpose natural language processing model (LLM), which has previously undergone unsupervised pre-training of grammar, word meanings, etc. using large amounts of text data. By fine-tuning the general-purpose natural language processing model to learn a set of appropriate "questions" and desirable "answers," the language model 40 is capable of generating highly specialized answer sentences required for review documents. Examples of general-purpose natural language processing models that can be used include Bidirectional Encoder Representations from Transformers (BERT), Generative Pre-trained Transformer (GPT)-3.5, and GPT-4. Instead of using a network system configuration in which the language model 40 is stored in a storage device within the bank's network 70, the language model 40 can also be read by accessing a fine-tuned large-scale language model server provided as an external service.

[0015] The terminal 50 is, for example, a PC (personal computer), smartphone, tablet terminal, etc., and is a user terminal used, for example, by a sales office staff member. A predetermined application program and a general-purpose web browser for accessing the support server 20 are pre-installed on the terminal 50. The staff member uses the terminal 50 to access the support server 20, obtain the business feasibility evaluation sheet created by the support server 20, and display it on the screen. Note that, since some of the sheet items in the business feasibility evaluation sheet include desired items that the staff member has filled in themselves, the support server 20 does not necessarily complete the business feasibility evaluation sheet by filling in all of the sheet items. The staff member can complete the final business feasibility evaluation sheet by reviewing and supplementing the business feasibility evaluation sheet initially created by the support server 20.

[0016] The network 70 is a communication network of the financial institution, and is preferably an in-house network from the viewpoint of security. However, the network 70 may include, for example, the Internet, a public line network, Wi-Fi (registered trademark), etc., provided that sufficient security is ensured.

[0017] (Knowledge Base 30) FIG. 2 is a diagram showing an example of knowledge data in the knowledge base 30 according to this embodiment. The knowledge base 30 is a knowledge database in which customer relationship information in a financial institution is stored and constructed in advance after, for example, vector conversion to make it easier to search. The knowledge data in the knowledge base 30 according to this embodiment includes customer relationship information for client companies in the financial institution, such as customer information, transaction data, financial data, and negotiation record data, for example, Company A, Company B, Company C, etc. Furthermore, the knowledge data in the knowledge base 30 according to this embodiment includes, for example, industry-specific audit dictionary information and corporate enterprise statistics information, which are stored as external reference materials.

[0018] Customer information is information about client companies that do business with financial institutions. It includes general company information such as trade name or name, address, telephone number, and representative name, as well as information required to be submitted when opening an account, such as industry and business type. Customer information can be obtained, for example, from the core system of the financial institution system 10. Customer information can be further supplemented and enhanced with information such as a message from the president, the company's strengths, future vision, business details, products and services, locations, sales area, history, major business partners, and shareholders, based on information collected, for example, from the client company's own website via scraping or from corporate information databases provided by external credit investigation companies. The business feasibility assessment sheet includes sheet items such as "corporate attribute information," "business details," "shareholder and investor information," "management information," "industry trends," and "SWOT analysis." Customer information can be used as one of the question-related information when responding to these questions using the language model 40.

[0019] The transaction data includes, for example, information on each account held by each customer, such as a savings account, current account, fixed-term deposit account, and foreign currency deposit account, credit card information, transaction details including foreign exchange transactions (liquidity transaction details information), and information on loans made in the past. All transaction details from the present to the past are recorded. The transaction data can be obtained, for example, from the core system of the financial institution system 10. The business feasibility assessment sheet includes, for example, a sheet item called "corporate attribute information," and the transaction data can be used as one piece of question-related information for these questions when the language model 40 responds.

[0020] The financial data includes not only financial statements (balance sheets, income statements, cash flow statements, etc.) for each business year of the client company, but also data related to the client company's finances, such as trial balance data for the period showing the progress of profits and losses for the year. The financial data can be obtained, for example, from the core system of financial institution system 10. The business feasibility assessment sheet includes sheet items such as "shareholder / investor information," "management information," "industry trends," "SWOT analysis," "break-even sales analysis," and "cash flow analysis," and the financial data can be used as one piece of question-related information for these questions when the language model 40 responds.

[0021] Negotiation record data is a record of past negotiations between financial institutions, such as branch staff, and client companies, such as management and finance department staff. It includes not only documents and emails exchanged between the two parties, but also conversations (such as the management's vision for the future, management challenges, and succession issues) held during face-to-face meetings (visits and branch visits), online conferences, and phone calls, as long as they are recorded in the form of text, audio, images, or video. Negotiation record data can be acquired, for example, from SFM or CRM, which manages customer information including loan history and customer negotiation records (sales records) in the financial institution system 10. A business feasibility assessment sheet includes, for example, sheet items such as "industry trends" and "SWOT analysis." The negotiation record data can be used as a reference for answering questions related to these items using the language model 40.

[0022] The industry-specific screening dictionary information covers all industries and business types in Japan, and is information from an industry information dictionary that compiles industry trends, information analysis, screening points, etc. for each industry. Financial institution personnel refer to this information when conducting loan screening to understand the industry of the borrower. A business feasibility assessment sheet may contain items such as "industry details," "industry trends," and "SWOT analysis," and the industry-specific screening dictionary information can be used as a reference when responding to these questions using language model 40. This is expected to enable language model 40 to provide answers that are more in line with the perspective of a financial institution's loan screening.

[0023] The Financial Statements Statistics of Corporations information is a fundamental statistical survey conducted as a sample survey based on the Statistics Act to understand the actual state of business activities of for-profit corporations and other entities in Japan. The survey results include financial indicators such as sales, capital investment, and current profits of for-profit corporations and other entities, as well as the status of business activities by industry, size, and region. The business feasibility assessment sheet includes, for example, items such as "industry trends" and "SWOT analysis," and the Financial Statements Statistics of Corporations information can be used as reference when responding to these questions using language model 40. This is expected to enable language model 40 to provide answers that are more in line with the perspective of financial institutions' loan screening.

[0024] (40 fine-tuned large-scale language models) 3 is a diagram showing an example of labeled training data for the fine-tuned large-scale language model 40 according to this embodiment. As described above, the fine-tuned large-scale language model 40 is a language model that further performs supervised learning of specific domain knowledge on a general-purpose natural language processing model (LLM), which has been pre-trained in an unsupervised manner using a large amount of text data to learn grammar, word meanings, and the like. By training (fine-tuning) a sentence set of "questions" and exemplary "answers" to elicit appropriate answers to the sheet items on a business feasibility assessment sheet, a model capable of generating highly specialized answer sentences required for appraisal work documents is created.

[0025] For example, the business feasibility evaluation sheet has a sheet item called "History." As a sample set of questions and model answers to obtain appropriate answers to this sheet item "History," multiple sets of "Question: Please summarize the history of Company A in chronological order using XXX characters" and "Answer: In 2020, XXX founded Company A in XX ward, Tokyo. In 2021..." (learning data a-1)... are trained (fine-tuned) in advance.

[0026] Furthermore, the business feasibility assessment sheet includes a sheet item called "SWOT analysis," and as a sample set of questions and model answers to obtain appropriate answers to this sheet item "SWOT analysis," the following are provided: "Question: You are a banker. As a banker conducting a loan review, please conduct a SWOT analysis of Company A from the perspective of the strengths and weaknesses of the internal environment and the opportunities and threats in the external environment. The analysis should take into account not only the company's financial data, but also the future vision and management issues as envisioned by the management." and "Answer: S (Strengths): ··, W (Weaknesses): ··, O (Opportunities): ··, T (Threats): ··" (learning data a-6). Multiple sets of ·· are trained in advance (fine-tuned).

[0027] Furthermore, for a client company for which the support server 20 has previously output a business feasibility assessment sheet via the processes of S1 to S5 in Fig. 6 and S21 to S23 in Fig. 7, which will be described later, the language model 40 may be trained (fine-tuned) using the questions (S3) and question-related information (S5) generated for each sheet item in the past, and the answers (S22) in the language model 40 entered corresponding to the sheet items. Furthermore, if the answers (S22) in the language model 40 have been further revised by the person in charge to provide more desirable answers, the revised answers obtained from the final business feasibility assessment sheet may be trained instead of the answers (S22) in the language model 40.

[0028] (Functional configuration) 4 is a diagram showing an example of the functional configuration of the review work document creation support server 20 according to this embodiment. The support server 20 has, as main functional units, a document creation request receiving unit 201, an item specifying unit 202, a question generating unit 203, a search unit 204, a question sending unit 205, an answer receiving unit 206, an answer input unit 207, a document output unit 208, and a memory unit 209.

[0029] The document creation request receiving unit 201 has a function of receiving, from the terminal 50 of the person in charge, a request to create an examination business document (for example, a business feasibility evaluation sheet) specifying the client company for which the document is to be created.

[0030] The item specification unit 202 has a function of specifying a predetermined answer input item according to the type of the review business document from among the input items provided in the format of the review business document.

[0031] The question generation unit 203 has a function of generating question information for causing the language model 40 to respond to input information for a predetermined answer input item.

[0032] The search unit 204 has a function of searching for question-related information related to the generated question information from a knowledge base in which customer-related information about customers is accumulated.

[0033] The question sending unit 205 has a function of sending question information and customer relationship information to the language model 40 .

[0034] The answer receiving unit 206 has a function of receiving answer information corresponding to question information from the language model 40 .

[0035] The answer input unit 207 has a function of inputting answer information into predetermined answer input items.

[0036] The document output unit 208 has the function of outputting the review business document (e.g., a business feasibility evaluation sheet) to the terminal 50 once answer information based on answers from the language model 40 has been entered into all specified answer input items in the review business document.

[0037] The storage unit 209 has a function of storing information such as the format of the examination work document and standard questions for each sheet item in a storage device.

[0038] The review work document preparation support server 20 can be implemented using a general-purpose computer. Specifically, the support server 20 includes hardware such as a central processing unit (CPU), memory, an input / output interface, and a communication interface. The functions of the support server 20 are realized by the processing unit executing processing in accordance with a computer program stored in memory. That is, each functional unit is realized by a computer program executed on the hardware resources, such as the processing unit and memory, of the computer constituting the support server 20. The review work document preparation support server 20 may also be referred to as a review work document preparation support device or a computing machine for supporting review work document preparation. These functional units may also be referred to as "means," "module," "unit," or "circuit." Each database may be located in the memory of the support server 20 or in an external storage device on the network 70. Each functional unit of the support server 20 may not only be implemented by a single server device, but may also be implemented as a system consisting of multiple devices with distributed functions. The computer program may also be stored on a computer-readable storage medium.

[0039] <Support for creating review documents> Fig. 5 is a diagram showing an outline of the operation of the review work document creation support system 100 according to this embodiment. Fig. 6 is a diagram showing the review work document creation process 1 according to this embodiment. The arithmetic processing unit of the support server 20 reads and executes a program capable of realizing the process, thereby realizing each of the following steps (hereinafter referred to as "S").

[0040] S1: The support server 20 receives a request to create an examination work document (e.g., a business feasibility evaluation sheet) specifying the client company for which the document is to be created, from the person in charge's terminal 50. The creation request includes at least an identifier (e.g., a client number, a client company name, etc.) for identifying the target client company.

[0041] S2: The support server 20 identifies sheet items (predetermined answer input items) to be answered by the language model 40 from among the sheet items (input items) predefined in the format of the review business document stored in the memory unit 209. Some sheet items in the review business document sheet are desired items that the person in charge fills in themselves, so the sheet items to be filled in by the support server 20 are predetermined according to the type of review business document. In the case of a business feasibility evaluation sheet, for example, the sheet items to be answered by the language model 40 can be identified as "corporate attribute information," "business content," "shareholder / investor information," "management information," "industry trends," "SWOT analysis," "break-even sales analysis," and "cash flow analysis."

[0042] S3: The support server 20 generates a question (inquiry) for answering the sheet item identified in S2 for each sheet item. The sentences of the question (inquiry) generated for each sheet item are determined for each sheet item and stored in a fixed format in the storage unit 209. In addition, in order to obtain a model answer from the language model 40, the question sentences are in the same format as the sample questions for obtaining an appropriate answer for the sheet item in question in the fine-tuning situation (FIG. 3).

[0043] Specifically, the support server 20 generates a prompt such as "Summarize the history of Company A in chronological order in XXX characters" as a question to answer the question in "History" in one sheet item "Corporate Attribute Information" identified in S2. Also, the support server 20 generates a prompt such as "You are a banker. As a banker conducting a loan screening, please conduct a SWOT analysis of Company A from the perspective of the strengths and weaknesses of the internal environment and the opportunities and threats in the external environment. The analysis should take into account not only the company's financial data but also the future vision and management issues as envisioned by the management." as a question to answer the question in one sheet item "SWOT analysis" identified in S2.

[0044] In addition, questions to answer all the sheet items identified in S2 can be created as one question, such as, "To create a business feasibility evaluation sheet for Company A, please summarize the following in a maximum of XXX characters: 1. Corporate attribute information, 2. Business content, 3. Shareholder / investor information, 4. Management information, 5. Industry trends, 6. SWOT analysis..."

[0045] S4: The support server 20 searches and acquires question-related information similar to the question generated in S3 from the knowledge base 30. Specifically, the support server 20 converts the question text generated in S3 directly into a vector as a query, searches the knowledge base 30 (knowledge vector DB), and acquires similar text with a close vector distance as question-related information.

[0046] Specifically, as a similar sentence to the question generated in S3, "Summarize the history of Company A in chronological order using XXX characters," for example, company information about Company A (≒ "Company A" and "History") can be obtained as question-related information. In addition, for the question generated in S3, "You are a banker. As a banker conducting a loan screening, please conduct a SWOT analysis of Company A from the perspective of the strengths and weaknesses of the internal environment and the opportunities and threats in the external environment. Your analysis should take into account not only the company's financial data but also the future vision and management issues as envisioned by the management," it is possible to obtain, as question-related information, the following: Company A's corporate information (≒ "Company A," "management," "strengths and weaknesses," "future vision"), Company A's financial data (≒ "Company A," "financial data," Company A's negotiation record data (≒ "Company A," "strengths and weaknesses," "future vision," "management issues"), screening dictionary information corresponding to Company A's industry (≒ "Company A," "loan screening," "banker"), and corporate statistics information for Company A's industry (≒ "external environment").

[0047] In addition, in order to answer the question with a SWOT analysis based on the perspective of a financial institution, which is unique to screening work documents, the question can include words such as "loan screening" or "banker," making it possible to search for similar sentences in the screening dictionary information corresponding to Company A's industry. As mentioned above, the industry-specific screening dictionary information covers all industries and business types in Japan, and is an industry information dictionary that summarizes industry trends, information analysis, screening key points, etc. for each industry. It is used by financial institution personnel to understand the industry of the borrower during loan screening. Therefore, when searching and retrieving screening dictionary information as a knowledge base, it is extremely important to identify the industry to which Company A belongs. When the support server 20 searches and retrieves question-related information similar to the question (question) generated in S3 from the knowledge base 30, by explicitly entering Company A's industry in the query, it is possible to reliably search and retrieve screening dictionary information corresponding to Company A's industry as similar sentences.

[0048] In this case, it is advisable for the support server 20 to identify the industry of Company A in advance at least before S4. A specific identification method can be, for example, acquisition from the core system of the financial institution system 10, or if acquisition from the core system is not possible or is difficult, acquisition can also be based on information collected by scraping or the like from the client company's own website or from a company information database provided by an external credit investigation company or the like.

[0049] S5: The support server 20 transmits the question generated in S3 and the question-related information searched and acquired in S4 to the language model 40. Specifically, the support server 20 embeds the generated question and the question-related information searched and acquired from the knowledge base 30 in a prompt and inputs them to the language model 40 as an inquiry.

[0050] FIG. 7 is a diagram showing the examination business document creation process 2 according to this embodiment. S21: The support server 20 receives an answer from the language model 40 as a response to the inquiry in S5. For example, as an answer to the question "Summarize the history of Company A in chronological order using XXX characters" and the question-related information "Company information for Company A," the answer text "2020: XX founded Company A in XX ward, Tokyo. In 2021, the headquarters moved to XX ward, Tokyo..." can be received. Note that the answer from the language model 40 is an answer text in the same format as a desirable sample answer based on fine tuning (Figure 3).

[0051] S22: The support server 20 inputs the answer received in S21 into the sheet item (predetermined answer input item) identified in S2 in the review business document. For example, into one sheet item "History" identified in S2, the answer text "In 2020, XX founded company A in XX ward, Tokyo. In 2021, the head office moved to XX ward, Tokyo..." is input. The answer text may be directly transcribed into the sheet item, or the format may be modified according to the form of the sheet item.

[0052] S23: When the support server 20 has finished inputting all sheet items in the document that should be answered by the language model 40, it outputs the examination work document to the terminal 50. In the business feasibility evaluation sheet according to this embodiment, the sheet items that the language model 40 should answer are, for example, "corporate attribute information," "business content," "shareholder / investor information," "management information," "industry trends," "SWOT analysis," "break-even sales analysis," and "cash flow analysis." Therefore, when all answers from the language model 40 corresponding to these sheet items have been inputted into the business feasibility evaluation sheet, the support server 20 outputs the business feasibility evaluation sheet for the designated client company to the terminal 50.

[0053] <Examples of questions and answers for sheet items> Among the sheet items provided in the format of the business feasibility evaluation sheet, examples of sheet items that the support server 20 should fill in using the answers from the language model 40 include "corporate attribute information," "business content," "shareholder / investor information," "management information," "industry trends," "SWOT analysis," "break-even sales analysis," and "cash flow analysis," and specific examples are given below.

[0054] FIG. 8A is a diagram showing an example of a question and answer for the sheet item "1. Corporate Attribute Information" according to this embodiment. Question Q1 is a question (prompt) generated to prompt the language model 40 to answer the sheet item "1. Corporate Attribute Information." Question Q1 is input to the language model 40 as a question, with the corporate information of Company A searched and acquired from the knowledge base 30 and its transaction data embedded in the prompt. Answer A1 is an answer received from the language model 40 as an answer to question Q1. The support server 20 inputs (transcribes) the text of answer A1 received from the language model 40 into sheet item "1. Corporate Attribute Information" IN1 of the business feasibility assessment sheet.

[0055] FIG. 8B is a diagram showing an example question and answer for the sheet item "2. Business Content" according to this embodiment. Question Q2 is a question (prompt) generated to prompt the language model 40 to answer the sheet item "2. Business Content." Question Q2 is input to the language model 40 as a question, with company information about Company A searched and acquired from the knowledge base 30 and examination dictionary information for Company A's industry embedded in the prompt. Answer A2 is an answer received from the language model 40 as an answer to question Q2. The support server 20 inputs (transcribes) the text of answer A2 received from the language model 40 into sheet item "2. Business Content" IN2 of the business feasibility assessment sheet.

[0056] It should be noted that the sheet item "2. Business Content" IN2 includes an item "(1) Industry", but since the "(1) Industry" of the client company can be obtained and entered from the customer information DB, etc., held by the financial institution system 10 without having to have the language model 40 answer the question, in this embodiment, question Q2 does not include any questions regarding "(1) Industry".

[0057] FIG. 8C is a diagram showing an example of a question and answer for the sheet item "3. Shareholder and Investor Information" according to this embodiment. Question Q3 is a question (prompt) generated to prompt the language model 40 to answer the sheet item "3. Shareholder and Investor Information." Question Q3 is input to the language model 40 as a question, with the corporate information of Company A and financial data of Company A searched and acquired from the knowledge base 30 embedded in the prompt. Answer A3 is an answer received from the language model 40 as an answer to question Q3. The support server 20 inputs (transcribes) the text of answer A3 received from the language model 40 into sheet item "3. Shareholder and Investor Information" IN3 of the business feasibility assessment sheet.

[0058] FIG. 8D is a diagram showing an example question and answer for the sheet item "4. Management Information" according to this embodiment. Question Q4 is a question (prompt) generated to prompt the language model 40 to answer the sheet item "4. Management Information." Question Q4 is input to the language model 40 as a question, with the corporate information of Company A and financial data of Company A searched and acquired from the knowledge base 30 embedded in the prompt. Answer A4 is an answer received from the language model 40 as an answer to question Q4. The support server 20 inputs (transcribes) the text of answer A4 received from the language model 40 into sheet item "4. Management Information" IN4 of the business feasibility assessment sheet.

[0059] FIG. 8E is a diagram showing an example question and answer for the sheet item "5. Industry Trends" according to this embodiment. Question Q5 is a question (prompt) generated to prompt the language model 40 to answer the sheet item "5. Industry Trends." Question Q5 is input to the language model 40 as a question, with the corporate information of Company A, financial data of Company A, audit dictionary information for Company A's industry, and corporate statistics information for Company A's industry searched and acquired from the knowledge base 30 embedded in the prompt. Answer A5 is an answer received from the language model 40 as an answer to question Q5. The support server 20 inputs (transcribes) the text of answer A5 received from the language model 40 into sheet item "5. Industry Trends" IN5 of the business feasibility assessment sheet.

[0060] FIG. 8F is a diagram showing an example of a question and answer for the sheet item "6. SWOT analysis" according to this embodiment. Question Q6 is a question (prompt) generated to prompt the language model 40 to answer the sheet item "6. SWOT analysis." Furthermore, question Q6 is input to the language model 40 as a question, with the corporate information of Company A, financial data of Company A, negotiation record data of Company A, audit dictionary information for Company A's industry, and corporate statistical information for Company A's industry searched and acquired from the knowledge base 30 embedded in the prompt. Answer A6 is a response received from the language model 40 as a response to question Q6. The support server 20 inputs (transcribes) the text of answer A6 received from the language model 40 into sheet item "6. SWOT analysis" IN6 of the business feasibility assessment sheet.

[0061] For items such as SWOT analysis in a business feasibility assessment sheet, quantitative information such as general information about the client company's industry or business sector or financial data is insufficient; qualitative information such as the client company's daily thoughts and perspective is essential for the analysis. For this reason, when creating a business feasibility assessment sheet, branch staff would previously fill in the sheet items by collecting quantitative information such as corporate information and financial statements from the company they are responsible for, and analyzing and summarizing qualitative information based on interviews with the client company's management.

[0062] For example, answer A6 in this embodiment is a response to question Q6 in which the prompt contains the negotiation record data of Company A retrieved from the knowledge base 30. The response from the language model 40 is based on the information recorded in the negotiation record data. As described above, the negotiation record data is a record history of past negotiations between financial institutions, such as branch staff, and client companies, such as management and finance department staff, originally retrieved from SFM or CRM. It includes not only documents and emails exchanged between the two parties, but also conversations (such as the management's vision for the future, management challenges, and succession issues) held during face-to-face meetings (visits, branch visits), online conferences, and phone calls, to the extent that they are recorded. Therefore, for example, the sentences in answer A6, such as "The management aims to achieve the top share in the industry within three years, and it is expected that this will be achieved with a high probability," and "Improving development capabilities is a management challenge," reflect the goal (achieving the top share in the industry within three years), the likelihood of achieving this (high), and the management challenge (improving development capabilities) stated by Company A's management, which were recorded in the negotiation record history.

[0063] FIG. 8G is a diagram showing an example of a question and an answer for the sheet item "Break-Even Sales Analysis" according to this embodiment. Question Q7 is a question (prompt) generated to prompt the language model 40 to answer the sheet item "x. Break-Even Sales Analysis." Furthermore, the question Q7 is input as a question to the language model 40 with the financial data (profit and loss statement) of Company A searched and acquired from the knowledge base 30 embedded in the prompt. Answer A7 is an answer received from the language model 40 as an answer to question Q7. The support server 20 inputs (transfers) the graph of answer A7 received from the language model 40 into the sheet item "x. Break-Even Sales Analysis" IN7 of the business feasibility assessment sheet.

[0064] FIG. 8H is a diagram showing an example of a question and an answer for the sheet item "cash flow analysis" according to this embodiment. Question Q8 is a question (prompt) generated to prompt the language model 40 to answer the sheet item "y. cash flow analysis." Furthermore, the question Q8 is input as a question to the language model 40 with the financial data (cash flow statement) of Company A searched and acquired from the knowledge base 30 embedded in the prompt. Answer A7 is an answer received from the language model 40 as an answer to question Q8. The support server 20 inputs (transcribes) the graph of answer A8 received from the language model 40 into sheet item "y. cash flow analysis" IN8 of the business feasibility assessment sheet.

[0065] In the business feasibility evaluation sheet, for sheet items (answer input items) that require the input of graphs or other figures, such as "Break-even sales analysis" and "Cash flow analysis," the language model 40 can be used to generate figures based on questions, and the figures based on the answers can be input (transcribed) into the sheet items.

[0066] (Business feasibility assessment sheet) Fig. 9 is a diagram showing an example of sheet items for a business feasibility assessment sheet according to this embodiment. Fig. 10 is a diagram showing an example of sheet items for a business feasibility assessment sheet according to this embodiment. The person in charge accesses the support server 20 using the terminal 50, acquires the business feasibility assessment sheet created by the support server 20, and displays it on the screen.

[0067] Of the sheet items provided in advance in the format of the business feasibility evaluation sheet according to this embodiment, it can be seen that sentences and graphs based on the answers of the language model 40 are input into sheet items IN1 to IN8.

[0068] On the other hand, for example, the title of sheet item IN0 and "(1) Industry" of sheet item IN2-2 are input with information acquired or extracted from the customer information DB etc. of the financial institution system 10. Depending on the sheet item, there are also input items for which information can be simply acquired or extracted from various DBs or the knowledge base 30 etc. of the financial institution system 10. For such sheet items, the support server 20 does not need to generate questions and have the language model 40 answer them, which can improve the processing speed etc. until the business feasibility evaluation sheet is output.

[0069] Also, for example, sheet item IN10, "Sales Office Opinion," is left blank. This is because some sheet items in the business feasibility evaluation sheet are desirable items that the person in charge has filled in themselves. The person in charge can complete the final business feasibility evaluation sheet by filling in the blank sheet items themselves in the business feasibility evaluation sheet that was initially created by the support server 20. Of course, the person in charge may also make corrections to the sentences or graphs based on the answers of the language model 40 for sheet items IN1-IN8 in the business feasibility evaluation sheet that was initially created, before completing the final business feasibility evaluation sheet.

[0070] (Processing type) The following describes the processing types executed by the support server 20 according to this embodiment. The processing types shown below are categorized from the perspective of how the support server 20 acquires input information to be input into the input fields of the review work document. The processing type to be applied to each input field is determined in advance by the system administrator of the support server 20, etc., as the processing type that is optimal for each input field depending on the content, characteristics, etc. of the input field.

[0071] · Search · Generation type (30 knowledge bases + 40 language models) Search and acquire question-related information similar to the generated question from the knowledge base 30, and then embed the generated question, the question-related information searched and acquired from the knowledge base 30, and the prompt into the language model 40 as an inquiry. The search and generation type corresponds to the sheet items IN1-IN8 described above, for example.

[0072] Extraction type (only various DBs and knowledge bases 30 of financial institution systems 10) Information is simply acquired and extracted from various DBs or the knowledge base 30 of the financial institution system 10. The extraction type corresponds to, for example, the title of sheet item IN0 or "(1) Industry" of sheet item IN2-2 described above.

[0073] Generative (Language Model 40 only) This type does not search or obtain question-related information similar to the generated question from the knowledge base 30, but inputs the generated question as an inquiry into the language model 40. This is applicable to input items for which a sufficient answer can be obtained from the language model 40 even if there is no question-related information.

[0074] <Summary> As described above, the creation of screening work documents such as business feasibility evaluation sheets is an important process for screening companies, and it is necessary to properly evaluate the target company's market environment, future prospects, management issues, etc. The screening work document creation support system according to this embodiment uses an appropriately tuned language model and specialized knowledge base to quickly and efficiently create detailed screening work documents, including information on industry trends, SWOT analysis, and management issues from the perspective of management. In other words, the screening work document creation support system according to this embodiment can support financial institution personnel in creating screening work documents.

[0075] [Embodiment 2] In the above-described first embodiment, answers from the language model 40 (so-called generative AI) are input into the sheet items of the business feasibility assessment sheet, thereby enabling the creation of the business feasibility assessment sheet (input of answers to sheet items) to be made quick and efficient. In the second embodiment, in the creation of the business feasibility assessment sheet, repeated dialogue between the AI ​​and a person (e.g., a financial institution employee) is carried out through an interactive user interface, thereby further reflecting human knowledge and insight in the AI's answers, thereby further enriching the content of the business feasibility assessment sheet (sheet items) and improving their accuracy. This will be explained in detail below.

[0076] <System configuration> Fig. 11 is a diagram showing an example of the configuration of a screening work document preparation support system 100 according to this embodiment. Similar to Fig. 1, the screening work document preparation support system 100 in Fig. 11 includes a financial institution system 10, a screening work document preparation support server 20, a knowledge base 30, a fine-tuned large-scale language model (LLM) 40, and a terminal 50, all of which are connected via a network 70.

[0077] The support server 20 according to this embodiment provides an interactive UI screen to the terminal 50 of the person in charge when creating a business feasibility evaluation sheet. Through the interactive UI screen on the terminal 50, the person in charge goes through a process of repeated dialogue with the AI ​​(inputting questions based on the person's knowledge and insight, and the AI ​​outputting answers), and can check the content of the AI's answers obtained for each sheet item one by one, and enter answers that the person in charge is satisfied with into the corresponding sheet item. The person in charge can also correct the AI's answers obtained for each sheet item based on the person's knowledge and insight, and enter the corrected answers into the sheet item.

[0078] Furthermore, if there is sufficient customer relationship information in the knowledge base 30, the language model 40 can provide an appropriate answer, allowing the support server 20 to input appropriate answers to the sheet items in the business feasibility assessment sheet. However, if there is no customer relationship information in the knowledge base 30, or the information is insufficient or inaccurate, an appropriate answer cannot be obtained for the sheet items. In this case, the person in charge can use an interactive UI screen to reflect knowledge based on the person's own knowledge and insight in the knowledge base 30. This ensures that the knowledge base 30 is filled with the information necessary for an answer, allowing an appropriate AI answer to be obtained. In this embodiment, the AI ​​refers to a conversation partner from the perspective of the person in charge, and directly includes the support server 20 and indirectly includes the language model 40. The AI's response corresponds to a response generated by the support server 20 and / or the language model 40.

[0079] (Functional configuration) 12 is a diagram showing an example of the functional configuration of the review work document creation support server 20 according to this embodiment. The support server 20 has, as main functional units, a document creation request receiving unit 201, an item specifying unit 202, a question generating unit 203, a search unit 204, a question sending unit 205, an answer receiving unit 206, an answer input unit 207, a document output unit 208, and a storage unit 209 shown in FIG. 4, as well as a UI providing unit 211, a question input unit 212, an answer output unit 213, a corrected answer input unit 214, an instruction input unit 215, a knowledge input unit 216, a knowledge reflecting unit 217, and a natural language processing unit 218.

[0080] The UI providing unit 211 has a function of providing an interactive UI screen.

[0081] The question input unit 212 has a function of allowing the user to input questions regarding input items provided in the format of the review work document via an interactive UI screen.

[0082] The answer output unit 213 has a function of outputting an answer sentence corresponding to a question sentence to the user based on answer information from the language model 40 via an interactive UI screen.

[0083] The corrected answer input unit 214 has a function of inputting an answer sentence corrected by the user from the user via an interactive UI screen.

[0084] The instruction input unit 215 has a function of allowing the user to input an instruction to input a response to an input item via an interactive UI screen.

[0085] The knowledge input unit 216 has a function for allowing the user to input question-related information via an interactive UI screen.

[0086] The knowledge reflecting unit 217 has a function of reflecting the question-related information input by the user in the knowledge base 30 .

[0087] The natural language processing unit 218 has the function of performing natural language processing on text sentences (question sentences, revised answer sentences, input instructions, question-related information, answer sentences, etc.) input and output via an interactive UI screen, thereby realizing an interactive UI between the user and AI.

[0088] <UI screen for creating a business feasibility evaluation sheet> FIG. 13 is a diagram showing an example of a UI screen for the sheet item "2. Business Content" according to this embodiment. The person in charge accesses the support server 20 using the terminal 50, and when creating a business feasibility evaluation sheet, the UI screen shown in FIG. 13 is displayed on the screen. The person in charge can create a business feasibility evaluation sheet by repeatedly interacting with the AI ​​through the interactive UI screen, further reflecting human knowledge and insight into the AI's answers.

[0089] When the person in charge inputs a text sentence into an input field 501 on the UI screen, the support server 20 displays the input text sentence as the person in charge's dialogue. For example, if the person in charge wants to get an answer to sheet item "2. Business content," the person in charge inputs a question 502 for requesting an answer to sheet item "2. Business content" into the input field 501. The text sentence input into the input field 501 may be voice input, as long as it is ultimately converted into text.

[0090] When a question 502 is input by the person in charge, the support server 20 generates a question to answer the sheet item "2. Business content" (S3 in FIG. 6), searches and acquires question-related information similar to the question from the knowledge base 30 (S4 in FIG. 6), queries the language model 40 for the question and the searched and acquired question-related information (S5 in FIG. 6), receives an answer from the language model 40 (S21 in FIG. 7), and displays the received answer as the AI's answer 503. In this way, on the UI screen, text indicating the question input by the person in charge and text indicating the answer generated by the AI ​​(language model 40) are displayed in an interactive format, so the person in charge can carefully check the content of the AI's output answer one by one.

[0091] Here, if the knowledge base 30 does not contain, or the customer relationship information related to the question 502 is insufficient or inaccurate, the language model 40 cannot generate an appropriate answer, and the AI ​​displays a response to that effect. For example, in the UI screen shown in Figure 13, in response to the question 502 from the employee, "Please summarize the branches and sales offices of Company A," the knowledge base 30 does not contain any customer information related to the branches and sales offices of Company A, so the language model 40 cannot generate an appropriate response regarding the branches and sales offices of Company A, and the AI ​​displays a response 504 saying, "I'm sorry, but there is not enough information regarding the branches and sales offices of Company A. Please check."

[0092] Upon receiving this response 504, if the salesperson obtains the information, for example, by researching the branch or sales office of Company A, the salesperson additionally inputs the obtained information about the branch or sales office into the input field 501. This additionally input information is, so to speak, the salesperson's knowledge and insight obtained based on, for example, daily sales activities. As a result, the customer information about Company A's branch or sales office is also reflected in the customer information DB and knowledge base 30, and after the reflection, the language model 40 can generate an appropriate response. For example, in the UI screen shown in FIG. 13, in response 505 from the salesperson, "Company A has a branch in Osaka," the customer information about Company A's branch or sales office is reflected in the customer information DB and knowledge base 30, and the language model 40 can generate an appropriate response about Company A's branch or sales office. As a result, the AI ​​displays a response 506 saying, "Thank you. Company A's customer information has been updated. Once again, Company A has a branch or sales office in Osaka."

[0093] Then, when the person in charge receives this response 506, for example, responds by saying "Enter in business details" 507, the AI ​​displays a response 508 saying "We will enter this content in 'Business Details'." The support server 20 then compiles the response sentences received from the language model 40 and inputs (transcribes) them into the sheet item "2. Business Details" of the business feasibility evaluation sheet.

[0094] The UI screen also provides a selection button 509 for selecting "appropriate" or "inappropriate" to assist the person in the "evaluation of the AI ​​answer" operation. By pressing the "appropriate" or "inappropriate" selection button as appropriate during the dialogue, the person in charge can provide feedback to the support server 20 and the language model 40 as to whether the AI ​​answer (the answer generated by the language model 40) is appropriate or not. In this case, for example, the language model 40 learns an AI answer that is determined to be "appropriate" as a desirable "answer" from the next time onwards, and learns an AI answer that is determined to be "inappropriate" as an undesirable "answer" from the next time onwards. This makes it possible to build a language model 40 that further reflects human knowledge and insight in the AI ​​answers and is capable of generating more specialized answer sentences required for review work documents.

[0095] The UI screen also provides selection buttons 510 for "Enter in business details," "Ask another question," and "Edit" to assist the person in the "Input sheet items" operation. If the person in charge determines that the AI ​​answer obtained through the dialogue can be entered into the sheet item "Business details," the person in charge presses the "Enter in business details" selection button. In response to this input instruction, the support manager 20 responds 508, "We will enter these details into 'Business details'," and can input (transcribe) the AI ​​answer (the answer generated by the language model 40) into the sheet item "2. Business details" of the business feasibility assessment sheet.

[0096] On the other hand, if the person in charge wants to switch to a different question without entering (transferring) the AI's answer into sheet item "2. Business Contents," they can reset the AI's answer that has been given so far by pressing the "Ask a different question" selection button.

[0097] If the person in charge receives response 504 and is able to obtain the information, for example by looking up information about Company A's branches or sales offices, he or she may directly add the information about the branches or sales offices to Company A's customer information in the customer information DB. In this case, the person in charge can make response 505 on the UI screen shown in FIG. 13 , saying, "Company A's customer information has been updated in the DB." In this case, the customer information about Company A's branches and sales offices is reflected in the customer information DB and knowledge base 30, and language model 40 can similarly generate appropriate response 506 upon receiving response 505.

[0098] 14A is a diagram showing UI screen example 1 for sheet item "6. SWOT analysis" according to this embodiment. Next, for example, if the person in charge wants to get an answer to "S (strengths)" in sheet item "6. SWOT analysis," the person in charge enters question 511 in input field 501 to request an answer to "S (strengths)" of Company A.

[0099] When a question 511 is input by the person in charge, the support server 20 generates a question to answer "S (strengths)" of the sheet item "6. SWOT analysis" (S3 in Figure 6), searches and acquires question-related information similar to the question from the knowledge base 30 (S4 in Figure 6), queries the language model 40 for the question and the searched and acquired question-related information (S5 in Figure 6), receives a response from the language model 40 (S21 in Figure 7), and displays the received response as the AI's response 512.

[0100] Here, in order to further enhance the content entered (transcribed) into "S (Strengths)" of sheet item "6. SWOT Analysis," the person in charge enters, for example, a question 513, "Please tell us what points to focus on in SWOT analysis," into input field 501. The intention is to have the AI ​​answer the points to focus on when conducting a SWOT analysis (business feasibility evaluation), and to further clarify the S (Strengths) in Company A's SWOT analysis from the perspective of those points of focus.

[0101] When a question 513 is input by the person in charge, the support server 20 generates a question to answer the viewpoints in SWOT analysis (S3 in FIG. 6), searches and acquires question-related information similar to the question from the knowledge base 30 (S4 in FIG. 6), queries the language model 40 for the question and the searched and acquired question-related information (S5 in FIG. 6), receives an answer from the language model 40 (S21 in FIG. 7), and displays the received answer as an AI answer 514. As described above, the knowledge base 30 stores audit dictionary information that covers all industries and business types in Japan and summarizes industry trends, information analysis, audit points, etc. for each business type. The support server 20 can search and acquire information on "viewpoints in business feasibility evaluation" from the audit dictionary information corresponding to Company A's industry in the knowledge base 30 as question-related information, and input the question and the searched and acquired "viewpoints in business feasibility evaluation" information into the language model 40 as a question.

[0102] The support server 20 receives the answer from the language model 40 and displays an answer 514 such as, "Even if it is a small or medium-sized manufacturer, it is important whether it can boast of its own technical equipment, whether it is a company that holds patent rights, and whether it is working to develop wholesale routes for electrical materials."

[0103] At this time, the UI screen displays, in the form of selection buttons 515, keywords of business feasibility evaluation that have been extracted and generated based on the answers 514 by the support server 20. For example, "What is Company A's proprietary technology?", "How many patent applications has Company A filed?", and "Who are Company A's major clients?" are all important topics when conducting a business feasibility evaluation in Company A's industry. By pressing the selection button for "How many patent applications has Company A filed?", for example, as a topic that the person in charge wants to confirm regarding the SWOT analysis (business feasibility evaluation) of Company A, the person in charge can input a question 516 such as "Please tell me how many patent applications has Company A filed?"

[0104] When a question 516 is input by the person in charge, the support server 20 generates a question to answer the question 516, searches and acquires question-related information similar to the question from the knowledge base 30, queries the language model 40 about the question and the searched and acquired question-related information, receives an answer from the language model 40, and displays the received answer 517. Starting from the business feasibility evaluation viewpoint generated from the examination dictionary information, the person in charge further engages in dialogue between questions and answers to obtain a more detailed answer regarding the sheet item in question.

[0105] Here, for example, if knowledge data related to question 516 does not exist in knowledge base 30, or is insufficient or inaccurate, language model 40 cannot generate an appropriate answer, and the AI ​​displays a response to that effect. For example, in the UI screen shown in Figure 14A, in response to question 516 from the person in charge, "Please tell me the number of patent applications filed by Company A," since knowledge base 30 does not contain any information or external reference materials (external information sources) related to the number of patent applications filed by Company A, language model 40 cannot generate an appropriate response regarding the number of patent applications filed by Company A, and the AI ​​displays response 517 saying, "We're sorry, but we do not have information related to the number of patents filed by Company A in our database. If you know of a database you would like to know, please enter the URL in the source field."

[0106] Upon receiving this response 517, the person in charge can obtain the information source, for example, by checking the URL (Uniform Resource Locator) of a database related to Company A's patent count. If the person in charge can obtain the information source, the person in charge registers the external information source in the knowledge base DB30 (by issuing an API addition request for the external information source). The person in charge's response 518, "https: / / www...", represents the person in charge's knowledge and insight, acquired, for example, through daily sales activities. This allows the knowledge base 30 to reflect the URL indicating the location of the database related to Company A's patent count, enabling the language model 40 to generate an appropriate response. For example, in the UI screen shown in FIG. 14A, in response to the person in charge's response 518, "https: / / www...", the AI ​​displays a response 519 saying, "Thank you. Please wait a moment until it becomes available."

[0107] 14B is a diagram showing a UI screen example 2 for the sheet item "6. SWOT analysis" according to this embodiment. Again, the person in charge inputs a question 521, for example, "Please tell me the number of patent applications filed by Company A."

[0108] When a question 521 is input by a person in charge, the support server 20 generates a question to answer the question 521, searches and acquires question-related information similar to the question from the knowledge base 30 (for example, a URL indicating the location of a database related to the number of patents of Company A), queries the language model 40 for the question and the searched and acquired question-related information, receives a response from the language model 40, and displays the received response 522.

[0109] After the URL of the database related to Company A's patent count is reflected in knowledge base 30, language model 40 can access the URL and generate an appropriate answer. For example, in the UI screen shown in FIG. 14B, in response to question 521 from the person in charge, the AI ​​displays answer 522: "According to the patent information form, Company A has filed XX patent applications." Furthermore, since the database URL reflected in knowledge base 30 allows for obtaining information on the number of patent applications for companies other than Company A, in response to question 523 from the person in charge, "Please tell me the number of patent applications filed by our competitors," the AI ​​displays answer 524: "According to the same information source, Company B has filed XX patent applications, and Company C has filed XX patent applications."

[0110] Next, if the person in charge wants to revise the AI ​​answer (the S (strengths) in Company A's SWOT analysis) that they have received so far, they can press the "revise" selection button to issue a response 525 saying "revise," followed by a statement of revision, for example, "·S (strengths): The company's DX products for production factories have rapidly expanded their industry share since their release, and are contributing greatly to the company's sales. The management is aiming to achieve the top share in the industry within three years, and it is expected that this will be achieved.

[0111] · According to the patent information xxx form, Company A has filed ○ patent applications. According to the same source, Company B has filed ○ patent applications, and Company C has filed ○ patent applications. Compared to other companies, the number of patent applications is overwhelmingly higher, indicating a strong desire for research and development. · Company A is currently planning bold cost-cutting measures, which are expected to improve profit margins. " Enter correction 526.

[0112] The revised sentence "Compared to other companies, the number of patent applications is overwhelming, indicating a strong desire for research and development" is, so to speak, the knowledge and insight of the person in charge, reflected based on the dialogue from Question 521 to Answer 524. Similarly, the revised sentence "Company A is currently planning bold cost-cutting measures, and an improvement in profit margins is expected" can also be considered the knowledge and insight of the person in charge, gained, for example, from daily sales activities.

[0113] Furthermore, when the person in charge presses the selection button for "Enter into S (Strengths)", a response 527 of "Enter into Strengths" is issued, and the AI ​​displays a response 528 saying "We will enter the contents here into 'S (Strengths)'." The support server 20 can then input (transcribe) the corrected AI response into "Strengths" in the sheet item "6. SWOT analysis" of the business feasibility evaluation sheet.

[0114] In addition to "Input to S (Strengths)," the UI screen shown in FIG. 14B also has a selection button 510 for "Input to W (Weaknesses)." For example, even in a dialogue process where a question is asked to identify Company A's strengths, if the AI ​​answer obtained is not necessarily a strength, but rather a weakness, and the person in charge determines that this is a weakness, by pressing the "Input to W (Weaknesses)" selection button, the AI ​​answer (the answer generated by the language model 40) can be entered (transcribed) into "W (Weaknesses)" in the sheet item "6. SWOT Analysis" of the business feasibility assessment sheet. Specifically, for example, this is the case when the answer obtained indicates that the number of patent applications filed by Company A is smaller than the number of patent applications filed by its competitors.

[0115] 15 is a diagram showing a UI screen example 3 for the sheet item "6. SWOT analysis" according to this embodiment. Next, for example, if the person in charge wants to get an answer to the "W (weakness)" of the sheet item "6. SWOT analysis," the person in charge enters a question 531 in the input field 501 to request an answer to the "W (weakness)" of Company A.

[0116] When a question 531 is input by the person in charge, the support server 20 generates a question to answer the "W (weakness)" of the sheet item "6. SWOT analysis" (S3 in Figure 6), searches and acquires question-related information similar to the question from the knowledge base 30 (S4 in Figure 6), queries the language model 40 for the question and the searched and acquired question-related information (S5 in Figure 6), receives a response from the language model 40 (S21 in Figure 7), and displays the received response as the AI's response 532.

[0117] Here, in order to further enhance the content entered (transcribed) into "W (Weakness)" of sheet item "6. SWOT Analysis," the person in charge enters, for example, question 533, "Please tell us what you consider to be important points regarding the company's cash flow," into input field 501. The intention is to have the AI ​​answer what to consider when evaluating the company's cash flow, and to clarify whether or not that point of view could be a W (weakness) in Company A's SWOT analysis.

[0118] When the person in charge inputs question 533, the support server 20 obtains information on "points to consider when managing a company's cash flow" from the examination dictionary information corresponding to Company A's industry, embeds the obtained "points to consider when managing a company's cash flow" together with question 533 in a prompt, and inputs it as a question into the language model 40.

[0119] The support server 20 receives the answer from the language model 40 and displays an answer 534 such as, "Is the proportion of payments made by bills payable high? Are collection terms prolonged? Is trading company financing occurring, such as advance payments to suppliers or customers? Is the financial burden excessive?"

[0120] Also, on the UI screen at this time, cash flow focus keywords extracted and generated based on the answers 534 are displayed in the form of selection buttons 535. For example, "What is the settlement ratio of Company A's bills payable?", "What are Company A's collection conditions?", and "What is Company A's trading company finance?" are all important points of focus when evaluating a company's cash flow. By pressing the selection button for, for example, "What is the settlement ratio of Company A's bills payable?" as a focus that the person in charge wants to confirm regarding Company A's cash flow, the person in charge can input question 536, "Please tell me the settlement ratio of Company A's bills payable."

[0121] When a question 536 is input by the person in charge, the support server 20 generates a question to answer the question 536, searches and acquires question-related information similar to the question from the knowledge base 30, queries the language model 40 about the question and the searched and acquired question-related information, receives an answer from the language model 40, and displays the received answer 537. Starting from the viewpoint of business feasibility evaluation generated from the examination dictionary information, the person in charge further engages in dialogue between questions and answers to obtain a more detailed answer regarding the sheet item in question.

[0122] When the person in charge presses the selection button for "Enter into W (Weakness)", the AI ​​displays a response 539 saying, "We will enter this content into 'W (Weakness)'." The support server 20 can then input (transcribe) this AI response into the "Weakness" section of the sheet item "6. SWOT analysis" on the business feasibility evaluation sheet.

[0123] <Summary> According to this embodiment, the following effects are achieved. When creating a business feasibility assessment sheet, the financial institution staff member can repeatedly interact with the AI ​​through an interactive UI screen, confirming the AI's answers for each sheet item one by one, and then input the answers that the staff member is satisfied with into the corresponding sheet item (507, 527, 538) and reflect them.

[0124] In addition to the AI ​​answers (512, 531) obtained for the sheet items, the financial institution staff can obtain further AI answers (514, 522, 524) for the sheet items by asking questions (513, 516, 521, 523, 536) based on the staff's knowledge and insight.

[0125] In addition, the financial institution staff can correct the AI ​​answers obtained for the sheet items based on the staff's knowledge and insight (525, 526) and input them into the corresponding sheet items (527).

[0126] Furthermore, if the knowledge data in the knowledge base 30 does not exist, is insufficient, or is inaccurate, the support server 20 cannot obtain appropriate answers (504, 517) for the sheet items from the language model 40. Through an interactive UI screen, the AI ​​prompts the financial institution staff member to add the missing knowledge data, and the staff member can add the missing knowledge data to the knowledge base 30 based on their own knowledge and insight (505, 518). Furthermore, by enriching or improving the accuracy of the knowledge base 30, it is expected that the accuracy and efficiency of subsequent business feasibility evaluation sheet creation will be improved.

[0127] According to the present embodiment, when creating a business feasibility assessment sheet, repeated dialogue between the AI ​​and a person (e.g., a financial institution employee) is carried out through an interactive user interface, which allows the AI's answers to be further influenced by human knowledge and insight, thereby further enriching the content of the business feasibility assessment sheet (sheet items) and improving its accuracy.

[0128] Although the present invention has been described with reference to specific examples according to the preferred embodiments of the present invention, it is apparent that various modifications and changes can be made to these examples without departing from the broad spirit and scope of the present invention as defined in the appended claims. In other words, the details of the examples and the accompanying drawings should not be construed as limiting the present invention.

[0129] The business feasibility assessment sheet according to this embodiment is an example of an examination work document. The examination work document created by the support server 20 may be the loan approval document itself for loan examination. In addition, financial institutions may call the business feasibility assessment sheet by various names, such as a "company profile," "examination form," "management issue sharing sheet," or "hearing sheet."

[0130] (Addendum) An examination business document creation support device (examination business document creation support server 20) in a financial institution (bank, etc.), UI providing means for providing an interactive UI screen (UI providing unit 211); A question input unit (question input unit 212) for inputting questions regarding input items (answer input items) provided in the format of the examination business document (business feasibility evaluation sheet) via the UI screen. a question generation unit (203, S3) for generating question information (prompt) for causing a language model (fine-tuned large-scale language model 40) to answer the question sentence; a question sending means for sending the question information to the language model (question sending unit 205, S5); an answer receiving means for receiving answer information corresponding to the question information from the language model (answer receiving unit 206, S21); an answer output means for outputting an answer corresponding to the question based on the answer information via the UI screen (answer output unit 213); an instruction input means for inputting an instruction to input the answer sentence for the input item via the UI screen (an instruction input unit 215); an answer input unit for inputting input information based on the answer sentence into the input item in response to the input instruction (answer input unit 207, S22); An examination business document creation support device comprising: [Explanation of symbols]

[0131] 10 Financial Institution System 20 Review document creation support server 30 Knowledge Base 40 Fine-tuned Large-Scale Language Models 50 devices 70 Network 100 Examination Document Creation Support System 201 Document Creation Request Receiving Unit 202 Item Specification Section 203 Question generation section 204 Search Department 205 Question Submission Department 206 Response Receiving Department 207 Answer input section 208 Document Output Unit 209 Memory section 211 UI provision department 212 Question input section 213 Answer output section 214 Corrected Answer Input Section 215 Instruction input section 216 Knowledge Input Unit 217 Knowledge Reflection Department 218 Natural Language Processing Unit

Claims

1. A screening business document creation support device in a financial institution, a UI providing means for providing an interactive UI screen; a question input means for inputting a question regarding an input item provided in the format of the examination work document via the UI screen; a question generation means for generating question information for causing a language model to answer the question sentence; a question sending means for sending the question information to the language model; an answer receiving means for receiving answer information corresponding to the question information from the language model; an answer output means for outputting an answer corresponding to the question based on the answer information via the UI screen; an instruction input means for inputting an instruction to input the answer sentence for the input item via the UI screen; an answer input means for inputting input information based on the answer sentence into the input item in response to the input instruction; An examination business document creation support device comprising:

2. The answer input means A first question sentence is input, and a first answer sentence corresponding to the first question sentence is output; After a second question sentence is input and a second answer sentence corresponding to the second question sentence is output, When the input instruction is input, inputting input information based on the first reply sentence and the second reply sentence into the input items; 2. The examination document creation support device according to claim 1, wherein:

3. The input items are input items related to a SWOT analysis of the customer, Information regarding the focus of the SWOT analysis based on the examination dictionary information is output via the UI screen, the question is a question for causing the language model to answer a point of view of the SWOT analysis; 3. The examination work document creation support device according to claim 1 or 2,

4. a corrected answer input means for inputting the answer sentence corrected by the user via the UI screen, The answer input means After a question is input and an answer corresponding to the question is output, When the input instruction for the corrected answer sentence is entered, inputting input information based on the corrected answer sentence into the input items; 2. The examination document creation support device according to claim 1, wherein:

5. a search means for searching a knowledge base for question-related information related to the generated question information; the question sending means for sending the question information and the question-related information to the language model; a knowledge input means for inputting the question-related information via the UI screen; a knowledge reflecting means for reflecting the input question-related information in the knowledge base; 2. The examination document creation support device according to claim 1, further comprising:

6. The input question-related information is location information of an external information source; 5. The examination document creation support device according to claim 4, wherein:

7. A document preparation support device for financial institutions a UI providing procedure for providing an interactive UI screen; a question input procedure for inputting a question regarding an input item provided in the format of the review work document via the UI screen; a question generation step of generating question information for causing a language model to answer the question sentence; a question transmission step of transmitting the question information to the language model; an answer receiving step of receiving answer information corresponding to the question information from the language model; an answer output step of outputting an answer corresponding to the question based on the answer information via the UI screen; an instruction input step of inputting an instruction to input the answer sentence for the input item via the UI screen; an answer input step of inputting input information based on the answer sentence into the input item in response to the input instruction; A method for supporting the preparation of examination documents.

8. Computer, a UI providing means for providing an interactive UI screen; a question input means for inputting a question regarding an input item provided in the format of the examination work document via the UI screen; a question generation means for generating question information for causing a language model to answer the question sentence; a question sending means for sending the question information to the language model; an answer receiving means for receiving answer information corresponding to the question information from the language model; an answer output means for outputting an answer corresponding to the question based on the answer information via the UI screen; an instruction input means for inputting an instruction to input the answer sentence for the input item via the UI screen; an answer input means for inputting input information based on the answer sentence into the input item in response to the input instruction; A support program for creating review documents to make the system function as a review process.

Images