Examination work document creation support device, examination work document creation support method, and examination work document creation support program

The screening business document creation support device addresses the issue of insufficient AI responses by incorporating an interactive UI and a fine-tuned language model, enhancing the accuracy and relevance of examination documents through human-AI interaction.

JP7579554B1Active Publication Date: 2024-11-08GENERIC SOLUTION CORP

Patent Information

Application Number
JP2024099560
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2024-06-20
Publication Date
2024-11-08
Estimated Expiration
2044-06-20

AI Technical Summary

Technical Problem

Existing AI language models and knowledge bases used in financial institution document creation often provide insufficient or inaccurate content, leading to misalignment with the desired viewpoints of financial institution personnel.

Method used

A screening business document creation support device that includes an interactive UI, question input, generation, transmission, and reception mechanisms, along with a fine-tuned large-scale language model and knowledge base to enrich and improve the accuracy of examination documents.

Benefits of technology

Enhances the content and accuracy of examination documents by integrating human knowledge and interaction with AI, allowing for more specialized and relevant responses.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007579554000001_ABST
    Figure 0007579554000001_ABST
Patent Text Reader

Abstract

We aim to further enrich the content and improve the accuracy of the preparation of review work documents. [Solution] The screening work document creation support device for financial institutions of the present invention has a UI providing means for providing an interactive UI screen, a question input means for inputting a question regarding an input field provided in the format of the screening work document via the UI screen, a question generating means for generating question information for allowing a language model to answer the question, a question sending means for sending question information to the language model, an answer receiving means for receiving answer information corresponding to the question information from the language model, an answer output means for outputting an answer corresponding to the question based on the answer information via the UI screen, an instruction input means for inputting an input instruction for the answer text for the input field via the UI screen, and an answer input means for inputting input information based on the answer text into the input field in accordance with the input instruction.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an examination work document creation support device, an examination work document creation support method, and an examination work document creation support program. [Background technology]

[0002] A business feasibility assessment is a process in which financial institutions evaluate a company when deciding whether to provide loans to a client company, taking into account not only quantitative information such as the company's financial statements, but also qualitative information such as industry trends, the company's strengths and weaknesses, future vision, and management issues. Generally, when conducting a business feasibility assessment, a document called a "business feasibility assessment sheet" is prepared as a sheet format for financial institutions to use for internal information communication. A business feasibility assessment sheet is a sheet format / format that has been uniquely devised to enable financial institutions to appropriately evaluate companies. The staff at the branch office fill in the sheet items by collecting quantitative information such as the company information and financial statements of the company they are in charge of, and summarizing qualitative information based on interviews with the management, such as industry trends, the company's strengths and weaknesses, future vision, and management issues. The completed business feasibility assessment sheet is mainly used, for example, as an attachment when drafting a loan request document.

[0003] As an example of a related technology, Patent Document 1 describes an examination work document creation support device for financial institutions, which uses an appropriately tuned language model and specialized knowledge base to enable financial institution personnel to quickly and efficiently create detailed examination work documents, including industry trends, SWOT analysis, and management issues from the perspective of management. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7396582 Summary of the Invention [Problem to be solved by the invention]

[0005] However, on the other hand, the language model and knowledge base described in the above Patent Document 1 have problems in that the content of the generated AI answers is sometimes insufficient or inaccurate, and may deviate from the perspective desired by the financial institution's personnel.

[0006] The present invention has been proposed in view of the above points, and in one aspect, it is an object of the present invention to further enrich the content and improve the accuracy in the preparation of examination work documents. [Means for solving the problem]

[0007] In order to solve the above problems, the screening work document creation support device of the present invention is a screening work document creation support device for a financial institution, and comprises a UI providing means for providing an interactive UI screen, a question input means for inputting a question regarding an input item provided in a screening work document format via the UI screen, a question generating means for generating question information for a language model to answer the question, a question sending means for sending the question information to the language model, an answer receiving means for receiving answer information corresponding to the question information from the language model, an answer output means for outputting an answer corresponding to the question based on the answer information via the UI screen, an instruction input means for inputting an input instruction for the answer text for the input item via the UI screen, and an answer input means for inputting input information based on the answer text into the input item in response to the input instruction. Effect of the Invention

[0008] According to an embodiment of the present invention, in one aspect, it is possible to further enrich the content and improve the accuracy in creating examination work documents. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram showing an example of the configuration of an examination work document preparation support system 100 according to an embodiment of the present invention. [Diagram 2]FIG. 2 is a diagram showing an example of knowledge data in a knowledge base 30 according to the present embodiment. [Diagram 3] FIG. 2 is a diagram showing an example of labeled training data for the fine-tuned large-scale language model 40 according to the present embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a functional configuration of an examination work document creation support server 20 according to the present embodiment. [Diagram 5] 1 is a diagram showing an outline of the operation of the review work document preparation support system 100 according to the present embodiment. [Figure 6] FIG. 2 is a diagram showing an examination work document creation process 1 according to the present embodiment. [Figure 7] FIG. 2 is a diagram showing an examination work document creation process 2 according to the present embodiment. [Figure 8A] 13 is a diagram showing example questions and answers for the sheet item "1. Corporate attribute information" according to the embodiment. FIG. [Figure 8B] FIG. 13 is a diagram showing example questions and answers for the sheet item “2. Business details” according to the present embodiment. [Figure 8C] FIG. 13 is a diagram showing example questions and answers for the sheet item “3. Shareholder / Investor Information” according to the present embodiment. [Figure 8D] FIG. 13 is a diagram showing example questions and answers for the sheet item “4. Management Information” according to the present embodiment. [Figure 8E] FIG. 13 is a diagram showing example questions and answers for the sheet item “5. Industry trends” according to the present embodiment. [Figure 8F] FIG. 13 is a diagram showing example questions and answers for the sheet item “6. SWOT analysis” according to the present embodiment. [Figure 8G] FIG. 13 is a diagram showing example questions and answers for the sheet item “break-even sales analysis” according to the embodiment. [Figure 8H] FIG. 13 is a diagram showing example questions and answers for a sheet item "cash flow analysis" according to the present embodiment. [Figure 9] FIG. 2 is a diagram showing an example 1 of sheet items of a business feasibility evaluation sheet according to the present embodiment. [Figure 10] FIG. 2 is a diagram showing example 2 of sheet items of a business feasibility evaluation sheet according to the present embodiment. [Figure 11] 1 is a diagram showing an example of the configuration of an examination work document preparation support system 100 according to an embodiment of the present invention. [Figure 12] 2 is a diagram illustrating an example of a functional configuration of an examination work document creation support server 20 according to the present embodiment. FIG. [Figure 13] 13 is a diagram showing an example of a UI screen for a sheet item “2. Business details” according to the present embodiment. FIG. [Figure 14A] FIG. 13 is a diagram showing a UI screen example 1 for a sheet item “6. SWOT analysis” according to the present embodiment. [Figure 14B] FIG. 13 is a diagram showing a UI screen example 2 for the sheet item “6. SWOT analysis” according to the present embodiment. [Figure 15] FIG. 13 is a diagram showing a UI screen example 3 for the sheet item “6. SWOT analysis” according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to the drawings. [Embodiment 1] <System configuration> Fig. 1 is a diagram showing an example of the configuration of a screening work document preparation support system 100 according to this embodiment. The screening work document preparation support system 100 in Fig. 1 includes a financial institution system 10, a screening work document preparation support server 20, a knowledge base (vector DB) 30, a fine-tuned large-scale language model (LLM) 40, and a terminal 50, which are connected via a network 70.

[0011] The financial institution system 10 includes various systems and DBs (databases) owned by a financial institution such as a bank. The financial institution system 10 includes, for example, a core system that manages transactions such as various deposits and loans, a sales management system that manages sales records of financial products, a front compliance system for appropriately managing transaction details in accordance with laws and regulations, an SFM (Sales Force Management System) and CRM (Customer Relationship Management) that manage customer information including loan history and negotiation records with customers (sales records), and each DB associated with each system. Each DB of the financial institution system 10 includes, for example, DBs for customer information, transaction data, financial data, negotiation record data, etc. of client companies at the financial institution.

[0012] The screening business document creation support server (hereinafter also simply referred to as the support server) 20 is a server device that supports the creation of screening business documents to be created by financial institution personnel. When the support server 20 receives a request to create, for example, a business feasibility evaluation sheet as a screening business document from the terminal 50 of the personnel, the support server 20 creates a business feasibility evaluation sheet by asking questions to a fine-tuned large-scale language model (hereinafter also simply referred to as the language model) 40 about predetermined input items (hereinafter also referred to as sheet items) in the business feasibility evaluation sheet and filling in the sheet items based on the answers obtained. A specific creation example will be described later.

[0013] The knowledge base 30 is a knowledge database in which information on client companies of financial institutions (called customer relationship information) is stored and constructed in advance, for example, by vector conversion. The customer relationship information includes not only public information about client companies that is publicly available, but also private information such as in-bank information. Since the language model 40 cannot answer questions about contents other than those included in the learning data, even if an attempt is made to answer a question specific to the client company, for example, an answer that the question is not known or an answer based on different knowledge may be obtained. As a method for solving this problem, in this embodiment, customer relationship information that is likely to be necessary to answer the question is stored (stored) in the knowledge base 30 in advance. The support server 20 first searches the customer relationship information stored in the knowledge base 30 for question-related information that is likely to be necessary to answer the question, and passes the searched question-related information to the language model 40 in addition to a prompt, which is an instruction / input sentence. This makes it possible to have the language model 40 think of an appropriate answer even if the content is not learned by the language model 40.

[0014] The fine-tuned large-scale language model 40 is a language model in which knowledge of a specific domain is further supervised and learned from a general-purpose natural language processing model (LLM: Large Language Models), which has been pre-trained in an unsupervised manner using a large amount of text data to learn grammar, word meanings, and the like. The language model 40 is made capable of generating highly specialized answer sentences required for review work documents by fine-tuning a set of appropriate "questions" and desirable "answers" in the general-purpose natural language processing model. For example, the general-purpose natural language processing model may be BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pre-trained Transformer)-3.5, GPT-4, etc. Note that instead of a network system configuration in which the language model 40 is stored in a storage device within the bank's network 70, it is also possible to access a server of a fine-tuned large-scale language model provided as an external service and read it out.

[0015] The terminal 50 is, for example, a PC (personal computer), a smartphone, a tablet terminal, or the like, and is, for example, a user terminal used by a person in charge of a sales office. A predetermined application program for accessing the support server 20, a general-purpose web browser, and the like are pre-installed in the terminal 50. The person in charge uses the terminal 50 to access the support server 20, obtains the business feasibility evaluation sheet created by the support server 20, and displays it on the screen. Note that, depending on the sheet items in the business feasibility evaluation sheet, there may be desirable items that the person in charge has filled in himself / herself, so the support server 20 does not necessarily complete the business feasibility evaluation sheet by filling in all the sheet items in the business feasibility evaluation sheet. The person in charge can complete the final business feasibility evaluation sheet by reviewing and supplementing the business feasibility evaluation sheet created primarily by the support server 20.

[0016] The network 70 is a communication network of the financial institution, and is preferably an in-bank network from the viewpoint of security. However, the network 70 may include, for example, the Internet, a public line network, WiFi (registered trademark), etc., provided that sufficient security is ensured.

[0017] (Knowledge Base 30) FIG. 2 is a diagram showing an example of knowledge data in the knowledge base 30 according to this embodiment. The knowledge base 30 is a knowledge database in which customer relationship information in a financial institution is stored and constructed in advance after, for example, being vector-converted so as to be easily searched. The knowledge data in the knowledge base 30 according to this embodiment includes, as customer relationship information of client companies in the financial institution, for example, customer information, transaction data, financial data, and negotiation record data of Company A, Company B, Company C, etc. Furthermore, the knowledge data in the knowledge base 30 according to this embodiment includes, for example, industry-specific examination dictionary information and corporate enterprise statistics information, which are stored as external reference materials.

[0018] The customer information is customer information of a customer company that does business with a financial institution. In addition to general company information such as trade name or name, location, telephone number, and representative name, the information includes information required to be submitted when opening an account, such as industry and business type. The customer information can be acquired, for example, from the core system of the financial institution system 10. The customer information can be further supplemented and enriched with information such as the president's greeting, our strengths, future vision, business content, products and services, bases, sales areas, history, major clients, and shareholders, based on information such as the customer company's own website posted by scraping or other means, or a corporate information database provided by an external credit investigation company. The business feasibility evaluation sheet includes sheet items such as "corporate attribute information," "business content," "shareholder and investor information," "management information," "industry trends," and "SWOT analysis," and the customer information can be used as one of the question-related information for these questions when answering them using the language model 40.

[0019] The transaction data includes, for example, information on each account held by each customer, such as ordinary savings accounts, current accounts, fixed term deposit accounts, and foreign currency deposit accounts, credit card information, and transaction details including foreign exchange transactions (liquidity transaction details information), as well as information on loans made in the past. All transaction details from the present to the past are recorded. The transaction data can be obtained, for example, from the core system of the financial institution system 10. The business feasibility evaluation sheet includes, for example, a sheet item called "corporate attribute information," and the transaction data can be used as one piece of question-related information for these questions when the language model 40 responds.

[0020] The financial data includes not only financial statements (balance sheets, profit and loss statements, cash flow statements, etc.) for each business year of the client company, but also data related to the finances of the client company, such as trial balance data for the period showing the progress of profit and loss for the year. The financial data can be acquired, for example, from the core system of the financial institution system 10. The business feasibility evaluation sheet includes sheet items such as "shareholder / investor information," "management information," "industry trends," "SWOT analysis," "break-even sales analysis," and "cash flow analysis," and the financial data can be used as one piece of question-related information for these questions when the language model 40 answers the questions.

[0021] The negotiation record data is a record history of negotiations that have been conducted in the past between the financial institution side, such as a person in charge of a branch office, and the client company side, such as the manager or a person in charge of the financial department. It includes not only documents and emails exchanged between the two parties, but also conversation items (such as the manager's future vision, management issues, succession issues, etc.) discussed in dialogues such as interviews (visits, visits to the branch), online conferences, and telephone calls, as long as they are recorded in the form of text, audio, images, videos, etc. The negotiation record data can be obtained, for example, from SFM or CRM that manages customer information including the loan history of the financial institution system 10 and negotiation records with customers (sales records). The business feasibility evaluation sheet has sheet items such as "industry trends" and "SWOT analysis," and the negotiation record data can be used as a piece of question-related information for these questions when the language model 40 answers them.

[0022] The industry-specific screening dictionary information is information from an industry information dictionary that covers all industries and business types in Japan and compiles industry trends, information analysis, screening points, etc. for each industry. It is used as a reference by financial institution personnel when conducting loan screening to understand the type of industry of the borrower. A business feasibility evaluation sheet may have sheet items such as "industry details," "industry trends," and "SWOT analysis," and the industry-specific screening dictionary information can be used as a piece of question-related information for these questions when the language model 40 responds. This is expected to enable the language model 40 to provide responses that are more in line with the perspective of the financial institution's loan screening.

[0023] The Corporate Enterprise Statistics Survey information is information from a core statistical survey conducted as a sample survey based on the Statistics Act in order to grasp the actual state of business activities of profit-making corporations and the like in Japan. Survey results include financial indicators such as sales, capital investment, and current profits of profit-making corporations and the like, as well as the status of business activities by industry, size, and region. For example, the business feasibility evaluation sheet has sheet items such as "industry trends" and "SWOT analysis," and the Corporate Enterprise Statistics Survey information can be used as a piece of question-related information for these questions when the language model 40 responds. This is expected to enable the language model 40 to provide responses that are more in line with the perspective of financial institutions' loan screening.

[0024] (40 fine-tuned large-scale language models) 3 is a diagram showing an example of labeled learning data of the fine-tuned large-scale language model 40 according to this embodiment. As described above, the fine-tuned large-scale language model 40 is a language model in which knowledge of a specific domain is further supervised-learned on a general-purpose natural language processing model (LLM) that has been pre-trained in an unsupervised manner on grammar, word meanings, and the like using a large amount of text data. A model capable of generating highly specialized answer sentences required for review work documents is created by learning (fine-tuning) a sentence set of "questions" and exemplary "answers" to elicit appropriate answers to the sheet items on a business feasibility evaluation sheet.

[0025] For example, the business feasibility evaluation sheet has a sheet item called "History," and as a sample set of questions and model answers to obtain appropriate answers to this sheet item "History," multiple sets of "Question: Summarize the history of Company A in chronological order using XXX characters" and "Answer: In 2020, XXX founded Company A in XX ward, Tokyo. In 2021..." (learning data a-1)... are trained in advance (fine-tuned).

[0026] In addition, the business feasibility evaluation sheet has a sheet item called "SWOT analysis," for example. As a sample set of questions and model answers to obtain an appropriate answer to this sheet item "SWOT analysis," the following are provided: "Question: You are a banker. As a banker conducting a loan review, please conduct a SWOT analysis from the perspective of the strengths and weaknesses of Company A's internal environment, and the opportunities and threats in the external environment. The analysis should take into account the company's financial data as well as the future vision and management issues as envisioned by the management." and "Answer: S (Strengths): ··, W (Weaknesses): ··, O (Opportunities): ··, T (Threats): ··" (learning data a-6) ··. Multiple sets of these are trained (fine-tuned) in advance.

[0027] Furthermore, for a client company whose business feasibility evaluation sheet has been output in the past by the support server 20 through the processes of S1 to S5 in Fig. 6 and S21 to S23 in Fig. 7 described below, the language model 40 may be trained (fine-tuned) on the questions (S3) and question-related information (S5) generated for each sheet item in the past, and the answers (S22) in the language model 40 input corresponding to the sheet items. Furthermore, instead of the answers (S22) in the language model 40, if the answers have been further revised by the person in charge to be more desirable, the revised answers obtained from the final business feasibility evaluation sheet may be trained.

[0028] (Functional configuration) 4 is a diagram showing an example of the functional configuration of the review work document creation support server 20 according to this embodiment. The support server 20 has, as main functional units, a document creation request receiving unit 201, an item identifying unit 202, a question generating unit 203, a search unit 204, a question sending unit 205, an answer receiving unit 206, an answer input unit 207, a document output unit 208, and a memory unit 209.

[0029] The document creation request receiving unit 201 has a function of receiving, from the terminal 50 of the person in charge, a request to create an examination business document (for example, a business feasibility evaluation sheet) specifying a client company for which the document is to be created.

[0030] The item specification unit 202 has a function of specifying a predetermined answer input item according to the type of the review business document from among the input items provided in the format of the review business document.

[0031] The question generation unit 203 has a function of generating question information for causing the language model 40 to answer input information for a predetermined answer input item.

[0032] The search unit 204 has a function of searching for question-related information related to the generated question information from a knowledge base in which customer relationship information on customers is accumulated.

[0033] The question sending unit 205 has a function of sending question information and customer relationship information to the language model 40 .

[0034] The answer receiving unit 206 has a function of receiving answer information corresponding to question information from the language model 40 .

[0035] The answer input unit 207 has a function of inputting answer information into predetermined answer input items.

[0036] The document output unit 208 has the function of outputting the review business document (e.g., a business feasibility evaluation sheet) to the terminal 50 once answer information based on answers from the language model 40 has been input into all specified answer input items in the review business document.

[0037] The memory unit 209 has a function of storing, in a storage device, information on the format of examination work documents, standard questions for each sheet item, and the like.

[0038] The review work document creation support server 20 can be realized by a general-purpose computer. Specifically, the support server 20 includes hardware such as a processing unit (CPU), a memory, an input / output interface, and a communication interface. The functions of the support server 20 are realized by the processing unit executing processing according to a computer program stored in the memory. That is, each functional unit is realized by a computer program executed on the hardware resources such as the processing unit and memory of the computer constituting the support server 20. The review work document creation support server 20 may be called a review work document creation support device or a computing machine for supporting review work document creation. These functional units may also be read as "means," "module," "unit," or "circuit." Each DB may be placed in the memory of the support server 20 or an external storage device on the network 70. Each functional unit of the support server 20 may not only be realized by a single server device, but may also be realized as a system consisting of multiple devices with distributed functions. The computer program may also be stored in a computer-readable storage medium.

[0039] <Support for creating review documents> Fig. 5 is a diagram showing an outline of the operation of the review work document creation support system 100 according to this embodiment. Fig. 6 is a diagram showing review work document creation processing 1 according to this embodiment. The arithmetic processing unit of the support server 20 reads and executes a program capable of realizing the processing, thereby realizing each of the following steps (hereinafter abbreviated as "S").

[0040] S1: The support server 20 receives a request to create an examination work document (e.g., a business feasibility evaluation sheet) specifying a client company for which the document is to be created, from the person in charge's terminal 50. The creation request includes at least an identifier (e.g., a client number, a client company name, etc.) for identifying the target client company.

[0041] S2: The support server 20 identifies sheet items (predetermined answer input items) to be answered by the language model 40 from among the sheet items (input items) predefined in the format of the review work document stored in the storage unit 209. Since some sheet items in the review work document sheet are desired items filled in by the person in charge himself / herself, the sheet items to be filled in by the support server 20 are predefined according to the type of review work document. In the case of a business feasibility evaluation sheet, for example, "corporate attribute information", "business content", "shareholder / investor information", "management information", "industry trends", "SWOT analysis", "break-even sales analysis", and "cash flow analysis" can be identified as sheet items to be answered by the language model 40.

[0042] S3: The support server 20 generates a question (problem) for answering the sheet item specified in S2 for each sheet item. Note that the sentences of the questions (problems) generated for each sheet item are determined for each sheet item and stored in a standard format in the storage unit 209, and in order to obtain an exemplary answer from the language model 40, the question sentences are in the same format as the sample questions for obtaining an appropriate answer to the sheet item in question in the fine-tuning situation (FIG. 3).

[0043] Specifically, the support server 20 generates a prompt such as "Summarize the history of Company A in chronological order in XXX characters" as a question to answer "History" in one sheet item "Corporate attribute information" identified in S2. The support server 20 also generates a prompt such as "You are a banker. As a banker conducting loan screening, please perform a SWOT analysis of Company A from the perspective of the strengths and weaknesses of the internal environment and the opportunities and threats in the external environment. The analysis should take into account the company's financial data as well as the future vision and management issues as envisioned by the management." as a question to answer "SWOT analysis" in one sheet item identified in S2.

[0044] In addition, questions to answer all of the sheet items identified in S2 may be created as one question, such as, "In order to create a business feasibility evaluation sheet for Company A, please summarize the following in a maximum of XXX characters or less."

[0045] S4: The support server 20 searches and acquires question-related information similar to the question generated in S3 from the knowledge base 30. Specifically, the support server 20 converts the sentence of the question generated in S3 directly into a vector as a query, searches the knowledge base 30 (knowledge vector DB), and acquires similar sentences with close vector distance as question-related information.

[0046] Specifically, as a similar sentence close to the question generated in S3, "Summarize the history of Company A in chronological order in XXX characters," for example, corporate information about Company A (≒ "Company A" and "History") can be obtained as question-related information. In addition, as a similar sentence to the question generated in S3, "You are a bank employee. As a bank employee conducting a loan screening, please conduct a SWOT analysis of Company A from the perspective of the strengths and weaknesses of the internal environment and the opportunities and threats of the external environment. The analysis should take into account the company's financial data as well as the future vision and management issues as envisioned by the management.", for example, corporate information of Company A (≒ "Company A", "Management", "Strengths and Weaknesses", "Future Vision"), financial data of Company A (≒ "Company A", "Financial Data", negotiation record data of Company A (≒ "Company A", "Strengths and Weaknesses", "Future Vision", "Management Issues"), screening dictionary information corresponding to Company A's industry (≒ "Company A", "Loan Screening", "Banker"), and corporate statistics information of Company A's industry (≒ "External Environment") can be obtained as question-related information.

[0047] In addition, in order to answer the question with a SWOT analysis based on the perspective of a financial institution, which is unique to screening work documents, words such as "loan screening" and "bank employee" can be included in the question, making it possible to search for similar sentences in the screening dictionary information corresponding to Company A's industry. As described above, the industry-specific screening dictionary information covers all industries and business types in Japan, and is information from an industry information dictionary that summarizes industry trends, information analysis, screening points, etc. for each business type. It is used by financial institution personnel to understand the type of business of the borrower when conducting loan screening. For this reason, when searching and acquiring screening dictionary information as a knowledge base, it is extremely important to identify the type of business that Company A belongs to. When searching and acquiring question-related information similar to the question (question) generated in S3 from the knowledge base 30, the support server 20 can explicitly enter Company A's business type in the query, thereby making it possible to search and acquire screening dictionary information corresponding to Company A's business type as a similar sentence.

[0048] In this case, it is advisable for the support server 20 to specify the industry of company A in advance at least before S4. A specific method of specification may be, for example, acquisition from the core system of the financial institution system 10, and if it is not possible to acquire the information from the core system or it is difficult to do so, acquisition may be based on information collected by scraping or the like from the customer company's own website or from a company information database provided by an external credit investigation company or the like.

[0049] S5: The support server 20 transmits the question generated in S3 and the question-related information searched and acquired in S4 to the language model 40. Specifically, the support server 20 embeds the generated question and the question-related information searched and acquired from the knowledge base 30 in a prompt, and inputs them to the language model 40 as an inquiry.

[0050] FIG. 7 is a diagram showing the examination work document creation process 2 according to this embodiment. S21: The support server 20 receives an answer from the language model 40 as a response to the inquiry in S5. For example, as a response to the question "Summarize the history of Company A in XXX characters starting from the oldest." and the question-related information "Company information on Company A", the answer text "In 2020, XX founded Company A in XX ward, Tokyo. In 2021, the head office moved to XX ward, Tokyo..." can be received. Note that the answer from the language model 40 is an answer text in the same format as the desirable sample answer based on fine tuning (Figure 3).

[0051] S22: The support server 20 inputs the answer received in S21 into the sheet item (predetermined answer input item) identified in S2 in the review work document. For example, into one sheet item "History" identified in S2, the answer text "In 2020, XX founded company A in XX ward, Tokyo. In 2021, the head office moved to XX ward, Tokyo..." is input. The answer text may be directly transcribed into the sheet item, or the format, etc. may be processed according to the form of the sheet item.

[0052] S23: When the support server 20 has finished inputting all sheet items in the document that should be answered by the language model 40, it outputs the examination work document to the terminal 50. In the business feasibility evaluation sheet according to this embodiment, the sheet items that the language model 40 should answer are, for example, "corporate attribute information," "business content," "shareholder / investor information," "management information," "industry trends," "SWOT analysis," "break-even sales analysis," and "cash flow analysis." Therefore, when all answers from the language model 40 corresponding to these sheet items have been inputted into the business feasibility evaluation sheet, the business feasibility evaluation sheet for the designated client company is output to the terminal 50.

[0053] <Examples of questions and answers for sheet items> Among the sheet items provided in the format of the business feasibility evaluation sheet, examples of sheet items that the support server 20 should fill in using the answers from the language model 40 include "corporate attribute information," "business content," "shareholder / investor information," "management information," "industry trends," "SWOT analysis," "break-even sales analysis," and "cash flow analysis." Specific examples are given below.

[0054] FIG. 8A is a diagram showing an example of a question and answer for the sheet item "1. Corporate attribute information" according to this embodiment. Question Q1 is a question (prompt) generated to have the language model 40 answer the sheet item "1. Corporate attribute information." Furthermore, question Q1 is input as a question to the language model 40 with the corporate information of Company A searched and acquired from the knowledge base 30 and transaction data of Company A embedded in the prompt. Answer A1 is an answer received from the language model 40 as an answer to question Q1. The support server 20 inputs (transcribes) the text of answer A1 received from the language model 40 into sheet item "1. Corporate attribute information" IN1 of the business feasibility evaluation sheet.

[0055] FIG. 8B is a diagram showing an example of a question and answer for the sheet item "2. Business content" according to this embodiment. Question Q2 is a question (prompt) generated to cause the language model 40 to answer the sheet item "2. Business content". Furthermore, question Q2 is input as a question to the language model 40 with the corporate information of Company A searched and acquired from the knowledge base 30 and the examination dictionary information for Company A's industry embedded in the prompt. Answer A2 is an answer received from the language model 40 as an answer to question Q2. The support server 20 inputs (transcribes) the text of answer A2 received from the language model 40 into sheet item "2. Business content" IN2 of the business feasibility evaluation sheet.

[0056] It should be noted that the sheet item “2. Business Content” IN2 includes an item “(1) Industry.” However, since the customer company’s “(1) Industry” can be obtained and input from the customer information DB or the like held by the financial institution system 10 without having to have the language model 40 answer the question, in this embodiment, question Q2 does not include a question regarding “(1) Industry.”

[0057] FIG. 8C is a diagram showing an example of a question and answer for the sheet item "3. Shareholder / Investor Information" according to this embodiment. Question Q3 is a question (prompt) generated to cause the language model 40 to answer the sheet item "3. Shareholder / Investor Information". Furthermore, question Q3 is input as a question to the language model 40 with the corporate information of Company A and financial data of Company A searched and acquired from the knowledge base 30 embedded in the prompt. Answer A3 is an answer received from the language model 40 as an answer to question Q3. The support server 20 inputs (transcribes) the text of answer A3 received from the language model 40 into the sheet item "3. Shareholder / Investor Information" IN3 of the business feasibility evaluation sheet.

[0058] FIG. 8D is a diagram showing an example of a question and answer for the sheet item "4. Management Information" according to this embodiment. Question Q4 is a question (prompt) generated to cause the language model 40 to answer the sheet item "4. Management Information". Furthermore, question Q4 is input as a question to the language model 40 with the corporate information of Company A searched and acquired from the knowledge base 30 and financial data of Company A embedded in the prompt. Answer A4 is an answer received from the language model 40 as an answer to question Q4. The support server 20 inputs (transcribes) the text of answer A4 received from the language model 40 into sheet item "4. Management Information" IN4 of the business feasibility evaluation sheet.

[0059] FIG. 8E is a diagram showing an example of a question and an answer for the sheet item "5. Industry trends" according to this embodiment. Question Q5 is a question (prompt) generated to make the language model 40 answer the sheet item "5. Industry trends". In addition, the question Q5 is input to the language model 40 as a question, with the corporate information of Company A, financial data of Company A, examination dictionary information for Company A's industry, and corporate statistical information for Company A's industry searched and acquired from the knowledge base 30 embedded in the prompt. Answer A5 is an answer received from the language model 40 as an answer to question Q5. The support server 20 inputs (transcribes) the text of answer A5 received from the language model 40 into sheet item "5. Industry trends" IN5 of the business feasibility evaluation sheet.

[0060] FIG. 8F is a diagram showing an example of a question and an answer for the sheet item "6. SWOT analysis" according to this embodiment. The question Q6 is a question (prompt) generated to make the language model 40 answer the sheet item "6. SWOT analysis". In addition, the question Q6 is input to the language model 40 as a question by embedding the corporate information of the company A searched and acquired from the knowledge base 30, the financial data of the company A, the negotiation record data of the company A, the examination dictionary information of the industry of the company A, and the corporate statistical information of the industry of the company A in the prompt. The answer A6 is an answer received from the language model 40 as an answer to the question Q6. The support server 20 inputs (transcribes) the sentence of the answer A6 received from the language model 40 into the sheet item "6. SWOT analysis" IN6 of the business feasibility evaluation sheet.

[0061] Here, for items such as SWOT analysis in a business feasibility evaluation sheet, quantitative information such as general information about the client's industry and business sector, or financial data is insufficient; qualitative information such as the client's daily thoughts and perspective is essential for analysis. For this reason, in the past, when creating a business feasibility evaluation sheet, sales office staff would fill in the sheet items by collecting quantitative information such as corporate information and financial statements of the company they were in charge of, or by analyzing and summarizing qualitative information based on interviews with the client's management.

[0062] For example, the answer A6 according to this embodiment is an answer to the question Q6 in which the negotiation record data of Company A searched and acquired from the knowledge base 30 is embedded in the prompt, and the answer from the language model 40 is an answer based on the information recorded in the negotiation record data. As described above, the negotiation record data is a record history of negotiations that were previously conducted between the financial institution side, such as a sales office staff member, and the client company side, such as the manager or a financial department staff member, which was originally acquired from SFM or CRM. It includes not only documents and emails exchanged between the two parties, but also conversation matters (the manager's future vision, management issues, succession issues, etc.) discussed in dialogues such as interviews (visits, visits to the branch), online conferences, and telephone calls, as long as they are recorded. Therefore, for example, the sentences in the answer A6, "The manager aims to acquire the top share in the industry within three years, and it is expected that this will be achieved." and "Improving development capabilities is a management issue," are answer sentences that reflect the goal (to acquire the top share in the industry within three years), the likelihood of achieving this (high), and the management issue (improving development capabilities) spoken by the manager of Company A, which were recorded as the negotiation record history.

[0063] FIG. 8G is a diagram showing an example of a question and an answer for the sheet item "Break-even sales analysis" according to this embodiment. Question Q7 is a question (prompt) generated to have the language model 40 answer the sheet item "x. Break-even sales analysis." Furthermore, the financial data (profit and loss statement) of company A searched and acquired from the knowledge base 30 is embedded in the prompt for question Q7, and the question is input to the language model 40 as a question. Answer A7 is an answer received from the language model 40 as an answer to question Q7. The support server 20 inputs (transcribes) a graph of answer A7 received from the language model 40 into the sheet item "x. Break-even sales analysis" IN7 of the business feasibility evaluation sheet.

[0064] FIG. 8H is a diagram showing an example of a question and an answer for the sheet item "cash flow analysis" according to this embodiment. Question Q8 is a question (prompt) generated to cause the language model 40 to answer the sheet item "y. cash flow analysis". Furthermore, the financial data (cash flow statement) of company A searched and acquired from the knowledge base 30 is embedded in the prompt for question Q8, and the question is input to the language model 40 as a question. Answer A7 is an answer received from the language model 40 as an answer to question Q8. The support server 20 inputs (transcribes) the graph of answer A8 received from the language model 40 into the sheet item "y. cash flow analysis" IN8 of the business feasibility evaluation sheet.

[0065] In the business feasibility evaluation sheet, for sheet items (answer input items) where graphs or other figures should be input, such as "break-even sales analysis" and "cash flow analysis," the language model 40 can be used to answer questions by providing figures, and the figures based on the answers can be input (transcribed) into the sheet items.

[0066] (Business feasibility evaluation sheet) Fig. 9 is a diagram showing a sheet item example 1 of a business feasibility evaluation sheet according to this embodiment. Fig. 10 is a diagram showing a sheet item example 2 of a business feasibility evaluation sheet according to this embodiment. The person in charge accesses the support server 20 using the terminal 50, acquires the business feasibility evaluation sheet created by the support server 20, and displays it on the screen.

[0067] Of the sheet items provided in advance in the format of the business feasibility evaluation sheet according to this embodiment, it can be seen that sentences and graphs based on the answers of the language model 40 are input into sheet items IN1-IN8.

[0068] Meanwhile, for example, the title of sheet item IN0 and "(1) Industry" of sheet item IN2-2 are input with information acquired / extracted from the customer information DB etc. of the financial institution system 10. Depending on the sheet item, there are also input items for which information can be simply acquired / extracted from various DBs or the knowledge base 30 etc. of the financial institution system 10. For such sheet items, the support server 20 does not need to generate questions and have the language model 40 answer them, which can improve the processing speed etc. up to the output of the business feasibility evaluation sheet.

[0069] Also, for example, sheet item IN10 "sales office opinion" is left blank. This is because some sheet items in the business feasibility evaluation sheet are desirable items that the person in charge has filled in themselves. The person in charge can complete the final business feasibility evaluation sheet by filling in the blank sheet items themselves in the business feasibility evaluation sheet that was primarily created by the support server 20. Of course, the person in charge may also make corrections to the text or graphs based on the answers of the language model 40 for sheet items IN1-IN8 in the primarily created business feasibility evaluation sheet before completing the final business feasibility evaluation sheet.

[0070] (Processing type) The processing types executed by the support server 20 according to this embodiment will be described below. The processing types shown below are categorized from the viewpoint of how the support server 20 acquires input information to be input into the input items of the examination work document. Note that which processing type is applied to each input item is determined in advance by the system administrator of the support server 20, etc., as the processing type that is optimal for each input item according to its subject matter, properties, etc.

[0071] ·Search and generation type (30 knowledge bases + 40 language models) Search and obtain question-related information similar to the generated question from the knowledge base 30, and embed the generated question, the question-related information searched and obtained from the knowledge base 30, and the prompt, and input them as a query to the language model 40. The search and generation type corresponds to the above-mentioned sheet items IN1-IN8, for example.

[0072] Extraction type (only various DBs and knowledge bases 30 of financial institution systems 10) Information is simply acquired and extracted from various DBs or the knowledge base 30 of the financial institution system 10. The extraction type corresponds to, for example, the title of sheet item IN0 or "(1) Industry" of sheet item IN2-2 described above.

[0073] ·Generative (Language Model 40 only) This type does not search or obtain question-related information similar to the generated question from the knowledge base 30, but inputs the generated question as an inquiry to the language model 40. This is applicable to input items for which a sufficient answer can be obtained from the language model 40 even if there is no question-related information.

[0074] <Summary> As described above, the creation of screening work documents such as a business feasibility evaluation sheet is an important process for screening companies, and it is necessary to properly evaluate the market environment, future prospects, management issues, etc. of the target company. According to the screening work document creation support system of this embodiment, by using an appropriately tuned language model and specialized knowledge base, it is possible to quickly and efficiently create detailed screening work documents such as industry trends, SWOT analysis, and management issues from the perspective of the management. In other words, the screening work document creation support system of this embodiment can support the creation of screening work documents by financial institution personnel.

[0075] [Embodiment 2] In the above-mentioned embodiment 1, it is possible to quickly and efficiently create a business feasibility evaluation sheet (enter answers to sheet items) by inputting answers from the language model 40 (so-called generative AI) into the sheet items of the business feasibility evaluation sheet. In embodiment 2, in creating a business feasibility evaluation sheet, repeated dialogue between the AI ​​and a person (e.g., a financial institution employee) is carried out through an interactive user interface, so that human knowledge and insight are further reflected in the AI's answers, thereby further enriching the content or improving the accuracy of the business feasibility evaluation sheet (sheet items). This will be explained in detail below.

[0076] <System configuration> Fig. 11 is a diagram showing an example of the configuration of a screening work document preparation support system 100 according to this embodiment. The screening work document preparation support system 100 in Fig. 11 is similar to Fig. 1, and includes a financial institution system 10, a screening work document preparation support server 20, a knowledge base 30, a fine-tuned large-scale language model (LLM) 40, and a terminal 50, which are connected via a network 70.

[0077] The support server 20 according to this embodiment provides an interactive UI screen to the terminal 50 for the person in charge when creating a business feasibility evaluation sheet. Through the interactive UI screen on the terminal 50, the person in charge can go through a process of repeatedly interacting with the AI ​​(inputting questions based on human knowledge and insight, and outputting answers by the AI), confirming the content of the AI's answers to the sheet items one by one, and inputting answers that the person in charge is satisfied with into the corresponding sheet item. The person in charge can also correct the AI's answers to the sheet items based on the person in charge's knowledge and insight, and input the corrected answers into the sheet items.

[0078] Furthermore, when there is sufficient customer relationship information in the knowledge base 30, an appropriate answer can be obtained from the language model 40, and the support server 20 can input appropriate answers to the sheet items of the business feasibility evaluation sheet. However, when there is no customer relationship information, or the information is insufficient or inaccurate in the knowledge base 30, an appropriate answer cannot be obtained for the sheet items. In this case, the person in charge can reflect knowledge based on the person's knowledge and insight in the knowledge base 30 through an interactive UI screen. This allows the knowledge base 30 to be filled with the information necessary for an answer, making it possible to obtain an appropriate AI answer. In this embodiment, the AI ​​refers to a conversation partner from the perspective of the person in charge, and directly includes the support server 20 and indirectly includes the language model 40. The AI's response corresponds to a response generated by the support server 20 and / or the language model 40.

[0079] (Functional configuration) Fig. 12 is a diagram showing an example of a functional configuration of the examination work document creation support server 20 according to this embodiment. The support server 20 has, as main functional units, a document creation request receiving unit 201, an item identifying unit 202, a question generating unit 203, a searching unit 204, a question transmitting unit 205, an answer receiving unit 206, an answer input unit 207, a document output unit 208, and a storage unit 209, as well as a UI providing unit 211, a question input unit 212, an answer output unit 213, a corrected answer input unit 214, an instruction input unit 215, a knowledge input unit 216, a knowledge reflecting unit 217, and a natural language processing unit 218, all of which are shown in Fig. 4.

[0080] The UI providing unit 211 has a function of providing an interactive UI screen.

[0081] The question input unit 212 has a function of allowing the user to input questions regarding input items provided in the format of the review work document via an interactive UI screen.

[0082] The answer output unit 213 has a function of outputting an answer sentence corresponding to a question sentence to a user based on answer information from the language model 40 via an interactive UI screen.

[0083] The corrected answer input unit 214 has a function of inputting an answer sentence corrected by the user via a dialogue-style UI screen.

[0084] The instruction input unit 215 has a function of inputting an instruction for inputting a response sentence to an input item from the user via a dialogue-type UI screen.

[0085] The knowledge input unit 216 has a function of allowing the user to input question-related information via an interactive UI screen.

[0086] The knowledge reflection unit 217 has a function of reflecting in the knowledge base 30 the question-related information input by the user.

[0087] The natural language processing unit 218 has the function of performing natural language processing on text sentences (question sentences, revised answer sentences, input instructions, question-related information, answer sentences, etc.) input and output via an interactive UI screen, and realizing an interactive UI between the user and AI.

[0088] <UI screen for creating business feasibility evaluation sheet> Fig. 13 is a diagram showing an example of a UI screen for sheet item "2. Business content" according to this embodiment. The person in charge uses the terminal 50 to access the support server 20, and when creating a business feasibility evaluation sheet, the UI screen shown in Fig. 13 is displayed on the screen. The person in charge can create a business feasibility evaluation sheet while repeatedly interacting with the AI ​​through the dialogue-style UI screen and further reflecting human knowledge and insight in the AI's answers.

[0089] When the person in charge inputs a text sentence in an input field 501 on the UI screen, the support server 20 displays the input text sentence as the person in charge's dialogue. For example, when the person in charge wants to get an answer to sheet item "2. Business content", the person in charge inputs a question 502 for requesting an answer to sheet item "2. Business content" in the input field 501. The text sentence input in the input field 501 may be a voice input, and may be converted into text eventually.

[0090] When a question 502 is input by the person in charge, the support server 20 generates a question to answer the sheet item "2. Business content" (S3 in FIG. 6), searches and acquires question-related information similar to the question from the knowledge base 30 (S4 in FIG. 6), queries the language model 40 for the question and the searched and acquired question-related information (S5 in FIG. 6), receives an answer from the language model 40 (S21 in FIG. 7), and displays the received answer as an AI answer 503. In this way, on the UI screen, text showing the question input by the person in charge and text showing the answer generated by the AI ​​(language model 40) are displayed in an interactive format, so that the person in charge can carefully check the contents of the output AI answer one by one.

[0091] Here, if the knowledge base 30 does not contain, or the customer relationship information related to the question 502 is insufficient or inaccurate, the language model 40 cannot generate an appropriate answer, and the AI ​​displays a response to that effect. For example, in the UI screen shown in FIG. 13, in response to the question 502 by the person in charge, "Please summarize the branches and sales offices of Company A," since there is no customer information related to the branches and sales offices of Company A in the knowledge base 30, the language model 40 cannot generate an appropriate response regarding the branches and sales offices of Company A, and the AI ​​displays a response 504 saying, "I'm sorry, but there is not enough information regarding the branches and sales offices of Company A. Please check."

[0092] When the person in charge receives this response 504 and is able to obtain the information, for example by searching for information on the branch or sales office of Company A, the person in charge additionally inputs the obtained information on the branch or sales office into the input field 501. This additionally input information is, so to speak, the person in charge's knowledge and insight obtained based on, for example, daily sales activities. As a result, the customer information on the branch or sales office of Company A is also reflected in the customer information DB and knowledge base 30, and after the reflection, the language model 40 becomes able to generate an appropriate response. For example, in the UI screen shown in FIG. 13, in response 505 of the person in charge, "Company A has a branch office in Osaka," the customer information on the branch or sales office of Company A is reflected in the customer information DB and knowledge base 30, and the language model 40 becomes able to generate an appropriate response regarding the branch or sales office of Company A, so that the AI ​​displays a response 506 saying, "Thank you. The customer information of Company A has been updated. Once again, Company A has a branch or sales office in Osaka."

[0093] Then, the person in charge who receives this answer 506 responds by saying, for example, "Enter in business details" 507, and the AI ​​displays a response 508 saying, "We will enter this content in 'Business details'." The support server 20 then compiles the answer sentences received from the language model 40 and inputs (transcribes) them into the sheet item "2. Business details" of the business feasibility evaluation sheet.

[0094] In addition, on the UI screen, a selection button 509 of "appropriate" and "inappropriate" is provided for the person in charge to assist the person in the "evaluation of the AI ​​answer" operation. By pressing the selection button of "appropriate" or "inappropriate" as appropriate during the dialogue, the person in charge can feed back to the support server 20 and the language model 40 whether the AI ​​answer (the answer generated by the language model 40) is appropriate or not. In this case, for example, the language model 40 learns the AI ​​answer that is determined to be "appropriate" as a desirable "answer" from the next time onwards, and learns the AI ​​answer that is determined to be "inappropriate" as an undesirable "answer" from the next time onwards. This makes it possible to construct a language model 40 that can generate more specialized answer sentences required for screening work documents by further reflecting human knowledge and insight in the AI ​​answers.

[0095] The UI screen also provides selection buttons 510 for "Enter into business details," "Ask another question," and "Modify" to assist the person in the "Input sheet items" operation. If the person in charge judges that the AI ​​answer obtained through the dialogue may be entered into the sheet item "Business details," he or she presses the selection button for "Enter into business details." In response to this input instruction, the support manager 20 responds 508, "We will enter these details into 'Business details'," and can input (transcribe) the AI ​​answer (the answer generated by the language model 40) into the sheet item "2. Business details" of the business feasibility evaluation sheet.

[0096] On the other hand, if the person in charge wants to switch to a different question, for example, without entering (transcribe) the AI's answer into sheet item "2. Business content," they can reset the AI's answer that has been given so far by pressing the "Ask a different question" selection button.

[0097] If the person in charge who receives the response 504 is able to obtain the information, for example by looking up information on the branch or sales office of Company A, the person in charge may directly add the information on the branch or sales office to the customer information of Company A in the customer information DB. In this case, the person in charge can make a response 505 saying "The customer information of Company A has been updated in the DB" on the UI screen shown in FIG. 13. In this case, the customer information on the branch or sales office of Company A is reflected in the customer information DB and knowledge base 30, so that language model 40 can similarly generate an appropriate response 506 in response to response 505.

[0098] 14A is a diagram showing a UI screen example 1 for the sheet item "6. SWOT analysis" according to this embodiment. Next, for example, if the person in charge wants to get an answer to "S (strengths)" of the sheet item "6. SWOT analysis", the person in charge inputs a question 511 for requesting an answer to "S (strengths)" of Company A in the input field 501.

[0099] When a question 511 is input by a person in charge, the support server 20 generates a question to answer "S (strengths)" of the sheet item "6. SWOT analysis" in question (S3 in FIG. 6), searches and acquires question-related information similar to the question from the knowledge base 30 (S4 in FIG. 6), queries the question and the searched and acquired question-related information to the language model 40 (S5 in FIG. 6), receives the answer from the language model 40 (S21 in FIG. 7), and displays the received answer as the AI's answer 512.

[0100] Here, in order to further enrich the content entered (transcribed) in "S (strengths)" of the sheet item "6. SWOT analysis", the person in charge enters, for example, a question 513, "Please tell us the points of focus in SWOT analysis" in the input field 501. The intention is to have the AI ​​answer the points of focus when conducting a SWOT analysis (business feasibility evaluation), and to clarify the S (strengths) in Company A's SWOT analysis from the perspective of those points of focus.

[0101] When a question 513 is input by the person in charge, the support server 20 generates a question to answer the viewpoint in SWOT analysis (S3 in FIG. 6), searches and acquires question-related information similar to the question from the knowledge base 30 (S4 in FIG. 6), queries the language model 40 for the question and the searched and acquired question-related information (S5 in FIG. 6), receives an answer from the language model 40 (S21 in FIG. 7), and displays the received answer as an AI answer 514. As described above, the knowledge base 30 stores screening dictionary information that covers all industries and all business types in Japan and summarizes industry trends, information analysis, screening points, etc. for each business type. The support server 20 can search and acquire information on "viewpoints in business feasibility evaluation" from the screening dictionary information corresponding to the business type of Company A from the knowledge base 30 as question-related information, and input the question and the searched and acquired "viewpoints in business feasibility evaluation" information to the language model 40 as a question.

[0102] The support server 20 receives the answer from the language model 40 and displays an answer 514 such as, "Even if it is a small or medium-sized manufacturer, it is important whether it can boast of its own technical equipment, whether it is a company that holds patent rights, and whether it is working to develop wholesale routes for electrical materials."

[0103] At this time, the UI screen displays, in the form of selection buttons 515, keywords of business feasibility evaluation extracted and generated based on the answers 514 by the support server 20. For example, "What is Company A's unique technology?", "How many patent applications has Company A filed?", and "Who are Company A's main clients?" are all important topics when evaluating the business feasibility of Company A's industry. By pressing the selection button for "How many patent applications has Company A filed?" among these, the person in charge can input a question 516 of "Please tell me how many patent applications has Company A filed," as a topic that the person in charge wants to confirm regarding the SWOT analysis (business feasibility evaluation) of Company A.

[0104] When a question 516 is input by the person in charge, the support server 20 generates a question to answer the question 516, searches and acquires question-related information similar to the question from the knowledge base 30, queries the language model 40 about the question and the searched and acquired question-related information, receives an answer from the language model 40, and displays the received answer 517. Starting from the viewpoint of business feasibility evaluation generated from the examination dictionary information, the person in charge further engages in dialogue of questions and answers to obtain a more detailed answer regarding the sheet item in question.

[0105] Here, for example, if knowledge data related to question 516 does not exist in knowledge base 30, or is insufficient or inaccurate, language model 40 cannot generate an appropriate answer, and the AI ​​displays an answer to that effect. For example, in the UI screen shown in FIG. 14A, in response to question 516 by the person in charge, "Please tell me the number of patent applications of Company A," since there is no information on the number of patent applications of Company A or external reference materials (external information sources) in knowledge base 30, language model 40 cannot generate an appropriate answer regarding the number of patent applications of Company A, and the AI ​​displays answer 517, "I'm sorry. There is no information on the number of patents of Company A in the database. If there is a database you would like, please enter the URL in the source field."

[0106] When the person in charge receives this response 517 and is able to obtain the information source by, for example, checking the URL (Uniform Resource Locator) of the database regarding the number of patents of Company A, the person in charge performs additional registration of the external information source in the knowledge base DB30 (API addition request for the external information source). The response 518 of the person in charge, "https: / / www...", is, so to speak, the knowledge and insight of the person in charge obtained based on, for example, daily sales activities. As a result, the URL indicating the location of the database regarding the number of patents of Company A is reflected in the knowledge base 30, and after the reflection, the language model 40 can generate an appropriate response. For example, in the UI screen shown in FIG. 14A, in response to the response 518 of the person in charge, "https: / / www...", the AI ​​displays a response 519 saying "Thank you. Please wait a while until it becomes available."

[0107] 14B is a diagram showing a UI screen example 2 for the sheet item "6. SWOT analysis" according to this embodiment. The person in charge again inputs a question 521, for example, "Please tell me the number of patent applications filed by Company A."

[0108] When a question 521 is input by a person in charge, the support server 20 generates a question to answer the question 521, searches and acquires question-related information similar to the question from the knowledge base 30 (for example, a URL indicating the location of a database regarding the number of patents of Company A), queries the language model 40 for the question and the searched and acquired question-related information, receives a response from the language model 40, and displays the received response 522.

[0109] After the URL of the database related to the number of patents of Company A is reflected in the knowledge base 30, the language model 40 can access the URL and generate an appropriate answer. For example, in the UI screen shown in FIG. 14B, in response to a question 521 from the person in charge, the AI ​​displays a response 522 saying, "According to the patent information xxx form, the number of patent applications of Company A is XX." Furthermore, since the URL of the database reflected in the knowledge base 30 is capable of acquiring information on the number of patent applications of companies other than Company A, in response to a question 523 from the person in charge, "Please tell me the number of patent applications of competitors," the AI ​​displays a response 524 saying, "According to the same information source, the number of patent applications of Company B is XX, and the number of patent applications of Company C is XX."

[0110] Next, if the person in charge wants to revise the AI ​​answer obtained so far (S (strengths) in Company A's SWOT analysis), they can press the "revise" selection button to issue a response 525 saying "revise," followed by, for example, a revised statement such as "·S (strengths): The company's DX products for production factories have rapidly expanded their industry share since their release, and are contributing greatly to the company's sales. The management has set a goal of capturing the top share in the industry within three years, and it is expected that this will be achieved.

[0111] According to the patent information xxx form, Company A has filed XX patent applications. According to the same source, Company B has filed XX patent applications, and Company C has filed XX patent applications. Compared to other companies, the number of patent applications is overwhelming, indicating a strong desire for research and development. Currently, Company A is planning bold cost-cutting measures, which are expected to improve profit margins. " Enter amendment 526.

[0112] The additional revised sentence "Compared to other companies, the number of patent applications is overwhelming, which shows the high motivation for research and development" is, so to speak, the knowledge and insight of the person in charge that is reflected based on the dialogue from question 521 to answer 524. Also, the additional revised sentence "Currently, bold cost-cutting measures are planned at Company A, and an improvement in profit margins is expected" can also be considered to be the knowledge and insight of the person in charge that is obtained based on, for example, daily sales activities.

[0113] Furthermore, when the person in charge presses the selection button "Enter into S (strengths)", a response "Enter into strengths" 527 is issued, and the AI ​​displays a response 528 saying "We will enter this content into 'S (strengths')." The support server 20 can then enter (transcribe) the corrected AI response into "strengths" in the sheet item "6. SWOT analysis" of the business feasibility evaluation sheet.

[0114] In addition to "Enter in S (Strengths)," the UI screen shown in FIG. 14B has a selection button 510 for "Enter in W (Weaknesses)." For example, even in a dialogue process where a question is asked to obtain the strengths of Company A, if the AI ​​answer obtained is not necessarily a strength, but rather a weakness judged by the person in charge, the selection button for "Enter in W (Weaknesses)" can be pressed to input (transcribe) the AI ​​answer (the answer generated by the language model 40) into "W (Weaknesses)" of the sheet item "6. SWOT analysis" of the business feasibility evaluation sheet. Specifically, for example, this is the case when an answer is obtained to the effect that the number of patent applications filed by Company A is smaller than the number of patent applications filed by competitors.

[0115] 15 is a diagram showing a UI screen example 3 for the sheet item "6. SWOT analysis" according to this embodiment. Next, for example, if the person in charge wants to get an answer to "W (weakness)" of the sheet item "6. SWOT analysis", the person in charge inputs a question 531 for requesting an answer to "W (weakness)" of Company A in the input field 501.

[0116] When a question 531 is input by a person in charge, the support server 20 generates a question to answer "W (weakness)" of the sheet item "6. SWOT analysis" in question (S3 in FIG. 6), searches and acquires question-related information similar to the question from the knowledge base 30 (S4 in FIG. 6), queries the question and the searched and acquired question-related information to the language model 40 (S5 in FIG. 6), receives an answer from the language model 40 (S21 in FIG. 7), and displays the received answer as the AI's answer 532.

[0117] Here, in order to further enrich the content to be entered (transcribed) into "W (Weakness)" of the sheet item "6. SWOT Analysis", the person in charge enters, for example, a question 533 such as "Please tell us your viewpoints regarding the company's cash flow." The intention is to have the AI ​​answer the viewpoints when evaluating the company's cash flow, and to clarify whether the viewpoints could be W (weakness) in Company A's SWOT analysis.

[0118] When a question 533 is input by a person in charge, the support server 20 obtains information on "perspectives on a company's cash flow" from the review dictionary information corresponding to Company A's industry, embeds the obtained "perspectives on a company's cash flow" together with question 533 in a prompt, and inputs it as a question to the language model 40.

[0119] The support server 20 receives the answer from the language model 40 and displays an answer 534 such as, "Is the proportion of payments made by bills payable high? Have collection terms become longer? Has trading company financing occurred, such as the payment of advances to suppliers or customers? Are the financial burdens excessive?"

[0120] Also, on the UI screen at this time, keywords of key points for cash flow extracted and generated based on answers 534 are displayed in the form of selection buttons 535. For example, "What is the settlement ratio of Company A's bills payable?", "What are Company A's collection conditions?", and "What is Company A's trading company financing?" are all important points of interest when evaluating a company's cash flow. By pressing a selection button for "What is the settlement ratio of Company A's bills payable?" as a point of interest that the person in charge wants to confirm regarding Company A's cash flow, the person in charge can input a question 536 of "Please tell me the settlement ratio of Company A's bills payable."

[0121] When a question 536 is input by the person in charge, the support server 20 generates a question to answer the question 536, searches and acquires question-related information similar to the question from the knowledge base 30, queries the language model 40 about the question and the searched and acquired question-related information, receives an answer from the language model 40, and displays the received answer 537. Starting from the viewpoint of business feasibility evaluation generated from the examination dictionary information, the person in charge further engages in dialogue of questions and answers to obtain a more detailed answer regarding the sheet item in question.

[0122] When the person in charge presses the selection button "Enter into W (Weakness)", the AI ​​displays a response 539 saying "We will enter this content into 'W (Weakness)'." The support server 20 can then enter (transcribe) this AI response into the "Weakness" field of the sheet item "6. SWOT analysis" of the business feasibility evaluation sheet.

[0123] <Summary> According to this embodiment, the following effects are obtained. When creating a business feasibility assessment sheet, the financial institution staff member can repeatedly interact with the AI ​​through an interactive UI screen, confirm the content of each of the AI's answers regarding the sheet items, and then input the answers that the staff member is satisfied with into the corresponding sheet item (507, 527, 538) to reflect this.

[0124] In addition to the AI ​​answers (512, 531) obtained for the sheet items, the financial institution staff can obtain further AI answers (514, 522, 524) for the sheet items by the process of asking questions (513, 516, 521, 523, 536) based on the staff's knowledge and insight.

[0125] In addition, the financial institution staff can correct the AI ​​answers obtained for the sheet items based on the staff's knowledge and insight (525, 526) and input the corrected answers into the corresponding sheet items (527).

[0126] Furthermore, if the knowledge data in the knowledge base 30 does not exist, is insufficient, or is inaccurate, the support server 20 cannot obtain appropriate answers (504, 517) for the sheet items from the language model 40. Through an interactive UI screen, the AI ​​prompts the financial institution staff to add the missing knowledge data, and the staff can add the missing knowledge data to the knowledge base 30 based on their own knowledge and insight (505, 518). Furthermore, by enriching or improving the accuracy of the knowledge base 30, it is expected that the accuracy and efficiency of the subsequent creation of business feasibility evaluation sheets will be improved.

[0127] According to the present embodiment, when creating a business feasibility assessment sheet, by having the AI ​​and a person (e.g., a financial institution employee) interact with each other through an interactive user interface, the AI's answers can be further influenced by human knowledge and insight, thereby further enhancing the content or improving the accuracy of the business feasibility assessment sheet (sheet items).

[0128] Although the present invention has been described by showing specific examples according to the preferred embodiment of the present invention, it is apparent that various modifications and changes can be made to these examples without departing from the broad spirit and scope of the present invention as defined in the claims. In other words, the details of the examples and the accompanying drawings should not be construed as limiting the present invention.

[0129] The business feasibility evaluation sheet according to this embodiment is an example of an examination work document. The examination work document created by the support server 20 may be a loan approval document for loan examination. In addition, each financial institution may call the business feasibility evaluation sheet by various names such as "company overview", "examination table", "management issue sharing sheet", "hearing sheet", etc.

[0130] (Additional Note) An examination business document creation support device (examination business document creation support server 20) in a financial institution (bank, etc.), A UI providing means for providing an interactive UI screen (UI providing unit 211); A question input unit (212) for inputting a question regarding an input item (answer input item) provided in the format of the examination business document (business feasibility evaluation sheet) via the UI screen; A question generation unit (203, S3) for generating question information (prompt) for causing a language model (fine-tuned large-scale language model 40) to answer the question sentence; A question sending means for sending the question information to the language model (question sending unit 205, S5); an answer receiving means for receiving answer information corresponding to the question information from the language model (answer receiving unit 206, S21); an answer output means for outputting an answer corresponding to the question sentence based on the answer information via the UI screen (answer output unit 213); an instruction input means for inputting an instruction to input the answer sentence for the input item via the UI screen (instruction input unit 215); an answer input unit for inputting input information based on the answer sentence into the input item in response to the input instruction (answer input unit 207, S22); An examination document creation support device comprising: [Explanation of symbols]

[0131] 10 Financial Institutions System 20 Review document creation support server 30 Knowledge Base 40 Fine-tuned Large-Scale Language Models 50 terminals 70 Network 100 Examination Document Creation Support System 201 Document Creation Request Reception Unit 202 Item Specification Section 203 Question generation section 204 Search Department 205 Question Sending Department 206 Response Receiving Department 207 Answer input section 208 Document Output Unit 209 Memory section 211 UI provision department 212 Question input section 213 Answer output section 214 Corrected Answer Input Section 215 Instruction input section 216 Knowledge Input Section 217 Knowledge Reflection Department 218 Natural Language Processing Unit

Claims

1. A document preparation support device for examination operations in a financial institution, comprising: A UI providing means for providing an interactive UI screen; A question input means for inputting a question regarding an input item provided in a format of the review work document via the UI screen; a question generation means for generating question information for causing a language model to answer the question sentence; a question sending means for sending the question information to the language model; an answer receiving means for receiving answer information corresponding to the question information from the language model; an answer output means for outputting an answer corresponding to the question based on the answer information via the UI screen; an instruction input means for inputting an instruction to input the answer sentence for the input item via the UI screen; an answer input means for inputting input information based on the answer sentence into the input field in response to the input instruction; An examination document creation support device comprising:

2. The answer input means is A first question sentence is input, and a first answer sentence corresponding to the first question sentence is output; A second question sentence is input, and a second answer sentence corresponding to the second question sentence is output, and then: When the input instruction is input, inputting input information based on the first answer sentence and the second answer sentence into the input item; 2. The examination document creation support device according to claim 1,

3. The input items are input items related to SWOT analysis of the customer, Information regarding the focus of the SWOT analysis based on the examination dictionary information is output via the UI screen, The question is a question for causing the language model to answer a point of view of the SWOT analysis; 3. The examination document creation support device according to claim 1 or 2,

4. a corrected answer input means for inputting the answer sentence corrected by a user via the UI screen, The answer input means is After a question is input and an answer corresponding to the question is output, When the input instruction for the corrected answer is input, inputting input information based on the corrected answer sentence into the input item; 2. The examination document creation support device according to claim 1,

5. a search means for searching a knowledge base for question-related information related to the generated question information; the question sending means for sending the question information and the question-related information to the language model; a knowledge input means for inputting the question-related information via the UI screen; a knowledge reflecting means for reflecting the input question-related information in the knowledge base; 2. The examination document preparation support device according to claim 1, further comprising:

6. The input question-related information is location information of an external information source; 6. The examination document creation support device according to claim 5,

7. A document preparation support device for financial institutions a UI providing step for providing an interactive UI screen; a question input step of inputting a question regarding an input item provided in a format of the review work document via the UI screen; a question generation step of generating question information for causing a language model to answer the question sentence; a question transmission step of transmitting the question information to the language model; an answer receiving step of receiving answer information corresponding to the question information from the language model; an answer output step of outputting an answer corresponding to the question based on the answer information via the UI screen; an instruction input step of inputting an instruction to input the answer sentence for the input item via the UI screen; a response input step of inputting input information based on the response sentence into the input field in response to the input instruction; A method for supporting the preparation of review work documents to carry out the above.

8. Computer, A UI providing means for providing an interactive UI screen; A question input means for inputting a question regarding an input item provided in a format of the review work document via the UI screen; a question generation means for generating question information for causing a language model to answer the question sentence; a question sending means for sending the question information to the language model; an answer receiving means for receiving answer information corresponding to the question information from the language model; an answer output means for outputting an answer corresponding to the question based on the answer information via the UI screen; an instruction input means for inputting an instruction to input the answer sentence for the input item via the UI screen; an answer input means for inputting input information based on the answer sentence into the input field in response to the input instruction; This is a program to assist in the creation of review documents to make the system function as a review process.

Citation Information

Patent Citations

  • Manuscript generation method and device, equipment and storage medium

    CN116741178A

  • Examination work document creation support device, examination work document creation support method, and examination work document creation support program

    JP7396582B1

  • JPP7396582B

  • JPP7406031B

  • Document auto-completion

    US20240005089A1

Cited By

  • Program, method, information processing apparatus, and system

    JP2026093191A