Program, method, information processor and system

Abstract

To provide a program, a method, an information processor and a system for outputting an appropriate reply sentence to a question sentence.SOLUTION: A method accepts a question sentence from a user; obtains determination results on whether the question sentence is appropriate as input data for a question reply system from which a reply sentence can be output upon an entry of the question sentence, by inputting the question sentence that is accepted into a large-scale language model as input data; and presents to a user the reply sentence to the question sentence that is output as a result of an entry of input data based on the question sentence into the question reply system, if the determination result that the question sentence is appropriate is obtained; or presents to the user a reply sentence for digging deeper into the question sentence instead of the reply sentence to the question sentence that is output as a result of an entry of input data based on the question sentence into the large-scale language model, if the determination result that the question sentence is appropriate is not obtained.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present disclosure relates to a program, a method, an information processing device, and a system. [Background technology]

[0002] 2. Description of the Related Art Systems (chatbot systems) that respond to online inquiries such as questions from users and output answers corresponding to those questions are known. Patent Document 1 discloses a technology that enables a chatbot to automatically respond to inquiries from users and, when transferring over to a human responder, seamlessly hand over the response to the respondent. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-189237 A Summary of the Invention [Problem to be solved by the invention]

[0004] There is a problem in that it is not possible to output an appropriate answer to a question. Therefore, the present disclosure has been made to solve the above problem, and has an object to provide a technology for outputting an appropriate answer to a question. [Means for solving the problem]

[0005] A program to be executed by a computer having a processor and a memory unit, the program executing the following steps: a question receiving step in which the processor receives a question from a user; a judgment step in which the question received in the question receiving step is input into a large-scale language model as input data, thereby obtaining a judgment result as to whether the question is appropriate as input data for a question answering system capable of outputting an answer in response to the input of the question; a first answering step in which, if the judgment result that the question is appropriate is obtained in the judgment step, an answer sentence to the question that is output by inputting input data based on the question sentence into the question answering system is presented to the user; and, if the judgment result that the question is appropriate is not obtained in the judgment step, a second answering step in which the program presents to the user a reply sentence for digging deeper into the question sentence, rather than an answer sentence to the question sentence, that is output by inputting the input data based on the question sentence into the large-scale language model. Effect of the Invention

[0006] According to the present disclosure, it is possible to output an appropriate answer to a question. [Brief description of the drawings]

[0007] [Figure 1] FIG. 2 is a block diagram showing the functional configuration of the system 1. [Diagram 2] FIG. 2 is a block diagram showing the functional configuration of the server 10. [Diagram 3] 2 is a block diagram showing the functional configuration of a user terminal 20. FIG. [Figure 4] FIG. 13 is a diagram showing the data structure of a user table 1012. [Diagram 5] FIG. 13 is a diagram showing the data structure of a question table 1013. [Figure 6] FIG. 13 is a diagram showing the data structure of a keyword table 1014. [Figure 7] FIG. 13 is a diagram showing the data structure of a candidate table 1015. [Figure 8] FIG. 13 is a diagram showing the data structure of a response table 1016. [Figure 9] 13 is a flowchart showing the operation of a search process. [Figure 10] 13 is a flowchart showing an operation of an interaction process. [Figure 11] 13 is a flowchart showing the operation of a proposal process. [Figure 12] 1 is an example of the first screen of a dialogue screen. [Figure 13] 13 is a second example of a dialogue screen. [Figure 14] FIG. 2 is a block diagram showing the basic hardware configuration of a computer 90. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In all the drawings explaining the embodiment, the same reference numerals are given to common components, and repeated explanations are omitted. Note that the following embodiment does not unduly limit the contents of the present disclosure described in the claims. In addition, not all of the components shown in the embodiment are essential components of the present disclosure. In addition, each figure is a schematic diagram and is not necessarily illustrated strictly.

[0009] <System 1 Configuration> The system 1 in the present disclosure is an information processing system that provides an information processing service (FAQ service) that outputs suitable answers to questions input by a user. For example, in a bank or the like, the system provides an information processing service that receives questions from users regarding account opening, domestic and international remittances, loan applications, ATMs, how to use cash cards, how to use mobile banking, etc., and provides the users with answers to solve their doubts and problems related to the questions. The application of the information processing service according to the present disclosure is not limited to banks, but can be applied to any business entity, such as public transportation such as railways, airplanes, and taxis, accommodation facilities such as hotels and guesthouses, software services, hospitals, etc. The system 1 includes an information processing device of a server 10, a user terminal 20, a generating AI 50, and a question and answer system 60, which are connected via a network N. FIG. 1 is a block diagram showing the functional configuration of the system 1. FIG. 2 is a block diagram showing the functional configuration of the server 10. As shown in FIG. FIG. 3 is a block diagram showing the functional configuration of the user terminal 20. As shown in FIG.

[0010] Each information processing device is configured by a computer equipped with an arithmetic unit and a storage device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by the hardware configuration will be described later. For each of the server 10, the user terminal 20, the generation AI 50, and the question and answer system 60, descriptions that overlap with the basic hardware configuration and basic functional configuration of the computer described later will be omitted.

[0011] <Server 10 Configuration> The server 10 is an information processing device that provides an information processing service (FAQ service) that outputs suitable answers to questions input by users by communicating with a generation AI 50, a question and answer system 60, and the like, which will be described later. The server 10 includes a storage unit 101 and a control unit 104 .

[0012] <Configuration of the storage unit 101 of the server 10> The storage unit 101 of the server 10 includes an application program 1011 , a user table 1012 , a question table 1013 , a keyword table 1014 , a candidate table 1015 , and an answer table 1016 .

[0013] The application program 1011 is a program for causing the control unit 104 of the server 10 to function as each functional unit. Application programs 1011 include applications such as a web browser application.

[0014] User table 1012 is a table that stores and manages information about member users (hereinafter, users) who use the service. When a user registers to use the service, the user's information is stored in a new record in user table 1012. This allows the user to use the service according to the present disclosure. The user table 1012 is a table having columns of user ID and user name, with the user ID as the primary key. FIG. 4 is a diagram showing the data structure of the user table 1012. As shown in FIG.

[0015] The user ID is an item for storing user identification information for identifying a user. The user identification information is an item for which a unique value is set for each user. The user name is an item for storing the name of the user. The user name may be set to any character string such as a nickname instead of a name.

[0016] The question table 1013 is a table for storing and managing information relating to questions (question information). The question table 1013 is a table having a question ID as a primary key, and columns of a question ID, a parent question ID, a user ID, a question text, and a date and time. FIG. 5 is a diagram showing the data structure of the question table 1013. As shown in FIG.

[0017] The question ID is an item for storing question identification information for identifying a question. The question identification information is an item for which a unique value is set for each piece of question information. The parent question ID is an item for storing parent question identification information for identifying a parent question. The user ID is an item for storing user identification information for identifying a user. The question sentence is an item for storing character string information (text information) related to the question sentence. The date and time is an item for storing the date and time when a record is newly stored or updated in the question table 1013.

[0018] The keyword table 1014 is a table for storing and managing information related to search keywords (search keyword information). The keyword table 1014 is a table having columns for question ID, input sentence, keyword, and date and time. FIG. 6 is a diagram showing the data structure of the keyword table 1014. As shown in FIG.

[0019] The question ID is an item for storing question identification information for identifying a question. The input sentence is an item that stores character string information (text information) related to the input sentence to the generation AI 50. The keyword is an item that stores character string information (text information) related to a search keyword. The keyword stores character string information related to one or more search keywords. The date and time is an item for storing the date and time when a record is newly stored or updated in the keyword table 1014.

[0020] The candidate table 1015 is a table for storing and managing information related to candidates (candidate information). The candidate table 1015 is a table having columns for question ID, input sentence, question candidate, and date and time. FIG. 7 is a diagram showing the data structure of the candidate table 1015. As shown in FIG.

[0021] The question ID is an item for storing question identification information for identifying a question. The input sentence is an item that stores character string information (text information) related to the input sentence to the generation AI 50. The question candidate is an item that stores character string information (text information) related to the question candidate. The question candidate stores character string information related to one or more question candidates. The date and time is an item for storing the date and time when a record is newly stored in the candidate table 1015 or updated.

[0022] The answer table 1016 is a table for storing and managing information related to answers (answer information). The answer table 1016 is a table having columns for question ID, processing type, input text, answer text, and date and time. FIG. 8 is a diagram showing the data structure of the answer table 1016. As shown in FIG.

[0023] The question ID is an item for storing question identification information for identifying a question. The processing type is an item for storing the type of processing executed to obtain the answer sentence. Specifically, the process type stores information indicating any one of an interactive process, a search process, and a proposal process. The input sentence is an item for storing character string information (text information) related to a sentence input to the generation AI 50 or the question and answering system 60. The answer sentence is an item for storing character string information (text information) related to the output sentence output from the generation AI 50 or the question and answering system 60. The date and time is an item for storing the date and time when a record is newly stored or updated in the answer table 1016.

[0024] <Configuration of the control unit 104 of the server 10> The control unit 104 of the server 10 includes a user registration control unit 1041, an AI control unit 1042, and a question answering unit 1043. The control unit 104 executes an application program 1011 stored in the storage unit 101 to realize each functional unit.

[0025] The user registration control unit 1041 performs processing to store, in the user table 1012, information on users who wish to use the service according to the present disclosure. The information stored in the user table 1012 is generated when a user opens a web page operated by a service provider from any information processing terminal, enters information into a specific input form, and transmits the information to the server 10. The user registration control unit 1041 stores the received information in a new record in the user table 1012, completing the user registration. This allows the user stored in the user table 1012 to use the service. Before the user registration control unit 1041 registers user information in the user table 1012, the service provider may carry out a predetermined examination to restrict whether or not the user is permitted to use the service. The user ID may be any character string or number that can identify the user, any character string or number desired by the user, or an arbitrary character string or number may be automatically set by the user registration control unit 1041.

[0026] The AI ​​control unit 1042 is a control unit that controls input and output to the generated AI 50. Details will be described later.

[0027] The question answering unit 1043 is a control unit that controls input and output to the question and answering system 60. Details will be described later.

[0028] <Configuration of User Terminal 20> The user terminal 20 is an information processing device operated by a user who uses a service. The user terminal 20 may be, for example, a mobile terminal such as a smartphone or a tablet, a stationary PC (Personal Computer), or a laptop PC. It may also be a wearable terminal such as an HMD (Head Mount Display) or a wristwatch-type terminal. The user terminal 20 includes a storage unit 201 , a control unit 204 , an input device 206 , and an output device 208 .

[0029] <Configuration of the storage unit 201 of the user terminal 20> The storage unit 201 of the user terminal 20 includes a user ID 2011 and an application program 2012 .

[0030] The user ID 2011 is an account ID of the user. The user transmits the user ID 2011 from the user terminal 20 to the server 10. The server 10 identifies the user based on the user ID 2011 and provides the user with the service according to the present disclosure. The user ID 2011 includes information such as a session ID temporarily assigned by the server 10 when identifying the user who is using the user terminal 20.

[0031] The application program 2012 may be stored in advance in the storage unit 201, or may be configured to be downloaded from a web server operated by a service provider via a communication IF. The application programs 2012 include applications such as a web browser application. The application program 2012 includes an interpreted programming language such as JavaScript™ that runs on a web browser application stored on the user terminal 20 .

[0032] <Configuration of the control unit 204 of the user terminal 20> The control unit 204 of the user terminal 20 includes an input control unit 2041 and an output control unit 2042. The control unit 204 executes an application program 2012 stored in the storage unit 201, thereby realizing each functional unit.

[0033] <Configuration of the input device 206 of the user terminal 20> The input device 206 of the user terminal 20 includes a camera 2061 , a microphone 2062 , a position information sensor 2063 , a motion sensor 2064 , and a touch device 2065 .

[0034] <Configuration of the output device 208 of the user terminal 20> The output device 208 of the user terminal 20 includes a display 2081 and a speaker 2082 .

[0035] <Configuration of generated AI50> Generative AI50 refers to large-scale artificial intelligence models used in the field of natural language processing (NLP). These models are trained on large amounts of text data (web pages, books, articles, etc.) to understand patterns in human language and perform natural language generation (NLG) tasks effectively. The generation AI 50 is used in many NLP tasks, such as generating responses to specific questions, automatically generating sentences, summarizing text, translation, and sentiment analysis. It can also be used for a variety of purposes, such as education, entertainment, customer service, and product development. The generation AI 50 includes the following types. In this disclosure, a large-scale language model that mainly outputs text information as output information will be described as a type of generation AI 50. OpenAI ChatGPT ·Google Bard Stable Diffusion Midjourney The generation AI 50 may be realized as a part of the functions of the server 10.

[0036] <Configuration of question answering system 60> The question and answering system 60 is an information processing system that outputs a character string related to an answer in response to an input of a character string related to a question. For example, the question and answering system 60 includes an FAQ system. Specifically, the question and answering system 60 is an information processing system that stores pairs of frequently asked questions from users and their answers (question and answer information). When the question and answering system 60 receives an input of an input character string, it searches for at least one of the questions or answers included in the question and answer information based on the input character string by full-text search or the like, and outputs the question and answer information related to the search result in association with the relevance (score) to the input character string. By inputting a character string such as a search keyword or a question sentence into question answering system 60, it is possible to obtain an output of question answering information related to the character string. There does not need to be just one question answering system 60, and for example, a configuration may be adopted in which a question answering system A storing question answering information for bank A, a plurality of question answering systems 60 are provided for each business operator providing a service related to questions, such as bank B, bank C, accommodation facility A, accommodation facility B, etc. The question and answer system 60 may be realized as a part of the functions of the server 10.

[0037] <System 1 Operation> Each process of the system 1 will be described below. FIG. 9 is a flowchart showing the operation of the search process. FIG. 10 is a flowchart showing the operation of the dialogue processing. FIG. 11 is a flowchart showing the operation of the proposal process. FIG. 12 is an example of the first screen of the dialogue screen. FIG. 13 is a second example of the interactive screen.

[0038] The search process, dialogue process, and suggestion process in this disclosure will be described below. In the information processing system of the present disclosure, a search process, a dialogue process, and a proposal process are each executed for one question received from a user. Note that it is not necessary to execute all of the search process, the dialogue process, and the proposal process for one question received from a user. A configuration may be adopted in which at least one of the search process, the dialogue process, and the proposal process is executed for one question received from a user. Furthermore, the search process, the dialogue process, and the proposal process may be executed in parallel or in sequence. When executed in sequence, depending on the execution result of a specific process, a configuration may be adopted in which some of the processes subsequent to the specific process are not executed (skiped). The start of the search process, the dialogue process, and the proposal process may be executed in parallel, and after each process result is obtained, the presentation of the process results may be executed in sequence. In this case, too, depending on the execution result of a specific process, a configuration may be adopted in which some of the process results subsequent to the specific process are not presented (skiped). For example, in a case where a search process, a proposal process, and an interactive process are executed in this order, the execution of the proposal process and the interactive process may be omitted depending on the processing result of the search process. Similarly, the execution of the interactive process may be omitted depending on the processing result of the proposal process.

[0039] The information processing system disclosed herein finally presents to the user the output result from the question answering system 60 resulting from the search process or the suggestion process. On the other hand, the user is not necessarily able to input an appropriate question. Therefore, until it is determined that the question answering system 60 can provide an appropriate answer to the question input by the user, a response text for specifically digging into the question text, which is generated by the generation AI 50 based on the question text, is presented to the user, thereby encouraging the user to input an additional question text. When it is determined that the question answering system 60 can provide an appropriate answer in response to a sufficient additional question text, the output result from the question answering system 60 is presented to the user.

[0040] In the information processing system of the present disclosure, in response to a question from a user, an answer sentence stored in advance in the question answering system 60 is presented, so that an inappropriate answer sentence is not presented to the user. On the other hand, if a similar process is performed in the conventional generation AI 50, problems such as hallucination (answering content that is different from the facts or content that is unrelated to the context) and generating an answer sentence containing an incorrect content occur. In addition, although the generation AI 50 does not provide an answer to the question received from the user, it can make abstract questions from the user more concrete by presenting the user with a sentence (answer sentence) that includes suggestions for naturally digging deeper into the question received from the user. The information processing system of the present disclosure utilizes generation AI 50 to delve into the content of a question that the user himself has not been able to verbalize, and provides the user with an appropriate answer that does not contain erroneous content and is pre-stored in the question answering system 60 based on the specified question sentence.

[0041] <Search process> The search process is a process of presenting to the user an output statement from the question and answer system 60 based on a search keyword corresponding to a question received from the user.

[0042] <Search process overview> The search process involves accepting a question from a user, inputting the input question into generation AI 50, obtaining search keywords output from generation AI 50, inputting the search keywords into question answering system 60, obtaining the output content output from question answering system 60, and presenting the output content as an answer.

[0043] <Search process details> The search process will be described in detail below.

[0044] The user operates the input device 206 of the user terminal 20 to execute a browser application or the like, and opens the interactive screen D1 by inputting the URL of a web page (interactive screen) for executing a search process. The control unit 204 of the user terminal 20 transmits a request including a user ID 2011 for opening the interactive screen D1 to the server 10.

[0045] When the server 10 receives the request, it generates an interactive screen D1 and transmits it to the user terminal 20. The control unit 204 of the user terminal 20 displays and presents the interactive screen D1 on the display 2081 of the user terminal 20.

[0046] 12 is an example of the first screen of the interactive screen. On the display 2081 of the user terminal 20, an interactive screen D1 is displayed. The dialogue screen D1 includes a question input field D11, questions D121 and D122, and answers D131, D132, D133, D134, and D135.

[0047] 13 is a second screen example of the interactive screen. An interactive screen D2 is displayed on the display 2081 of the user terminal 20. The dialogue screen D2 includes a question input field D21, questions D221, D222, D223, and D224, answers D231, D232, D233, D234, and D235, and question candidates D251 and D252.

[0048] The first and second example screens of the dialogue screen are both examples of screens that present the results of the search process, dialogue process, and suggestion process as answers. Specifically, the first example screen is an example of a screen that appears when the user first inputs "overseas" as a question, and the second example screen is an example of a screen that appears when the user first inputs "account" as a question. Different answers are displayed depending on the content of the question received from the user. In the second screen example, question candidates D251 and D252 are displayed. The question candidates will be described later in the proposal process.

[0049] In step S101, the AI ​​control unit 1042 of the server 10 executes a question receiving step of receiving a question from a user. The user operates the input device 206 of the user terminal 20 to input a question into the input field D11. The control unit 204 of the user terminal 20 transmits the question input into the input field D11 to the server 10. For example, the user can transmit the question input into the input field D11 to the server 10 by pressing an enter key or the like on the user terminal 20. The request transmitted to the server 10 includes a user ID 2011. The control unit 104 of the server 10 receives and accepts the question and user ID 2011. The control unit 104 of the server 10 stores the accepted question and user ID 2011 in the question and user ID items of a new record in the question table 1013, respectively.

[0050] In step S101, the question receiving step includes a step of receiving a first question sentence, and a step of receiving a second question sentence. Specifically, in the search process of the present disclosure, the user can check the answer sentence D132 outputted to the question sentence D121 (first question sentence), and can further send a new additional question sentence D122 (second question sentence) based on the answer sentence D132. In this case, the control unit 104 of the server 10 stores the question ID, question D122, and user ID 2011 in the question table 1013 of the first question, which is the first question of the second question, in the parent question ID, question, and user ID fields of the new record in the question table 1013, respectively.

[0051] In step S101, the question receiving step executes a step capable of receiving an additional question sentence if one or more search keywords cannot be acquired in the keyword acquiring step. Specifically, the control unit 204 of the user terminal 20 may control the question input field D11 to be capable of accepting input operations from the input device 206 of the user terminal 20 when a search keyword cannot be obtained in step S103 described below, and may control the question input field D11 to be unable to accept input operations from the input device 206 of the user terminal 20 when a search keyword can be obtained. In other words, if the search process does not obtain an answer from question and answer system 60, an additional question may be accepted, and if the search process obtains an answer from question and answer system 60, an additional question may not be accepted. When an answer sentence is obtained from the question and answering system 60 by the search process according to the present disclosure, the system may be configured not to accept any further input of a question sentence from the user. This makes it possible to end the search process, dialogue process, and proposal process by presenting the answer sentence from the question and answering system 60 to the user. In other words, when an answer to the search process is obtained, the system may be configured to end the dialogue with the user.

[0052] In step S102, the AI ​​control unit 1042 of the server 10 transmits the question received in the question receiving step to the generation AI 50. Specifically, the AI ​​control unit 50 of the server 10 generates a prompt by inserting the question received in step S101 into the [Question] section of the following prompt template as an input sentence (prompt) for the generation AI 50, which is a large-scale language model, and transmits the generated prompt to the generation AI 50. The prompt template in the present disclosure includes a command statement that instructs an answer to delve deeper into the question without providing an answer to the question from the user. In other words, in the information processing system according to the present disclosure, the generation AI 50 does not provide an answer to the question from the user. In the information processing system according to the present disclosure, an answer to a question from a user is provided based on the answer result (search result) by the question answering system 60. In the search process, the prompt includes an instruction to restrict the output data format of the generation AI 50 to output search keywords instead of answer sentences. Specifically, in OpenAI's GPT, an instruction to extract search keywords to be input to the question answering system 60 is included using a mechanism called Function Calling. [Prompt template] You are a very kind and capable AI assistant. It resides on the FAQ site and supports users' FAQ searches. If the user's question is unclear, ask questions to delve deeper into the issue. Since you are an AI assistant, you should never give users specific solutions to their problems. Once the issue is clarified, we will suggest what search keywords to use to search the FAQ site. #User Questions [Question]

[0053] In step S102, the AI ​​control unit 1042 of the server 10 inputs the first question, the answer to the second question, and the second question to the large-scale language model. Specifically, when the AI ​​control unit 50 of the server 10 receives an additional question D122 in the question receiving step, it generates a prompt by inserting a string combining the question D121, the answer D132, and the question D122 into the [Question] section of the prompt template, and sends the prompt to the generation AI 50. In other words, if one or more question sentences have been received in the past in step S101, and one or more answer sentences have been obtained for the one or more question sentences, a string formed by combining the one or more question sentences and one or more answer sentences with the newly received question sentence is sent to generation AI 50 as a question sentence. The control unit 104 of the server 10 can acquire one or more question sentences included in one or more pieces of question information received in the past by searching the question ID item of the question table 1013 based on the parent question ID included in the acquired one or more pieces of question information. The control unit 104 of the server 10 can acquire any parent generation question information and question sentence by searching the question ID item of the question table 1013 based on the parent question ID included in the acquired one or more pieces of question information. The control unit 104 of the server 10 can obtain one or more answer sentences included in one or more pieces of answer information by searching the items in the answer table 1016 based on one or more question IDs included in one or more pieces of question information previously received. The question information and the answer information can determine their corresponding relationships (order relationship, which answer sentence corresponds to which question sentence) based on their respective question IDs. The order of the one or more answer sentences may also be the order of the dates and times included in the answer information. The control unit 104 of the server 10 combines one or more question sentences and one or more answer sentences in order to generate a character string. The one or more answer sentences to be combined may be answer sentences obtained by a search process, or may include answer sentences obtained by a dialogue process or a proposal process. In addition, it is not necessary to combine all of the one or more question sentences and one or more answer sentences. It is also possible to combine a predetermined number of question sentences and answer sentences in order, starting with the most recent, up to the number of characters that the generation AI 50 can accept.

[0054] In step S103, the AI ​​control unit 1042 of the server 10 executes a judgment step of inputting the question sentence received in the question receiving step as input data into a large-scale language model, and obtaining a judgment result as to whether the question sentence is suitable as input data for a question answering system capable of outputting an answer sentence in response to the input of a question sentence. Specifically, the judgment step includes a step of acquiring one or more keywords contained in the question sentence by inputting the question sentence received in the question receiving step into a large-scale language model as input data, a step of acquiring a judgment result indicating that the question sentence is suitable as input data if the one or more keywords are acquired, and a step of acquiring a judgment result indicating that the question sentence is not suitable as input data if the one or more keywords are not acquired. The determining step includes a keyword obtaining step.

[0055] In step S103, the AI ​​control unit 1042 of the server 10 executes a keyword acquisition step of acquiring one or more keywords that are output by inputting the question sentence accepted in the question acceptance step as input data into the large-scale language model. Specifically, the AI ​​control unit 1042 of the server 10 receives and accepts the search keywords output from the large-scale language model.

[0056] In step S103, a keyword acquisition step executes a step of acquiring one or more keywords that are output by inputting the question sentence and the additional question sentence as input data into the large-scale language model. Specifically, in step S102, when the AI ​​control unit 50 of the server 10 receives an additional question D122 in the question receiving step, it receives and accepts the search keywords output from the large-scale language model in response to inputting the string combining the question D121, answer D132, and question D122 into the generation AI 50 as input data, with the prompt inserted into the [Question] section of the prompt template.

[0057] For example, the keyword acquiring step includes a step of acquiring a first keyword output by inputting the first question sentence as input data into the large-scale language model, and a step of acquiring a second keyword output by inputting the first question sentence, the second question sentence, and the first answer sentence as input data into the large-scale language model.

[0058] The control unit 104 of the server 10 stores the question ID of the new record in the question table 1013 created in step S101, the prompt generated in step S102, and the search keyword in the question ID, input sentence, and keyword fields of the new record in the keyword table 1014, respectively.

[0059] In addition, the large-scale language model may not output a search keyword in step S103. For example, if the user's question included in the prompt does not include enough search keywords, the generation AI 50 does not output a search keyword. In such a case, the process skips steps S104 and after and ends the search process. In this case, the control unit 104 of the server 10 may be configured to execute step S302 and subsequent steps of the dialogue processing. In this case, the control unit 104 of the server 10 may be configured to execute step S502 and subsequent steps of the proposal process.

[0060] In step S104, a question input step executes a step of inputting one or more keywords acquired in the keyword acquisition step into the question and answer system. Specifically, the question and answering unit 1043 of the server 10 transmits the search keyword output from the generation AI 50 in step S103 to the question and answering system 60. For example, the question and answering unit 1043 of the server 10 transmits to the question and answering system 60 the first search keyword and the second search keyword output from the generation AI 50 in step S103.

[0061] In step S105, the question answering section 1043 of the server 10 executes an answer acquiring step of acquiring a predetermined answer sentence in response to the input of one or more keywords to the question and answering system in the question inputting step. Specifically, the question answering unit 1043 of the server 10 receives and accepts the output contents output from the question answering system 60. For example, when the question answering system 60 receives an input character string related to a search keyword, it searches for at least one of the questions or answers included in the question answering information stored in the question answering system 60 based on the character string by full-text search or the like, and outputs the question answering information related to the search result in association with the relevance (score) to the input character string. The question and answer unit 1043 of the server 10 receives and acquires the answer text included in the question and answer information.

[0062] For example, the answer acquiring step includes a step of acquiring a first answer sentence in response to input of a first keyword to the question answering system in the question inputting step, and a step of acquiring a second answer sentence in response to input of a second keyword to the question answering system in the question inputting step.

[0063] In step S105, the question answering unit 1043 of the server 10 receives and accepts the answer sentence output from the question and answering system 60. The control unit 104 of the server 10 stores the question ID of the new record in the question table 1013 created in step S101, the prompt generated in step S102, and the answer sentence in the question ID, input sentence, and answer sentence items of the new record in the answer table 1016, respectively.

[0064] In step S106, the control unit 104 of the server 10 presents the answer sentence output from the question and answer system 60. Specifically, the control unit 104 of the server 10 transmits the answer message received in step S105 to the user terminal 20. The control unit 204 of the user terminal 20 displays the received answer message on the display 2081 of the user terminal 20 for presentation. For example, when the control unit 204 of the user terminal 20 receives the question D121 in step S101, the answer to the question D121 is not stored in the answer table 1016 and is not displayed on the dialogue screen D1 because a search keyword cannot be obtained from the question D121. When the control unit 204 of the user terminal 20 receives the question D122 in step S101, the answer D135 is displayed on the dialogue screen D1. Furthermore, a sentence including a keyword stored in the keyword table 1014 may be displayed as an answer sentence D134 on the dialogue screen D1. By referring to the answer sentence D134, the user can visually confirm what search keyword has been acquired from the question sentence D122 and what answer sentence D135 has been obtained according to the search keyword. In addition, in the interactive screen D2 of the second screen example of the interactive screen, since the search keyword could not be acquired, the answer sentence to the question sentence from the question and answer system 60 is not stored in the answer table 1016 and is not displayed on the interactive screen D2.

[0065] In step S106, if the control unit 104 of the server 10 obtains a judgment result in the judgment step that the question is appropriate, it executes a first answering step of presenting to the user an answer to the question that is output by inputting input data based on the question into the question answering system. The first answering step includes a step of presenting to the user an answer to the question sentence that is output by inputting the one or more keywords acquired in the determining step as input data to the question and answering system. The first answering step includes a step of presenting to the user an answer to the question sentence that is output by inputting the second keyword as input data to the question and answering system.

[0066] The control unit 104 of the server 10 executes an answering step of presenting a first answer sentence based on at least one of a first answering step and a second answering step, based on the first question sentence. The AI ​​control unit 1042 of the server 10 executes an answering step of presenting a second answer sentence based on at least one of the first answering step, the second answering step, and the third answering step, based on the first question sentence. Specifically, if one or more question sentences have been received in the past in step S101 and one or more answer sentences have been obtained for the one or more question sentences, the one or more question sentences and one or more answer sentences are combined with the newly received question sentence to generate a character string, which is then transmitted to the generation AI 50. The one or more answer sentences to be combined may be answer sentences obtained by a search process, or may include answer sentences obtained by a dialogue process or a suggestion process. The keywords obtained from the generation AI 50 are input as input data to the question and answer system 60, and the answer to the question obtained thereby is presented to the user.

[0067] In step S106, the control unit 104 of the server 10 acquires the question information stored in the question table 1013. The control unit 104 of the server 10 searches the item of the question ID in the answer table 1016 based on the question ID included in the acquired question information, and acquires the answer information. The control unit 104 of the server 10 transmits the acquired question information and answer information to the user terminal 20. The control unit 204 of the user terminal 20 displays and presents the received question information and answer information on the display 2081 of the user terminal 20 in the order of the dates and times included therein. As a result, a first screen example and a second screen example including the processing results of all the processes, namely, the search process, the dialogue process, and the proposal process, are displayed and presented on the display 2081 of the user terminal 20. In other words, the steps related to presenting answer sentences, namely step S106 of the search process, step S304 of the dialogue process, and step S508 of the proposal process, may be a common process of displaying a list of the question information stored in the question table 1013 and the answer information stored in the answer table 1016.

[0068] <Interactive processing> The dialogue process is a process of presenting to the user an output sentence from the generation AI 50 in response to a question received from the user.

[0069] <Overview of interactive processing> The dialogue process is a process of accepting a question input from a user, inputting the input question to the generation AI 50, obtaining the output content output from the generation AI 50, and presenting the output content as an answer.

[0070] <Details of interaction processing> The details of the interaction process will be described below.

[0071] In step S301, the AI ​​control unit 1042 of the server 10 executes a question receiving step of receiving a question from a user. The question reception step is the same as step S101 of the search process, and therefore the process is omitted. In this disclosure, dialogue processing is executed according to the question received in step S101 of the search process. In other words, step S301 of the dialogue processing is common to step S101 of the search process. It is not necessary to execute the search process, and the dialogue process may be executed. In the present disclosure, the dialogue process may be executed when the search keyword cannot be acquired in step S103 of the search process (when the question sentence is not determined to be appropriate).

[0072] In step S302, the AI ​​control unit 1042 of the server 10 transmits the question received in the question receiving step to the generation AI 50. Specifically, the AI ​​control unit 50 of the server 10 generates a prompt by inserting the question received in step S301 into the [Question] section of the prompt template as an input sentence (prompt) for the generation AI 50, which is a large-scale language model, and sends the prompt to the generation AI 50.

[0073] In step S302, the AI ​​control unit 1042 of the server 10 inputs the first question, the answer to the second question, and the second question to the large-scale language model. Specifically, when the AI ​​control unit 50 of the server 10 receives an additional question D122 in the question receiving step, it generates a prompt by inserting a string combining the question D121, the answer D132, and the question D122 into the [Question] section of the prompt template, and sends the prompt to the generation AI 50. In other words, if one or more question sentences have been received in the past in step S301, and one or more answer sentences have been obtained for those one or more question sentences, a string formed by combining those one or more question sentences and one or more answer sentences with the newly received question sentence is sent to generation AI 50 as a question sentence. The control unit 104 of the server 10 can acquire one or more question sentences included in one or more pieces of question information received in the past by searching the question ID item of the question table 1013 based on the parent question ID included in the acquired one or more pieces of question information. The control unit 104 of the server 10 can acquire any parent generation question information and question sentence by searching the question ID item of the question table 1013 based on the parent question ID included in the acquired one or more pieces of question information. The control unit 104 of the server 10 can obtain one or more answer sentences included in one or more pieces of answer information by searching the items in the answer table 1016 based on one or more question IDs included in one or more pieces of question information previously received. The question information and the answer information can determine their corresponding relationships (order relationship, which answer sentence corresponds to which question sentence) based on their respective question IDs. The order of the one or more answer sentences may also be the order of the dates and times included in the answer information. The control unit 104 of the server 10 combines one or more question sentences and one or more answer sentences in order to generate a character string. The one or more answer sentences to be combined may be answer sentences obtained by dialogue processing, or may include answer sentences obtained by search processing and suggestion processing, which will be described later. In addition, it is not necessary to combine all of the one or more question sentences and one or more answer sentences. It is also possible to combine a predetermined number of question sentences and answer sentences in reverse chronological order, with the number of characters that the generation AI 50 can accept as the upper limit.

[0074] In this way, in the information processing system according to the present disclosure, the user can check the content of the answer to the question he or she inputted and then input a further question. This allows the user to take into consideration the content of the answer from the information processing system and gradually input a question in an interactive format to obtain the information he or she desires.

[0075] In step S303, the AI ​​control unit 1042 of the server 10 receives and accepts the answer sentence (not an answer sentence to the question sentence but a response sentence for digging deeper into the question sentence) output from the large-scale language model. The response sentence output from the generation AI 50 is a sentence that does not include the answer content to the question sentence accepted by the user in step S301, and is a sentence (response sentence) that includes a suggestion for digging deeper into the question sentence accepted from the user in more detail. The control unit 104 of the server 10 stores the question ID of the new record in the question table 1013 created in step S301, the prompt generated in step S302, and the answer sentence in the question ID, input sentence, and answer sentence fields of the new record in the answer table 1016, respectively.

[0076] In step S304, if the control unit 104 of the server 10 does not obtain a judgment result indicating that the question sentence is appropriate in the judgment step, it executes a second answering step of presenting to the user a response sentence for digging deeper into the question sentence, rather than an answer sentence to the question sentence, which is output by inputting input data based on the question sentence into a large-scale language model. Specifically, the control unit 104 of the server 10 transmits the answer text received in step S303 to the user terminal 20. The control unit 204 of the user terminal 20 displays and presents the received answer text on the display 2081 of the user terminal 20. The control unit 104 of the server 10 may be configured to transmit the answer text received in step S303 to the user terminal 20 when a search keyword cannot be acquired in step S103 of the search process. For example, when the control unit 204 of the user terminal 20 receives the question sentence D121 in step S301, the answer sentence D132 is displayed on the interactive screen D1. When the control unit 204 of the user terminal 20 receives the question sentence D122 in step S301, the answer sentence D133 is displayed on the interactive screen D1. For example, when the control unit 204 of the user terminal 20 receives the question sentence D221 in step S301, the answer sentence D232 is displayed on the interactive screen D2. When the control unit 204 of the user terminal 20 receives the question sentence D222 in step S301, the answer sentence D233 is displayed on the interactive screen D2.

[0077] <Proposal processing> The search process is a process of presenting to the user output sentences from the question and answering system 60 based on question candidates corresponding to the question sentence received from the user.

[0078] <Proposal process overview> The suggestion process is a process of accepting input of a question sentence from a user, inputting the input question sentence to generation AI 50, obtaining question candidates output from generation AI 50, presenting the question candidates to the user so that they can be selected, inputting one or more question candidates selected by the user to question answering system 60, obtaining the output content output from question answering system 60, and presenting the output content as an answer sentence.

[0079] <Details of the proposal process> The proposal process will be described in detail below.

[0080] In step S501, the AI ​​control unit 1042 of the server 10 executes a question receiving step of receiving a question from a user. The question reception step is the same as step S301 of the dialogue processing, and therefore the process is omitted. Note that in the present disclosure, a proposal process is executed in response to the question received in step S301 of the dialogue processing. In other words, step S501 of the search process is common to step S301 of the dialogue processing. It should be noted that the dialogue process does not necessarily have to be executed, and the configuration may be such that a proposal process is executed.

[0081] In step S502, the AI ​​control unit 1042 of the server 10 transmits the question received in the question receiving step to the generation AI 50. Specifically, the AI ​​control unit 50 of the server 10 generates a prompt by inserting the question received in step S501 into the [Question] section of the prompt template as an input sentence (prompt) for the generation AI 50, which is a large-scale language model, and sends the prompt to the generation AI 50. In the search process, the prompt includes an instruction to restrict the output data format of the generation AI 50 to output one or more search keyword candidates instead of an answer sentence. Specifically, in OpenAI's GPT, an instruction to extract one or more search keyword candidates to be input to the question answering system 60 is included using a mechanism called Function Calling.

[0082] In step S502, the AI ​​control unit 1042 of the server 10 inputs the first question, the answer to the second question, and the second question to the large-scale language model. Specifically, when the AI ​​control unit 50 of the server 10 receives an additional question D122 in the question receiving step, it generates a prompt by inserting a string combining the question D121, the answer D132, and the question D122 into the [Question] section of the prompt template, and sends the prompt to the generation AI 50. In other words, if one or more question sentences have been received in the past in step S501, and one or more answer sentences have been obtained for those one or more question sentences, a string formed by combining those one or more question sentences and one or more answer sentences with the newly received question sentence is sent to generation AI 50 as a question sentence. The control unit 104 of the server 10 can acquire one or more question sentences included in one or more pieces of question information received in the past by searching the question ID item of the question table 1013 based on the parent question ID included in the acquired one or more pieces of question information. The control unit 104 of the server 10 can acquire any parent generation question information and question sentence by searching the question ID item of the question table 1013 based on the parent question ID included in the acquired one or more pieces of question information. The control unit 104 of the server 10 can obtain one or more answer sentences included in one or more pieces of answer information by searching the items in the answer table 1016 based on one or more question IDs included in one or more pieces of question information previously received. The question information and the answer information can determine their corresponding relationships (order relationship, which answer sentence corresponds to which question sentence) based on their respective question IDs. The order of the one or more answer sentences may also be the order of the dates and times included in the answer information. The control unit 104 of the server 10 combines one or more question sentences and one or more answer sentences in order to generate a character string. The one or more answer sentences to be combined may be answer sentences obtained by the suggestion process, or may include answer sentences obtained by the search process or the dialogue process. In addition, it is not necessary to combine all of the one or more question sentences and one or more answer sentences. It is also possible to combine a predetermined number of question sentences and answer sentences in reverse chronological order, with the number of characters that the generation AI 50 can accept as the upper limit.

[0083] In step S503, a step of acquiring one or more question candidates that are output by inputting the question sentence and the additional question sentence as input data into a large-scale language model is executed. Specifically, the AI ​​control unit 1042 of the server 10 receives and accepts the search keyword candidates output from the large-scale language model.

[0084] In addition, the large-scale language model may not output search keyword candidates in step S103. For example, if the user's question included in the prompt does not include enough search keyword candidates, the generation AI 50 does not output search keyword candidates. In such a case, the process skips steps S504 and after and ends the suggestion process.

[0085] In step S504, the AI ​​control unit 1042 of the server 10 executes a candidate presentation step of presenting to the user one or more question candidates output by inputting the question sentence received in the question receiving step as input data into a large-scale language model. Specifically, the AI ​​control unit 1042 of the server 10 receives and accepts one or more search keyword candidates output from the large-scale language model. The control unit 104 of the server 10 stores the question ID of the new record in the question table 1013 created in step S501, the prompt generated in step S502, and one or more search keyword candidates in the question ID, input sentence, and question candidate fields of the new record in the candidate table 1015, respectively. The question and answer unit 1043 of the server 10 transmits one or more search keyword candidates to the user terminal 20. The control unit 204 of the user terminal 20 displays and presents the received one or more search keyword candidates on the display 2081 of the user terminal 20. For example, when the control unit 204 of the user terminal 20 receives the question D224 in step S501, it displays and presents it as question candidates D251 and D252 on the dialogue screen D2.

[0086] For example, the candidate presenting step includes a step of presenting a first question candidate based on the first question sentence, and a step of presenting a second question candidate based on the first question sentence, the second question sentence, and the first answer sentence.

[0087] In step S504, a candidate presenting step executes a step of presenting one or more question candidates to the user when one or more search keywords cannot be acquired in the keyword acquiring step. Specifically, in a configuration in which the proposal process is executed after the search process, the proposal process may be executed if the search keyword cannot be obtained in step S103 of the search process, and the proposal process may not be executed if the search keyword can be obtained in step S103 of the search process. In other words, when an answer sentence is obtained from question answering system 60 by the search process according to the present disclosure, there is no need to present question candidates to the user, so the suggestion process may be executed when an answer sentence is not obtained from question answering system 60 by the search process, and the suggestion process may not be executed when an answer sentence is obtained from question answering system 60 by the search process. This makes it possible to configure the system so that the suggestion process is not executed when an answer sentence is presented to the user from question and answering system 60. In other words, when an answer to the search process is obtained, the dialogue with the user may be ended.

[0088] In step S505, the AI ​​control unit 1042 of the server 10 executes a candidate selection step of accepting, from the user, a selection of a predetermined question candidate which is at least one of the one or more question candidates presented in the candidate presentation step. Specifically, the user operates the input device 206 of the user terminal 20 to select at least one of one or more question candidates D251, D252 presented on the interactive screen D2. Note that a configuration may be adopted in which multiple question candidates D251, D252 can be selected. The control unit 204 of the user terminal 20 transmits information related to the selected predetermined one or more question candidates D251, D252 to the server 10.

[0089] For example, the candidate selection step includes a step of accepting a selection of a predetermined first question candidate which is at least one of the first question candidates, and a step of accepting a selection of a predetermined second question candidate which is at least one of the second question candidates.

[0090] In step S506, the question answering unit 1043 of the server 10 executes a question input step of inputting the predetermined question candidate selected in the candidate selection step to a question and answering system capable of outputting an answer sentence in response to an input of a question sentence. Specifically, the question and answering section 1043 of the server 10 transmits to the question and answering system 60 the predetermined one or more candidate questions D251, D252 selected in step S505. For example, the question input step includes a step of inputting a predetermined first candidate question to the question and answering system, and a step of inputting a predetermined second candidate question to the question and answering system.

[0091] In step S507, the question answering unit 1043 of the server 10 executes an answer obtaining step of obtaining a predetermined answer sentence to be output in response to input of a predetermined question candidate to the question and answering system in the question input step. Specifically, the question answering unit 1043 of the server 10 receives and accepts the output contents output from the question answering system 60. For example, when the question answering system 60 receives an input character string related to a question candidate, it searches for at least one of the questions or answers included in the question answering information stored in the question answering system 60 by full-text search or the like based on the character string, and outputs the question answering information related to the search result in association with the relevance (score) to the input character string. The question and answer unit 1043 of the server 10 receives and acquires the answer text included in the question and answer information.

[0092] For example, the answer acquiring step includes a step of acquiring a first predetermined answer sentence in response to an output of a first predetermined candidate question to the question and answering system, and a step of acquiring a second predetermined answer sentence in response to an output of a second predetermined candidate question to the question and answering system.

[0093] In step S507, the question answering unit 1043 of the server 10 receives and accepts the answer sentence output from the question and answering system 60. The control unit 104 of the server 10 stores the question ID of the new record in the question table 1013 created in step S101, the prompt generated in step S102, and the answer sentence in the question ID, input sentence, and answer sentence items of the new record in the answer table 1016, respectively.

[0094] In step S508, the control unit 104 of the server 10 presents the answer sentence output from the question and answer system 60. Specifically, the control unit 104 of the server 10 transmits the answer message received in step S507 to the user terminal 20. The control unit 204 of the user terminal 20 displays the received answer message on the display 2081 of the user terminal 20 for presentation.

[0095] <Basic computer hardware configuration> 14 is a block diagram showing the basic hardware configuration of a computer 90. The computer 90 includes at least a processor 901, a main storage device 902, an auxiliary storage device 903, and a communication IF 991 (interface). These are electrically connected to each other by a communication bus 921.

[0096] The processor 901 is hardware for executing an instruction set described in a program, and is composed of an arithmetic unit, a register, a peripheral circuit, and the like.

[0097] The main memory device 902 is for temporarily storing programs, data to be processed by the programs, etc. For example, it is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0098] The auxiliary storage device 903 is a storage device for saving data and programs, such as a flash memory, a hard disk drive (HDD), a magneto-optical disk, a CD-ROM, a DVD-ROM, or a semiconductor memory.

[0099] The communication IF 991 is an interface for inputting and outputting signals for communicating with other computers via a network using a wired or wireless communication standard. The network is composed of the Internet, a LAN, various mobile communication systems constructed by wireless base stations, etc. For example, the network includes 3G, 4G, 5G mobile communication systems, LTE (Long Term Evolution), wireless networks that can connect to the Internet via a specified access point (e.g., Wi-Fi (registered trademark)), etc. In the case of wireless connection, communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), Bluetooth (registered trademark), etc. In the case of wired connection, the network also includes a network that is directly connected by a USB (Universal Serial Bus) cable or the like.

[0100] It should be noted that the computer 90 can be virtually realized by distributing all or part of each hardware configuration among multiple computers 90 and connecting them together via a network. In this way, the computer 90 is a concept that includes not only a computer 90 housed in a single housing or case, but also a virtualized computer system.

[0101] <Basic functional configuration of computer 90> A description will now be given of the functional configuration of a computer realized by the basic hardware configuration (FIG. 14) of a computer 90. The computer comprises at least the functional units of a control unit, a storage unit, and a communication unit.

[0102] The functional units of the computer 90 can also be realized by distributing all or part of the functional units among multiple computers 90 connected to each other via a network. The computer 90 is a concept that includes not only a single computer 90 but also a virtualized computer system.

[0103] The control unit is realized by the processor 901 reading out various programs stored in the auxiliary storage device 903, expanding the programs in the main storage device 902, and executing processes according to the programs. The control unit can realize functional units that perform various information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0104] The storage unit is realized by a main storage device 902 and an auxiliary storage device 903. The storage unit stores data, various programs, and various databases. Furthermore, the processor 901 can secure a storage area corresponding to the storage unit in the main storage device 902 or the auxiliary storage device 903 in accordance with a program. Furthermore, the control unit can cause the processor 901 to execute processes of adding, updating, and deleting data stored in the storage unit in accordance with the various programs.

[0105] The term database refers to a relational database, which is used to manage sets of data called masters and tables in a tabular format structurally defined by rows and columns, by associating them with each other. In a database, a table is called a table or master, a column in a table is called a column, and a row in a table is called a record. In a relational database, relationships between tables and masters can be set and associated. Usually, a column that serves as a primary key for uniquely identifying a record is set in each table and each master, but setting a primary key in a column is not essential. The control unit can cause the processor 901 to add, delete, or update records in a specific table or master stored in the storage unit according to various programs. Furthermore, by storing data, various programs, and various databases in the storage unit, it can be considered that the information processing device and information processing system according to the present disclosure have been manufactured.

[0106] In addition, the database and master in this disclosure may include any data structure (such as a list, a dictionary, an associative array, or an object) in which information is structurally defined. The data structure also includes data that can be considered as a data structure by combining data with a function, class, method, or the like written in any programming language.

[0107] The communication unit is realized by the communication IF 991. The communication unit realizes a function of communicating with other computers 90 via a network. The communication unit can receive information transmitted from other computers 90 and input the information to the control unit. The control unit can cause the processor 901 to execute information processing on the received information in accordance with various programs. In addition, the communication unit can transmit information output from the control unit to other computers 90.

[0108] <Additional Notes> The matters described in the above embodiments will be supplemented below.

[0109] (Appendix 1) A program to be executed by a computer having a processor and a memory unit, the program comprising: a question receiving step (S101, S301, S501) in which the processor receives a question from a user; a judgment step (S103) in which the question received in the question receiving step is input as input data into a large-scale language model, thereby obtaining a judgment result as to whether the question is appropriate as input data for a question answering system capable of outputting an answer sentence in response to the input of the question sentence; a first answering step (S106) in which, if the judgment result that the question sentence is appropriate is obtained in the judgment step, an answer sentence to the question sentence that is output by inputting input data based on the question sentence into the question answering system is presented to the user; and a second answering step (S304) in which, if the judgment result that the question sentence is appropriate is not obtained in the judgment step, a reply sentence for digging deeper into the question sentence, rather than an answer sentence to the question sentence, is output by inputting the input data based on the question sentence into the large-scale language model. This allows the user to receive an answer from the question answering system if the question received from the user is appropriate as input data for the question answering system, and allows the user to receive a response to delve deeper into the question if the question is not appropriate.

[0110] (Appendix 2) 2. The program of claim 1, wherein the question answering system is a system different from the large-scale language model. This allows the user to receive an answer from the question answering system if the question received from the user is appropriate as input data for the question answering system, and allows the user to receive a response to delve deeper into the question if the question is not appropriate.

[0111] (Appendix 3) The question answering system is a system that searches for answer sentences based on keywords and outputs a predetermined answer sentence related to the search result, and the judgment step (S103) includes a step of acquiring one or more keywords contained in the question sentence by inputting the question sentence accepted in the question receiving step into a large-scale language model as input data, a step of acquiring a judgment result that the question sentence is suitable as input data if the one or more keywords are acquired, and a step of acquiring a judgment result that the question sentence is not suitable as input data if the one or more keywords are not acquired, and the first answering step (S106) is a step of presenting to a user the answer sentence to the question sentence output by inputting the one or more keywords acquired in the judgment step into the question answering system as input data. As a result, if a question received from a user contains keywords that are appropriate as input data for the question answering system, the user can be presented with an answer from the question answering system based on those keywords, and if the keyword is not included, the user can be presented with a response that delves deeper into the question.

[0112] (Appendix 4) A program as described in Appendix 3, wherein the question receiving step (S101) is a step for receiving an additional question text when one or more search keywords cannot be obtained in the judgment step, and the judgment step (S103) includes a step for obtaining one or more keywords that are output by inputting the question text and the additional question text as input data into a large-scale language model. This makes it possible to accept an additional question if a search keyword cannot be obtained based on the question entered by the user, and the search keyword output based on the additional question can be input to the question and answering system.

[0113] (Appendix 5) The program described in Appendix 1, wherein the processor executes a candidate presentation step (S504) of presenting to a user one or more question candidates that are output by inputting the question sentence received in the question receiving step into a large-scale language model as input data, a candidate selection step (S505) of accepting from the user a selection of a predetermined question candidate which is at least one of the one or more question candidates presented in the candidate presentation step, and a third answering step (S508) of presenting to the user an answer sentence to the question sentence that is output by inputting the predetermined question candidate whose selection was accepted in the candidate selection step into a question answering system. This makes it possible to input a question to the question answering system in order to output a more appropriate answer based on question candidates output from the large-scale language model, even if the question received from the user is insufficient.

[0114] (Appendix 6) The program according to claim 5, wherein the candidate presenting step (S504) is a step of presenting one or more question candidates to the user when one or more keywords included in the question sentence cannot be acquired in the determining step. This allows the system to present candidate questions to the user when a search keyword cannot be obtained based on the question sentence entered by the user. Based on the candidate questions, a question sentence for outputting a more suitable answer sentence can be input to the question answering system.

[0115] (Appendix 7) The program described in Appendix 3, wherein the question receiving step (S101, S301, S501) includes a step of receiving a first question sentence, and an answering step (S106, S304, S508) of presenting a first answer sentence based on at least one of a first answering step and a second answering step based on the first question sentence, the question receiving step (S101) includes a step of accepting a second question sentence, the judgment step (S103) includes a step of acquiring second keywords output by inputting the first question sentence, the second question sentence, and the first answer sentence as input data into a large-scale language model, and the first answering step (S106) includes a step of presenting to a user an answer sentence to the question sentence output by inputting the second keywords as input data into the question answering system. This allows the user to ask multiple questions and input search keywords for the content including the answers obtained in response to the questions to the question and answering system. A second answer based on the search keywords can be obtained from the question and answering system and presented.

[0116] (Appendix 8) The program described in Appendix 5, wherein the question receiving step (S101, S301, S501) includes a step of receiving a first question sentence, and an answering step (S106, S304, S508) of presenting a second answer sentence based on at least one of a first answering step, a second answering step, and a third answering step based on the first question sentence, the question receiving step (S501) includes a step of receiving a second question sentence, the candidate presenting step (S504) includes a step of presenting to a user second question candidates output by inputting the first question sentence, the second question sentence, and the second answering sentence as input data into a large-scale language model, the candidate selecting step (S505) includes a step of accepting a selection of a predetermined second question candidate which is at least one of the second question candidates, and the third answering step (S508) includes a step of presenting to the user an answer sentence to the question sentence output by inputting the predetermined second question candidate into a question and answering system. This allows a user to ask multiple questions, and candidate questions for the content including answers obtained in response to the questions can be obtained from the large-scale language model. A second answer based on the candidate questions can be obtained from the question answering system and presented.

[0117] (Appendix 9) A method implemented on a computer having a processor and a memory, the processor performing all of the steps performed in the invention according to any one of appendixes 1 to 8. This allows the user to receive an answer from the question answering system if the question received from the user is appropriate as input data for the question answering system, and allows the user to receive a response to delve deeper into the question if the question is not appropriate.

[0118] (Appendix 10) An information processing device comprising a control unit and a memory unit, wherein the control unit executes all of the steps executed in the invention according to any one of Supplementary Note 1 to Supplementary Note 8. This allows the user to receive an answer from the question answering system if the question received from the user is appropriate as input data for the question answering system, and allows the user to receive a response to delve deeper into the question if the question is not appropriate.

[0119] (Appendix 11) A system comprising means for performing all the steps performed in any of the inventions according to appendix 1 to appendix 8. This allows the user to receive an answer from the question answering system if the question received from the user is appropriate as input data for the question answering system, and allows the user to receive a response to delve deeper into the question if the question is not appropriate. [Explanation of symbols]

[0120] 1 system, 10 server, 101 memory unit, 104 control unit, 106 input device, 108 output device, 20 user terminal, 201 memory unit, 204 control unit, 206 input device, 208 output device, 50 generation AI, 501 memory unit, 504 control unit, 506 input device, 508 output device, 60 question answering system, 601 memory unit, 604 control unit, 606 input device, 608 output device

Claims

1. A program to be executed by a computer having a processor and a storage unit, The processor, a question receiving step of receiving a question from a user; a determination step of inputting the question sentence received in the question receiving step into a large-scale language model as input data, thereby obtaining a determination result as to whether or not the question sentence is suitable as input data for a question answering system capable of outputting an answer sentence in response to an input of a question sentence; a first answering step of presenting to the user an answer to the question, which is output by inputting input data based on the question into the question answering system, when a judgment result that the question is appropriate is obtained in the judging step; a second answering step of presenting to the user, when a judgment result that the question sentence is appropriate is not obtained in the judgment step, a response sentence for digging deeper into the question sentence, which is output by inputting input data based on the question sentence into the large-scale language model, instead of an answer sentence to the question sentence; A program that executes.

2. The question answering system is a system different from the large-scale language model. The program according to claim 1.

3. the question and answering system is a system for searching for an answer sentence based on a keyword, and outputting a predetermined answer sentence related to the search result, The determining step includes: a step of inputting the question sentence received in the question receiving step as input data into the large-scale language model to acquire one or more keywords included in the question sentence; acquiring a determination result that the question sentence is appropriate as input data when the one or more keywords are acquired; acquiring a determination result that the question sentence is not suitable as input data when the one or more keywords cannot be acquired; Including, the first answering step is a step of presenting to the user an answer to the question, the answer being output by inputting the one or more keywords acquired in the determining step as input data to the question answering system; The program according to claim 1.

4. the question receiving step is a step of receiving an additional question sentence when the one or more search keywords cannot be acquired in the determining step, the determining step includes a step of acquiring one or more keywords output by inputting the question sentence and the additional question sentence as input data into the large-scale language model; The program according to claim 3.

5. The processor, a candidate presenting step of presenting to the user one or more question candidates output by inputting the question sentence accepted in the question accepting step as input data into the large-scale language model; a candidate selection step of accepting, from the user, a selection of a predetermined question candidate which is at least one of the one or more question candidates presented in the candidate presentation step; a third answering step of presenting to the user an answer sentence to the question sentence that is output by inputting the predetermined question candidate selected in the candidate selecting step to the question and answering system; Execute The program according to claim 1.

6. the candidate presentation step is a step of presenting the one or more question candidates to the user when the one or more keywords included in the question sentence cannot be acquired in the determination step; The program according to claim 5.

7. The question receiving step includes a step of receiving a first question sentence, an answering step of presenting a first answer sentence based on at least one of the first answering step and the second answering step based on the first question sentence; The question receiving step includes a step of receiving a second question sentence, the determining step includes a step of acquiring second keywords output by inputting the first question sentence, the second question sentence, and the first answer sentence as input data into the large-scale language model; the first answering step includes a step of presenting to the user an answer sentence to the question sentence, the answer sentence being output by inputting the second keyword as input data to the question and answering system; The program according to claim 3.

8. The question receiving step includes a step of receiving a first question sentence, an answering step of presenting a second answer sentence based on at least one of the first answering step, the second answering step, and the third answering step based on the first question sentence; The question receiving step includes a step of receiving a second question sentence, the candidate presenting step includes a step of presenting to the user second question candidates output by inputting the first question sentence, the second question sentence, and the second answer sentence as input data into the large-scale language model; the candidate selection step includes a step of accepting a selection of a predetermined second question candidate, which is at least one of the second question candidates; the third answering step includes a step of presenting to the user an answer sentence to the question sentence, the answer sentence being output by inputting the predetermined second question candidate into the question and answering system; The program according to claim 5.

9. A method implemented on a computer having a processor and a memory, the processor performing all of the steps performed in any one of claims 1 to 8.

10. 9. An information processing apparatus comprising a control unit and a storage unit, the control unit executing all steps executed in the invention according to any one of claims 1 to 8.

11. A system comprising means for executing all the steps performed in any one of claims 1 to 8.

Images