Information provision device, information provision method, and program

Abstract

This makes it possible to provide users with more appropriate response information in response to their requests. [Solution] An information providing device comprising a response generation unit that generates response information to a request information in response to a first prompt containing request information corresponding to input information entered by a user, one or more first output pieces generated by a trained first generative model in response to a first prompt containing request information corresponding to input information entered by a user, and one or more second output pieces generated by a trained second generative model in response to a second prompt containing the request information, wherein the one or more first output pieces and the one or more second output pieces are candidates for the response information.

Description

Technical Field

[0001] The present invention relates to a technique for presenting more appropriate response information for request information from a user.

Background Art

[0002] In recent years, techniques have been proposed for outputting information corresponding to information input by a user using a large language model capable of natural language processing. When a large language model is fine-tuned using a dialogue dataset, dialogue-based exchanges become possible. A user can obtain a response sentence for a question sentence by inputting the question sentence into such a large language model. Specifically, a prompt including the question sentence from the user is input into the large language model.

[0003] The response sentence output by the large language model largely depends on the content of the question sentence input by the user. For example, if the vocabulary selection or way of speaking in the question sentence is different, the response from the large language model may also be different. Therefore, depending on the content of the question sentence, a response sentence that is not the response desired by the user may be output. As described above, there is a problem that there are cases where the output from the large language model is not appropriate.

[0004] Therefore, various techniques have been proposed to make the output from the large language model appropriate. For example, Patent Document 1 discloses a configuration in which reference information related to the question sentence input by the user is generated, and a prompt including the question sentence and the reference information is input into the large language model to obtain a response sentence for the question sentence. In Patent Document 1, it is considered that a more specific response sentence can be generated as compared with a configuration for generating a prompt including only the question sentence.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

[0006] However, the technology described in Patent Document 1 still has room for improvement in terms of providing users with appropriate answers to their questions. Taking these circumstances into consideration, the present invention aims to provide users with more appropriate response information to the information they request. [Means for solving the problem]

[0007] An example of a terminal device according to the present invention comprises a receiving unit that receives input information entered by a user, and an acquisition unit that acquires response information generated in response to request information corresponding to the input information, wherein the response information is information generated in response to one or more first output pieces generated by a first generative model trained for a first prompt including the request information, and one or more second output pieces generated by a second generative model trained for a second prompt including the request information.

[0008] An information providing device according to an example of the present invention comprises a response generation unit that generates response information to the request information in accordance with one or more first output pieces generated by a trained first generative model in response to a first prompt containing request information corresponding to input information entered by a user, and one or more second output pieces generated by a trained second generative model in response to a second prompt containing the request information.

[0009] An example of the present invention is an information gathering system that can communicate with the aforementioned terminal device, and comprises a first acquisition unit that acquires the response information and the requested information, and a collection unit that stores the evaluation, the requested information and the response information in a storage device.

[0010] An example of an information gathering system according to the present invention is a system that can communicate with the aforementioned terminal device and comprises: a first acquisition unit that acquires related information related to a fourth prompt containing the input information, which is generated by a trained fourth generative model; a second acquisition unit that acquires response information generated by a trained fifth generative model in response to a fifth prompt containing the related information; and a collection unit that stores the related information and the response information in a storage device.

[0011] The information processing system according to the present invention comprises: a receiving unit that receives input information entered by a user; a response generation unit that generates response information to the request information in accordance with one or more first output pieces generated by a first generative model that has been trained for a first prompt containing request information corresponding to the input information, and one or more second output pieces generated by a second generative model that has been trained for a second prompt containing the request information; and an acquisition unit that acquires the response information.

[0012] An example of an information acquisition method according to the present invention is implemented by a computer system that receives input information entered by a user, acquires response information generated in response to request information corresponding to the input information, and the response information is generated in response to one or more first output pieces of information generated by a first generative model trained for a first prompt including the request information, and one or more second output pieces of information generated by a second generative model trained for a second prompt including the request information.

[0013] A program according to an example of the present invention causes a computer system to function as a receiving unit that receives input information entered by a user, and an acquisition unit that acquires response information generated in response to request information corresponding to the input information, and the computer system functions as if the response information is information generated in response to one or more first output pieces of information generated by a first generative model trained for a first prompt including the request information, and one or more second output pieces of information generated by a second generative model trained for a second prompt including the request information. [Brief explanation of the drawing]

[0014] [Figure 1] It is a block diagram illustrating the configuration of the information processing system according to the first embodiment. [Figure 2] It is a block diagram illustrating the configuration of the terminal device according to the first embodiment. [Figure 3] It is a block diagram illustrating the functional configuration of the control device in the terminal device according to the first embodiment. [Figure 4] It is a block diagram illustrating the configuration of the information generation system according to the first embodiment. [Figure 5] It is a block diagram illustrating the functional configuration of the control device in the information generation system according to the first embodiment. [Figure 6] It is a block diagram illustrating the configuration of the information generation system according to the first embodiment. [Figure 7] It is a block diagram illustrating the configuration of the information acquisition device according to the first embodiment. [Figure 8] It is a block diagram illustrating the configuration of the first processing device according to the first embodiment. [Figure 9] It is a block diagram illustrating the functional configuration of the control device in the first processing device according to the first embodiment. [Figure 10] It is a block diagram illustrating the configuration of the second processing device according to the first embodiment. [Figure 11] It is a block diagram illustrating the functional configuration of the control device in the second processing device according to the first embodiment. [Figure 12] It is a block diagram illustrating the configuration of the information providing device according to the first embodiment. [Figure 13] It is a block diagram illustrating the functional configuration of the control device in the information providing device according to the first embodiment. [Figure 14] It is a flowchart illustrating the procedure of the processing executed by the entire information processing system. [Figure 15] It is a diagram showing the input information and related information according to the first embodiment. [Figure 16]It is a flowchart exemplifying the procedure of the process executed by the information providing system according to the first embodiment. [Figure 17] It is a diagram showing the first output information and the second output information according to the first embodiment. [Figure 18] It is a flowchart exemplifying the procedure of the process executed by the response generation unit of the information providing apparatus according to the first embodiment. [Figure 19] It is a block diagram exemplifying the configuration of the information processing system according to the second embodiment. [Figure 20] It is a block diagram exemplifying the configuration of the information collection system according to the second embodiment. [Figure 21] It is a block diagram exemplifying the functional configuration of the control device in the information collection system according to the second embodiment. [Figure 22] It is a block diagram exemplifying the functional configuration of the control device in the information collection system according to the third embodiment.

Modes for Carrying Out the Invention

[0015] <First Embodiment> FIG. 1 is a block diagram exemplifying the configuration of an information processing system 100 according to the first embodiment. The information processing system 100 according to the first embodiment is a computer system including a terminal device 10 and an information management system 20. The information management system 20 provides various types of information to the user U of the terminal device 10. The information management system 20 includes an information generation system 30 and an information providing system 40.

[0016] In general terms, the information management system 20 provides user U with information (hereinafter referred to as "response information") R in response to the request information from user U. Typically, the request information is information for user U to request the information they desire, and the response information R is information indicating the response to the request information (typically the information requested by the request information). In the following description, we will illustrate the case where the request information is an inquiry about a predetermined object, and the response information R is the answer to that inquiry. The predetermined object is, for example, various products and services. In the first embodiment, we will conveniently illustrate the case where a specific product (hereinafter referred to as "target product") is the predetermined object. The target product is, for example, software used for a predetermined purpose.

[0017] The terminal device 10 and the information management system 20 can communicate with each other via a communication network 900, such as a mobile communication network or the Internet.

[0018] The terminal device 10 is implemented by, for example, a portable information device such as a smartphone or tablet terminal, or a portable or stationary information device such as a personal computer. On the other hand, the information generation system 30 and the information provision system 40 are implemented by, for example, a server system.

[0019] [Terminal device 10] Figure 2 is a block diagram illustrating the configuration of the terminal device 10. As illustrated in Figure 2, the terminal device 10 comprises a control device 11, a storage device 12, a communication device 13, a display device 14, and an operating device 15.

[0020] The control device 11 is one or more processors that control the operation of the terminal device 10. Specifically, the control device 11 is composed of one or more types of processors, such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), SPU (Sound Processing Unit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), or ASIC (Application Specific Integrated Circuit).

[0021] The storage device 12 is one or more memories that store the program executed by the control device 11 and various data used by the control device 11. For example, known recording media such as semiconductor recording media and magnetic recording media, or a combination of multiple types of recording media, can be used as the storage device 12. Alternatively, for example, a portable recording media that can be attached to and detached from the terminal device 10, or a recording media accessible by the control device 11 via the communication network 900 (e.g., cloud storage), may be used as the storage device 12.

[0022] The communication device 13 is a communication device that communicates with the information management system 20 via the communication network 900. Alternatively, the communication device 13, which is separate from the terminal device 10, may be connected to the terminal device 10 by wire or wireless connection.

[0023] The display device 14 displays various types of information under the control of the control device 11. For example, various display panels such as liquid crystal display panels or organic EL panels can be used as the display device 14. Alternatively, an operating device 15 or display device 14, separate from the information processing system 100, may be connected to the terminal device 10 by wire or wireless connection.

[0024] The operating device 15 is an input device that receives operations from user U. For example, an operator operated by user U, or a touch panel that detects contact by user U, may be used as the operating device 15. The operating device 15 may be a separate device from the terminal device 10 and connected to the terminal device 10 by wire or wireless connection. The operating device 15 receives various types of information input by user U. The operating device 15 may also be a sound-receiving device (microphone) that receives voice input from user U.

[0025] Figure 2 is a block diagram illustrating the functions realized by the control device 11 executing a program stored in the storage device 12. The control device 11 implements multiple functions (reception unit 111, prompt generation unit 112, acquisition unit 113, presentation unit 114) for providing answers to inquiries.

[0026] The reception unit 111 receives various types of information entered by user U through operations on the operating device 15. Firstly, the reception unit 111 receives input information entered by user U. The input information is information for requesting information desired by user U. In the first embodiment, the input information is information indicating an inquiry about the target product. The content of the inquiry (information desired by user U) is arbitrary and may include, for example, how to use the target product, a specific function of the target product, or an error that occurred while using the target product. User U enters a text indicating the inquiry through operations on the operating device 15.

[0027] Secondly, the reception unit 111 receives the request information entered by user U. The request information in the first embodiment is the information selected by user U from the input information and one or more related pieces of information G. That is, the input information and one or more related pieces of information G are candidates for the request information.

[0028] Related information G, like input information, is information used by user U to request desired information and is related to the input information. Typically, related information G is information that shows the same type of content as the input information (e.g., a query). In other words, the content of the input information and the content of related information G are similar. The requested information, which is information selected by user U from the input information and one or more related pieces of related information G, is information corresponding to the input information. Details of related information G will be described later.

[0029] Thirdly, the reception unit 111 receives the identification information entered by user U. The identification information includes various types of information such as the name, type, version, and product number of the target product. The identification information may also include information to identify user U of the target product.

[0030] The prompt generation unit 112 generates a prompt P that includes various pieces of information entered by the user U. Prompt P is a request to the information management system 20. The prompt P generated by the prompt generation unit 112 is transmitted from the communication device 13 to the information management system 20.

[0031] Firstly, the prompt generation unit 112 generates a prompt P1 (an example of a "third prompt") that includes input information. Specifically, prompt P1 is a prompt for requesting one or more pieces of related information G related to the input information. Prompt P1 is transmitted to the information management system 20 (information generation system 30). Note that prompt P1 that includes input information includes cases where the input information is directly included, and cases where the input information is indirectly included as information corresponding to the input information (for example, information that has been processed or modified from the input information).

[0032] Secondly, the prompt generation unit 112 generates prompts Pz2 and P3 that include the request information. The prompts Pz2 and P3 that include the request information include both cases where the request information is directly included, and cases where the request information is indirectly included as information corresponding to the request information (for example, information that has been processed or modified from the request information).

[0033] Specifically, prompts Pz2 and P3 are prompts to request output information W in response to the requested information. Output information W is candidate information for response information R (i.e., the answer). In other words, prompts Pz2 and P3 can also be rephrased as prompts to request response information R.

[0034] Prompts Pz2 and P3 are sent to the information management system 20 (information provision system 40). One or more output pieces of information (hereinafter referred to as "first output pieces of information") W1 are generated for prompt Pz2, and one or more output pieces of information (hereinafter referred to as "second output pieces of information") W2 are generated for prompt P3. The information management system 20 uses the multiple output pieces of information W (W1, W2) generated to generate response information R for the request information. Details of the output pieces of information W and response information R will be described later.

[0035] In the first embodiment, one or more first output information W1 are actually generated for prompt P2, which is generated from prompt Pz2. The generation of prompt P2 is performed by the information provision system 40. In the first embodiment, a configuration is illustrated in which the information contained in prompt P2 and the information contained in prompt P3 are different. Request information and identification information are included in common to both prompt P2 and prompt P3.

[0036] On the other hand, prompt P2 includes reference information that is referenced in the generation of the first output information W1, in addition to the request information and identification information. In the information provision system 40 described later, prompt P2 is generated when reference information is added to prompt Pz2 transmitted from the terminal device 10. The reference information is information that the information processing system 100 refers to when generating the first output information W1 (candidate response information). Specifically, the reference information is, for example, information that describes the target product in detail. For example, a document describing the use, purpose, function, configuration, technical specifications, and usage / setting method of the target product is exemplified as reference information. In the first embodiment, prompt P2 is an example of the "first prompt", and prompt P3 is an example of the "second prompt".

[0037] The acquisition unit 113 acquires information generated by the various prompts P1-P3 described above. In the first embodiment, the acquisition unit 113 acquires information generated by prompts P1-P3 from the information management system 20 by receiving it via the communication device 13.

[0038] Firstly, the acquisition unit 113 acquires one or more related pieces of information G generated in response to a prompt P1 containing input information. Secondly, the acquisition unit 113 acquires response information R generated in response to the input information and the request information which is selected by the user U from among the one or more related pieces of information G.

[0039] The presentation unit 114 presents various types of information (for example, related information G and response information R) to the user U. Specifically, the presentation unit 114 presents various types of information to the user U by displaying them on the display device 14. However, the presentation of information by the presentation unit 114 is not limited to display; for example, it may also be presented by voice.

[0040] Firstly, the presentation unit 114 presents the user U with one or more related pieces of information G generated by the prompt P1. Specifically, one or more related pieces of information G are presented to the user U along with the input information.

[0041] [Information generation system 30] Figure 4 is a block diagram illustrating the configuration of the information generation system 30. The information generation system 30 is a device for generating one or more related pieces of information G related to input information. For example, a server system that can communicate with a terminal device 10 via a communication network 900 is exemplified as the information generation system 30.

[0042] As illustrated in Figure 4, the information generation system 30 comprises, for example, a control device 31, a storage device 32, and a communication device 33.

[0043] The control device 31 is one or more processors that control the operation of the information generation system 30. Specifically, the control device 31 is composed of one or more types of processors, such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), SPU (Sound Processing Unit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), or ASIC (Application Specific Integrated Circuit).

[0044] The storage device 32 is one or more memories that store the program executed by the control device 31 and various data used by the control device 31. For example, known recording media such as semiconductor recording media and magnetic recording media, or a combination of multiple types of recording media, can be used as the storage device 32. For example, a portable recording media that is attached to and detached from the information generation system 30, or a recording media that the control device 31 can access via the communication network 900 (e.g., cloud storage), may be used as the storage device 32.

[0045] The communication device 33 is a communication device that communicates with the terminal device 10 via the communication network 900. Alternatively, a separate communication device 33 may be connected to the information generation system 30 by wire or wireless connection.

[0046] Figure 5 is a block diagram illustrating the functions realized by the control device 31 executing a program stored in the storage device 32. The control device 31 of the first embodiment realizes multiple functions (acquisition unit 311, related information generation unit 312) for generating one or more related pieces of information G related to the input information.

[0047] The acquisition unit 311 acquires input information transmitted from the terminal device 10 by receiving it via the communication device 33. Specifically, the acquisition unit 311 acquires a prompt P1 containing the input information.

[0048] The related information generation unit 312 generates one or more related information G related to the input information. In the first embodiment, a trained generation model M1 is used to generate the related information G. The related information generation unit 312 generates one or more related information G by inputting a prompt P1 containing the input information to the generation model M1. The prompt P1 can also be described as a prompt that instructs the generation model M1 to generate one or more related information G related to the input information.

[0049] The one or more pieces of related information G generated by the related information generation unit 312 are transmitted to the terminal device 10 by the communication device 33 and presented to the user U. As described above, one or more pieces of related information G are presented to the user U along with the input information.

[0050] The generative model M1 is composed of, for example, a Transformer model (sequence transformation model). However, the generative model M1 may be composed of any other form of neural network (for example, a recurrent neural network or a convolutional neural network). Furthermore, the generative model M1 may be composed of a combination of multiple types of deep neural networks.

[0051] In the first embodiment, an example is provided in which a large-scale language model (LLM), pre-trained using a large amount of text data and in which the probability of word occurrence is modeled, is used as the generative model M1. The large-scale language model is capable of natural language processing. In particular, it is preferable to use an interactive large-scale language model, which has been fine-tuned using a dialogue dataset, as the generative model M. An interactive large-scale language model can generate dialogue-style text. As an interactive large-scale language model, any known model such as ChatGPT (registered trademark, manufactured by OpenAI), Bard (manufactured by Google), or Bing AI (manufactured by Microsoft) can be used as the generative model M1. It is preferable that the generative model M1 is fine-tuned using a dataset that includes multiple combinations of expected input information (prompt P1) and related information G (answer) for that input information. Alternatively, a pre-trained model that has learned the relationship between input information and related information G by machine learning may be used as the generative model M1.

[0052] As can be understood from the above explanation, the generative model M1 outputs statistically valid related information G for the input information. The multiple parameters that define the generative model M1 are set by machine learning (especially deep learning) using multiple training data and stored in the storage device 32. In the third embodiment, the generative model M1 can refer to the database D2 described later, and the information in database D2 can be used as training data for the generative model M1.

[0053] [Information Provision System 40] Figure 6 is a block diagram illustrating the configuration of the information provision system 40. The information provision system 40 is a device that generates response information R in response to request information corresponding to input information. In other words, the information provision system 40 generates the answer to the inquiry. For example, a computer system that can communicate with a terminal device 10 via a communication network 900 is exemplified as the information provision system 40.

[0054] As illustrated in Figure 6, the information provision system 40 comprises a first processing unit 41, a second processing unit 42, an information provision device 43, and an information acquisition device 44. The information provision device 43 can communicate with each of the first processing unit 41 and the second processing unit 42 via the communication network 900.

[0055] The information acquisition device 44 is a server system for generating prompt P2 from prompt Pz2 transmitted from the terminal device 10. In the first embodiment, the information acquisition device 44 generates prompt P2 by adding reference information to prompt Pz2. That is, prompt P2 is a prompt that includes request information, identification information, and reference information. The information acquisition device 44 can communicate with the terminal device 10 and the first processing device 41 via the communication network 900. For example, the information acquisition device 44 is composed of a control device 441, a storage device 442, and a communication device 443.

[0056] The communication device 443 receives the prompt Pz2 transmitted from the terminal device 10 and transmits the prompt P2 generated from the prompt Pz2 to the first processing unit 41.

[0057] The control device 441 generates prompt P2 from prompt Pz2. Specifically, it obtains reference information corresponding to the request information and identification information contained in prompt Pz2, and generates prompt P2 by adding this reference information to prompt Pz2. The method for obtaining the reference information is not particularly limited, but in the following description, a configuration is given in which the control device 441 obtains the reference information using a data table (hereinafter referred to as the "reference information table") Dq that has been registered in advance in the storage device 442.

[0058] The reference information table Dq registers reference information associated with each combination of identification information and a string that may be included in the request information (hereinafter referred to as the "target string"). The target string that may be included in the request information is, for example, a string (including words) that represents a function that may be the target of the inquiry for the target product. As illustrated in Figure 7, in the reference information table Dq, different reference information is associated with each combination of identification information and a target string. Note that common reference information may be registered for multiple combinations, or multiple pieces of reference information may be registered for a single combination. Then, in the reference information table Dq, reference information is obtained when the string in the request information contained in prompt Pz2 matches (or has a high degree of similarity to) the identification information. However, the control device 441 may also obtain reference information by inputting the request information and identification information into the generation model. This generation model is composed of a Transformer model or an arbitrary form of neural network, for example, similar to the generation model M1.

[0059] Figure 8 is a block diagram illustrating the configuration of the first processing unit 41. The first processing unit 41 is a server system that generates one or more first output information W1 in response to a prompt P2. As described above, the first output information W1 is a candidate for response information R (answer). Specifically, the first processing unit 41 comprises a control device 411, a storage device 412, and a communication device 413.

[0060] The control device 411 is one or more processors that control the operation of the first processing unit 41. Specifically, the control device 411 is composed of one or more types of processors, such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), SPU (Sound Processing Unit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), or ASIC (Application Specific Integrated Circuit).

[0061] The storage device 412 is one or more memories that store programs executed by the control device 411 and various data used by the control device 411. For example, known recording media such as semiconductor recording media and magnetic recording media, or a combination of multiple types of recording media, can be used as the storage device 412. For example, a portable recording media that is attached to and detached from the information provision system 40, or a recording media that the control device 411 can access via the communication network 900 (e.g., cloud storage), may be used as the storage device 412.

[0062] The communication device 413 is a communication device that communicates with the terminal device 10 via the communication network 900. Alternatively, a separate communication device 413, distinct from the information provision system 40, may be connected to the first processing unit 41 by wire or wireless connection.

[0063] Figure 9 is a block diagram illustrating a function realized by the control device 411 executing a program stored in the storage device 412. The control device 411 of the first embodiment realizes multiple functions (acquisition unit 141, processing unit 142) for generating one or more first output information W1 according to the requested information.

[0064] The acquisition unit 141 acquires the request information transmitted from the information acquisition device 44 by receiving it via the communication device 413. Specifically, the acquisition unit 141 acquires the prompt P2 transmitted from the information acquisition device 44. As described above, the prompt P2 includes reference information in addition to the request information and identification information.

[0065] The processing unit 142 generates one or more first output information W1 in response to the request information. A trained generation model M2 is used to generate the first output information W1. Specifically, the processing unit 142 generates one or more first output information W1 by inputting a prompt P2 acquired by the acquisition unit 141 to the generation model M2. In the first embodiment, the generation model M2 generates the first output information W1 in response to the request information, taking into account the identification information and the reference information. The prompt P2 can also be described as a prompt that instructs the generation of one or more first output information W1 in response to the request information, taking into account the identification information and the reference information. Details of the generation model M2 will be described later. The first output information W1 generated by the processing unit 142 is transmitted to the information providing device 43 by the communication device 413.

[0066] Figure 10 is a block diagram illustrating the configuration of the second processing unit 42. The second processing unit 42 is a server system that generates one or more second output information W2 in response to prompt P3. As described above, the second output information W2 is a candidate for response information R (answer), similar to the first output information W1. Specifically, the second processing unit 42 comprises a control device 421, a storage device 422, and a communication device 423.

[0067] The control device 421, the storage device 422, and the communication device 423 employ the same configuration as, for example, the control device 411, storage device 412, and communication device 413 of the first processing unit 41.

[0068] Figure 11 is a block diagram illustrating a function realized by the control device 421 executing a program stored in the storage device 422. The control device 421 of the first embodiment realizes multiple functions (acquisition unit 241, processing unit 242) for generating one or more second output information W2 according to the requested information.

[0069] The acquisition unit 241 acquires request information transmitted from the terminal device 10 by receiving it via the communication device 423. Specifically, the acquisition unit 241 acquires prompt P3 transmitted from the terminal device 10. As described above, prompt P3 includes request information and identification information. In the first embodiment, reference information is not included in prompt P3.

[0070] The processing unit 242 generates one or more second output information W2 for the requested information. A trained generation model M3 is used to generate the second output information W2. Specifically, the processing unit 242 generates one or more second output information W2 by inputting a prompt P3 acquired by the acquisition unit 241 to the generation model M3. In the first embodiment, the generation model M3 generates one or more second output information W2 for the requested information, taking identification information into account. Prompt P3 can also be described as a prompt that instructs the generation of one or more second output information W2 for the requested information, taking identification information into account. The second output information W2 generated by the processing unit 242 is transmitted to the information providing device 43 by the communication device 423.

[0071] The generative models M2 and M3 are described below. Generative models M2 and M3 are composed of, for example, Transformer models, similar to those described above for generative model M1. In the first embodiment, we illustrate a configuration in which large-scale language models, pre-trained using a large amount of text data (training data) and in which the probability of word occurrence is modeled, are used as generative models M2 and M3.

[0072] Large-scale language models are capable of natural language processing. In particular, it is preferable to use interactive large-scale language models, which have been fine-tuned using dialogue datasets, as generative models M2 and M3. Interactive large-scale language models can generate dialogue-style text. As interactive large-scale language models, any known model such as ChatGPT (OpenAI), Bard (Google), or Bing AI (Microsoft) can be used as generative models M2 and M3. Generative models M2 and M3 may be the same type of model or different types of models. The various parameters that define generative models M2 and M3 are set by machine learning (especially deep learning) using multiple training datasets and stored in memory devices 412 and 422.

[0073] To establish the generative model M2, various types of data related to the target product are used as training data. In the first embodiment, for example, a dataset containing multiple combinations of anticipated request information (prompt P2) and first output information W1 (response) for that request information is used as training data. In particular, highly reliable information related to the target product is preferably used as training data. Alternatively, the generative model M2 may be established by fine-tuning a generative model that has been previously trained with a large amount of text data using training data related to the target product.

[0074] Reliable information regarding the target product includes, for example, information collected by the information collection system 50 described later (information stored in databases D1 and D2). Information made public or collected by the manufacturer or distributor of the target product is also considered reliable information regarding the target product. Furthermore, the generation model M2 can also refer to the information collected by the information collection system 50 when generating the first output information W1.

[0075] In establishing the generative model M3 of the first embodiment, various types of data that are generally publicly available (e.g., published on websites, books, academic papers, and newspapers) are mainly used as training data. The training data used to establish the generative model M3 may include information about the target product that is generally publicly available, but it is information that covers a wide range of topics other than the target product. The generative model M3 of the first embodiment is trained with a wider range of information than the generative model M2, but it can also be said that it is a model that is not trained using highly reliable information about the target product as training data, unlike the generative model M2.

[0076] As can be understood from the above explanation, in the first embodiment, the training data for generative model M2 and generative model M3 differs. Generative model M2 uses information specific to the target product (i.e., information selected as relevant to a specific target) as training data, while generative model M3 uses a wide range of publicly available information as training data. In other words, the training data used to train generative model M2 is more limited (limited in content) than the training data used to train generative model M3. Generative model M2 can generate answers specific to the information of the target product (first output information W1), while generative model M3 can generate answers (second output information W2) that take into account a wide range of information other than the target product. However, in establishing generative model M2, in addition to various data related to the target product, a wide range of publicly available information may also be used as training data, similar to generative model M3.

[0077] Figure 12 is a block diagram illustrating the configuration of the information providing device 43. The information providing device 43 is a server system that generates response information R in response to requested information. Specifically, the information providing device 43 comprises a control device 431, a storage device 432, and a communication device 413.

[0078] The control device 431, the storage device 432, and the communication device 433 employ the same configuration as, for example, the control device 411, storage device 412, and communication device 413 of the first processing unit 41.

[0079] Figure 13 is a block diagram illustrating the functions realized by the control device 431 executing a program stored in the storage device 432. The control device 431 of the first embodiment realizes multiple functions (acquisition unit 341, index calculation unit 342, response generation unit R43) for generating response information R to request information.

[0080] The acquisition unit 341 acquires one or more first output information W1 transmitted from the first processing unit 41 and one or more second output information W2 transmitted from the second processing unit 42 by receiving them via the communication device 433.

[0081] The index calculation unit 342 identifies an index K indicating the validity of the requested information for each of the one or more first output information W1s acquired by the acquisition unit 341 and each of the one or more second output information W2s acquired by the acquisition unit 341. In other words, an index K is identified for each output information W (for each first output information W1 and each second output information W2).

[0082] Indicator K is an index that represents how appropriate (valid) the output information W is to the requested information, and can be a numerical value (e.g., a score). For example, a higher score indicates higher validity of the output information W, and a lower score indicates lower validity of the output information W.

[0083] In the first embodiment, a generation model M4 is used to calculate the index K. Specifically, the index calculation unit 342 inputs the request information and the first output information W1 for each of the first output information W1s to the generation model M4, and inputs the request information and the second output information W2 for each of the second output information W2s to identify the index K for each output information W.

[0084] The generative model M4, like the generative model M1 described above, is composed of a Transformer model, and may use, for example, a large-scale language model or a pre-trained model that has learned the relationship between output information W, request information and index K through machine learning. When the generative model M4 is a Transformer model, it is preferable that it is trained on a dataset containing multiple combinations of expected output information W and index K for said output information W. As can be understood from the above explanation, the generative model M4 outputs an index K that is statistically valid for the output information W. The multiple parameters that define the generative model M4 are set by machine learning (especially deep learning) using multiple training datasets and stored in the memory device 432.

[0085] The response generation unit 343 generates response information R for the request information. Specifically, the response generation unit 343 generates response information R for the request information in accordance with one or more first output information W1 generated by the generation model M2 in response to prompt P2 and one or more second output information W2 generated by the generation model M3 in response to prompt P3.

[0086] In the first embodiment, the response generation unit 343 generates response information R according to an index K specified for each of the one or more first output information W1s and each of the one or more second output information W2s. Specifically, the response generation unit 343 generates response information R according to the result of comparing the index K specified for each of the one or more first output information W1s and each of the one or more second output information W2s with a threshold. In this embodiment, the method of generating response information R is varied according to the number of output information Ws among the one or more first output information W1s and one or more second output information W2s in which the index K exceeds the threshold.

[0087] The method for generating response information R differs depending on whether there is one output information W among one or more first output information W1 and one or more second output information W2 in which index K exceeds the threshold, whether there are multiple output information W among one or more first output information W1 and one or more second output information W2 in which index K exceeds the threshold, or whether there is no output information W among one or more first output information W1 and one or more second output information W2 in which index K exceeds the threshold. The specific method by which the response generation unit 343 generates response information R for these three cases will be described later. The threshold is arbitrarily set by the administrator of the information provision system 40. The response information R identified by the response generation unit 343 is transmitted to the terminal device 10 by the communication device 433.

[0088] As can be understood from the above explanation, the response information R is information generated in response to one or more first output information W1 generated by a trained generative model M2 in response to a prompt P2 containing request information, and one or more second output information W2 generated by a trained generative model M3 in response to a prompt P3 containing request information.

[0089] Figure 14 is a flowchart illustrating the specific steps of the processing performed by the entire information processing system 100. User U of the terminal device 10 inputs input information to request (inquire about) desired information about the target product by operating the operating device 15. Figure 15 illustrates an example of input information entered by user U. Figure 15 illustrates input information indicating an inquiry about an error that occurred in function X of the target product.

[0090] The reception unit 111 of the terminal device 10 receives input information entered by user U (SA1). The prompt generation unit 112 generates a prompt P1 containing the input information (SA2) and transmits the prompt P1 from the communication device 13 to the information generation system 30 (SA3). As described above, prompt P1 is a prompt P for requesting one or more related pieces of information G related to the input information.

[0091] The acquisition unit 311 of the information generation system 30 acquires the prompt P1 transmitted from the terminal device 10 by receiving it via the communication device 33 (SB1). The related information generation unit 312 generates one or more related pieces of information G for the prompt P1 acquired by the acquisition unit 311 (SB2). Statistically valid related pieces of information G (related pieces of information G estimated to be highly relevant based on the content of the input information) are generated from the content of the input information. The related information generation unit 312 transmits the generated one or more pieces of related information G from the communication device 33 to the terminal device 10 (SB3).

[0092] Figure 15 also illustrates an example of related information G generated by the related information generation unit 312. Figure 15 illustrates a case where three related information G (G1, G2, G3) are generated. As illustrated in Figure 15, related information G is information related (similar) to the input information and, like the input information, indicates a query regarding function X. For example, related information G1 is assumed to be a more concrete representation of the content of the input information (e.g., an error that occurred in function X or the information requested), and related information G2 and G3 are assumed to be related to the content of the input information (e.g., content regarding errors that occurred in functions X1 and X2 related to function X).

[0093] The acquisition unit 113 of the terminal device 10 receives one or more related pieces of information G transmitted by the information generation system 30 via the communication device 13 (SA4). The presentation unit 114 presents the input information and one or more related pieces of information G to the user U by displaying them on the display device 14 (SA5). That is, the input information and three pieces of related pieces of information G (G1, G2, G3) exemplified in Figure 15 are displayed on the display device 14. The user U selects the information that they consider most appropriate from the input information and three pieces of related pieces of information G displayed on the display device 14 as requested information. The selection of requested information is performed by operating the operating device 15. The reception unit 111 receives the requested information that the user U has entered (selected) by operating the operating device 15 (SA6).

[0094] Furthermore, user U may modify the input information and selected information from one or more related information G before using it as requested information. Alternatively, the information selected by user U may be processed (e.g., converted or combined with other information) in the terminal device 10 before being used as requested information. In other words, the requested information may be the input information or related information G itself, or it may be information corresponding to the input information or related information G.

[0095] Furthermore, user U inputs identification information to identify the subject of the inquiry (target product) by operating the operating device 15. For example, the user U may input identification information (e.g., the version of the target product) by selecting it from several pre-prepared options, or user U may input arbitrary identification information. The multiple options are displayed on the display device 14. The reception unit 111 receives the identification information input by user U through the operation of the operating device 15 (SA7).

[0096] The prompt generation unit 112 generates prompts Pz2 and P3 containing request information and identification information (SA8). The prompt generation unit 112 transmits the generated prompts Pz2 and P3 from the communication device 13 to the information provision system 40 (SA9).

[0097] When the information provision system 40 (first processing unit 41, second processing unit 42, information provision device 43, information acquisition device 44) receives prompts Pz2 and P3 transmitted from the terminal device 10, it executes a process to generate response information R (SC1). Figure 16 is a flowchart illustrating the detailed procedure of step SC1.

[0098] As illustrated in Figure 16, the control device 441 of the information acquisition device 44 acquires the prompt Pz2 transmitted from the terminal device 10 by receiving it via the communication device 443 (SC01). Next, the control device 441 generates prompt P2 from prompt Pz2 (SC02). Specifically, the control device 441 generates prompt P2, which includes the request information, identification information, and reference information, by adding reference information to prompt Pz2. As described above, the reference information is obtained using the reference information table Dq. For example, reference information is obtained from the reference information table Dq that associates a target string that matches (or has a high degree of similarity to) a string included in the reference information of prompt P2 (hereinafter referred to as the "search string") with identification information that matches (or has a high degree of similarity to) the identification information of prompt P2. For example, if the request information is "I switched from product A to product B. Please tell me the procedure for activating function X," the string "procedure for activating function X" may be used as the search string. Note that multiple reference information can be obtained using multiple different search strings. Then, the control device 441 sends prompt P2 to the first processing unit 41 (SC03).

[0099] The acquisition unit 141 of the first processing unit 41 acquires the prompt P2 transmitted from the information acquisition device 44 by receiving it via the communication device 413 (SC11). The processing unit 142 generates one or more first output information W1 in response to the prompt P2 acquired by the acquisition unit 141 (SC12). Specifically, the processing unit 142 generates one or more first output information W1 by inputting the prompt P2 into the generation model M2. The processing unit 142 transmits the generated one or more first output information W1 to the information providing device 43 (SC13).

[0100] The acquisition unit 241 of the second processing unit 42 acquires the prompt P3 transmitted from the terminal device 10 by receiving it with the communication device 423 (SC21). The processing unit 242 generates one or more second output information W2 for the prompt P3 acquired by the acquisition unit 241 (SC22). Specifically, the processing unit 242 generates one or more second output information W2 by inputting the prompt P3 into the generation model M3. The acquisition unit 241 transmits the generated one or more second output information W2 to the information providing device 43 (SC23). Steps SC11-SC13 and steps SC21-SC23 may be performed in parallel or in reverse order.

[0101] Figure 17 illustrates the first output information W1 and the second output information W2. Figure 17 illustrates the case where two first output information W1(W11,W12) and one second output information W2 are generated. Output information W1 and W2 are candidates for response information R (answer).

[0102] The acquisition unit 341 of the information providing device 43 acquires one or more first output information W1 transmitted from the first processing device 41 and one or more second output information W2 transmitted from the second processing device 42 by receiving them via the communication device 433 (SC31, SC32). The order of steps SC31 and SC32 is arbitrary and may be performed in parallel.

[0103] The index calculation unit 342 identifies an index K (SC33) that indicates the validity of the requested information for each of the one or more first output information W1s and each of the one or more second output information W2s. In the following explanation, the case where index K is a score (out of 100 points) is used as an example. The generation model M4 is used to identify index K. Specifically, index K is identified by inputting the requested information and the output information W for each of the multiple output information Ws (W11, W12, W2) into the generation model M4. As illustrated in Figure 17, index K (score) is identified for each output information W.

[0104] The response generation unit 343 executes a process (hereinafter referred to as "response generation process") to generate response information R according to one or more first output information W1s and one or more second output information W2s acquired by the acquisition unit 341 (SC34). In the response generation process of this embodiment, response information R is generated according to an index K specified for each of the one or more first output information W1s and each of the one or more second output information W2s. Figure 18 is a flowchart illustrating the specific steps of the response generation process.

[0105] First, the response generation unit 343 determines whether there is any output information W among the multiple output information W(W1, W2) in which index K exceeds the threshold (SC34-1). If there is any output information W among the multiple output information W in which index K exceeds the threshold (SC34-1; YES), the response generation unit 343 generates response information R (SC34-2). If there is only one output information W in which index K exceeds the threshold, the response information R is generated according to that one output information W. The response information R generated according to the output information W may be the output information W itself, or it may be information that has been processed or modified from the output information W, or information that has other information added to the output information W.

[0106] If there are multiple output information Ws where the index K exceeds the threshold, response information R is generated according to those multiple output information Ws. The response information R generated according to the multiple output information Ws may be, for example, the information itself that combines the multiple output information Ws, or it may be, for example, information that has been processed or modified from the combined information, or information that has other information added to the multiple output information Ws.

[0107] On the other hand, if there is no output information W among the multiple output information W in which the index K exceeds the threshold (SC34-1; NO), the response generation unit 343 determines whether the process of SC34-1 has been repeated a predetermined number of times (for example, twice) (SC34-3). If the process of SC34-1 has not been repeated a predetermined number of times (SC34-3; NO), the process returns to step SC02 in Figure 16, and prompt P2 is generated again. Specifically, the information acquisition device 44 generates prompt P2 with the reference information changed to other reference information. In the first embodiment, the information acquisition device 44 uses the reference information table Dq to change the search string and then acquires the reference information again. For example, if the request information is "I switched from product A to product B. Please tell me the procedure for activating function X," then, for example, if the string "procedure for activating function X" was used as the search string in the previous step SC02, then the string "procedure for activating function X in product B" may be used as the search string. In other words, the search string may be a string that combines multiple words included in the request information. The modified search string may be, for example, a string that combines the previous search string with other words or strings (i.e., a string that partially overlaps before and after the modification), or a string that is completely different from the previous search string. The modified prompt P2 may include the reference information of the original prompt P2 along with the newly acquired reference information. On the other hand, the request information and identification information are maintained. The modified prompt P2 is sent to the first processing unit 41 (SC03).

[0108] Next, the first processing unit 41 repeats steps SC11-SC13 in Figure 16. Then, the information providing device 43 obtains one or more first output information W1 generated for the changed prompt P2 (SC31) and executes steps SC33 and SC34. The process of changing prompt P2 can also be rephrased as the process of updating the first output information W1. Note that the steps SC32, which receive the second output information W2, and SC33, which calculate the index K for the second output information W2, are omitted because there is no change in prompt P3. In other words, the second output information W2 is not updated and remains unchanged.

[0109] The response generation unit 343 of the information providing device 43 executes step SC34-1 in Figure 18 for each of the one or more updated first output information W1s identified by index K and each of the one or more maintained second output information W2s identified by index K (i.e., the index K identified in the previous step SC33). That is, if there is no output information W whose index K exceeds the threshold, the response generation unit 343 generates response information R according to the index K identified for each of the one or more first output information W1s newly generated by the generation model M2 in response to a prompt P2 in which the reference information has been changed to other reference information, and for each of the one or more maintained second output information W2s.

[0110] In the example shown in Figure 17, for example, if the threshold is set to 80 points, response information R is generated according to the first output information W11; if the threshold is set to 70 points, response information R is generated according to the first output information W11 and the first output information W12; and if the threshold is set to 95 points, response information R is not generated, and prompt P2 is updated.

[0111] In contrast, if the process of SC34-1 in Figure 18 is repeated a predetermined number of times (SC34-3; YES), the response generation unit 343 generates response information R (SC34-2). In this case, the response information R is generated as a response indicating that valid information for the requested information could not be generated (i.e., a response indicating that the desired information cannot be provided to the user U). Note that if there is no output information in which the index K exceeds the threshold, the system may proceed to step SC34-2 without performing the process in step SC34-3 and generate response information R (a response indicating that valid information for the requested information could not be generated).

[0112] As can be understood from the above explanation, the response generation unit 343 generates response information R according to the result of comparing an index K, which is specified for each of the one or more first output information W1s and one or more second output information W2s, with a threshold. The method of generating the response information R differs depending on the number of output information Ws in which the index K exceeds the threshold. Specifically, the method of generating the response information R differs depending on whether there is one output information W in which the index K exceeds the threshold among the one or more first output information W1s and one or more second output information W2s, whether there are multiple output information Ws in which the index K exceeds the threshold, or whether there are no output information Ws in which the index K exceeds the threshold.

[0113] As illustrated in Figure 14, the response information R generated by the response generation unit 343 is transmitted from the communication device 433 to the terminal device 10 (SC2). The acquisition unit 113 of the terminal device 10 acquires the response information R by receiving it from the communication device 13 (SA10). The presentation unit 114 then presents the response information R acquired by the acquisition unit 113 to the user U by displaying it on the display device 14 (SA11).

[0114] In the first embodiment, response information R generated in response to one or more first output information W1 generated by the generative model M2 in response to prompt P2 and one or more second output information W2 generated by the generative model M3 in response to prompt P3 can be obtained. Therefore, compared to a configuration in which, for example, response information R generated in response to output information W generated by one generative model is obtained and provided to the user U, it becomes possible to present user U with more appropriate response information R in response to the request information from user U.

[0115] <Second Embodiment> A second embodiment will now be described. For elements whose function is the same as in the first embodiment in each of the embodiments exemplified below, the reference numerals used in the description of the first embodiment will be reused, and detailed explanations of each will be omitted as appropriate.

[0116] Figure 19 is a block diagram illustrating the configuration of the information processing system 100 according to the second embodiment. The information processing system 100 according to the second embodiment has an information collection system 50 added to the information management system 20 of the first embodiment. The other configurations are the same as those of the first embodiment. The information management system 20 is a computer system for collecting various types of information after response information R is presented to the user U.

[0117] In the second embodiment, after the response information R for the request information is presented, the validity of the response information R is determined. In the second embodiment, for example, user U determines (inputs) an evaluation of the validity (hereinafter referred to as "feedback evaluation") of the response information R generated for the request information by operating the operating device 15. The feedback evaluation includes, for example, an index that indicates how appropriate the response information R was for the request information (hereinafter referred to as "evaluation index") and a sentence that user U can arbitrarily input regarding the validity of the response information R (hereinafter referred to as "evaluation sentence"). The evaluation index may be, for example, an index selected from a set of options that divide the evaluation of validity into multiple levels (e.g., "good," "average," "bad"), or it may be a score that user U can arbitrarily input. However, the feedback evaluation may be performed by a processing device (not shown) instead of user U. The processing device outputs the feedback evaluation for the response information R using, for example, a generative model. Such a generative model may be, for example, a model in which the relationship between the response information R and the feedback evaluation has been machine-learned, or a large-scale language model.

[0118] The receiving unit 111 of the terminal device 10 receives the identified feedback evaluation. The terminal device 10 then transmits information F including the feedback evaluation (hereinafter referred to as "evaluation information") to the information collection system 50. The evaluation information F includes the response information R that was the subject of evaluation and the request information corresponding to the response information R. Furthermore, the evaluation information F in the second embodiment includes identification information. The evaluation information F may also include other information (for example, input information, related information G, reference information, and output information W).

[0119] Figure 20 is a block diagram illustrating the configuration of the information gathering system 50. The information gathering system 50 is a computer system that collects various types of information after presenting response information R to the user U. Specifically, the information gathering system 50 comprises a control device 51, a storage device 52, and a communication device 53.

[0120] The control device 51 is one or more processors that control the operation of the information collection system 50. The storage device 52 is one or more memories that store programs executed by the control device 51 and various data used by the control device 51. The communication device 53 is a communication device that communicates with the terminal device 10 via the communication network 900. The control device 51, storage device 52, and communication device 53 employ a configuration similar to that described for the control device 411, storage device 412, and communication device 413 of the first processing unit 41, for example.

[0121] Figure 21 is a block diagram illustrating the functions realized by the control device 51 executing a program stored in the storage device 52. The control device 51 implements multiple functions (acquisition unit 511, collection unit 512) for collecting various types of information after presenting response information R to the user U.

[0122] The acquisition unit 511 receives evaluation information F transmitted from the terminal device 10 via the communication device 53. In other words, the acquisition unit 511 (an example of the "first acquisition unit") functions as an element that performs an evaluation of the validity (feedback evaluation) of the response information R generated in response to the request information corresponding to the input information entered by the user U, and acquires the request information and the response information R.

[0123] The collection unit 512 stores the evaluation information F acquired by the acquisition unit 511 in the storage device 60. The feedback evaluation, request information, response information R, and identification information included in the evaluation information F are stored in the storage device 60. For example, the feedback evaluation, request information, response information R, and identification information are registered in association with each other in the database D1 stored in the storage device 60.

[0124] The storage device 60 is, for example, a recording medium (e.g., cloud storage) that can be accessed by the information collection system 50 via the communication network 900. As described above in the first embodiment, it is preferable that the storage device 60 is accessible by the processing unit 142 (generation model M2) of the first processing unit 41. Alternatively, the storage device 52 of the information collection system 50 may be used as the storage device 60.

[0125] Before storing the evaluation information F in the storage device 60, there is a process to verify the contents of the evaluation information F. First, it is checked whether the feedback evaluation is appropriate or not. For example, based on the request information, identification information, and response information R, it is checked whether the feedback evaluation is appropriate or not (i.e., whether the evaluation is valid or not). This check on whether the feedback evaluation is appropriate or not may be performed by, for example, the administrator of the information collection system 50, or by a processing unit composed of a computer system. The processing unit (not shown) checks whether the feedback evaluation is appropriate or not using, for example, a generative model. Such a generative model may include, for example, a model in which the relationship between the request information, identification information, response information R, and feedback evaluation and the validity of the feedback evaluation has been learned through machine learning, or a large-scale language model.

[0126] If the feedback evaluation is deemed inappropriate, it will be appropriately modified. The modification of the feedback evaluation may be performed, for example, by the administrator of the information collection system 50, or by the processing unit. The processing unit (not shown in the figure) modifies the feedback evaluation using, for example, a generative model. Such a generative model may be a machine learning model that studies the relationship between the request information, identification information, response information R and the feedback evaluation and the modified feedback evaluation, or a large-scale language model.

[0127] Secondly, it is checked whether the response information R is appropriate for the requested information. For example, it is checked whether the response information R is appropriate (i.e., whether the response information R is valid) based on the requested information, identification information, and feedback evaluation. This check of whether the response information R is appropriate may be performed by, for example, the administrator of the information collection system 50, or by a processing unit composed of a computer system. The processing unit (not shown) checks whether the response information R is appropriate using, for example, a generative model. Such a generative model may include, for example, a model in which the relationship between the requested information, identification information, and response information R and the validity of the response information R is mechanically represented, or a large-scale language model.

[0128] If the response information R is deemed inappropriate, the response information R is appropriately modified. The modification of the response information R may be performed, for example, by the administrator of the information collection system 50, or by the processing unit. The processing unit (not shown) modifies the response information R using, for example, a generative model. Such a generative model may include, for example, a machine learning model that has studied the relationship between the request information, identification information, feedback evaluation, and response information R and the modified response information R, or a large-scale language model.

[0129] In this embodiment, if the data collection unit 512 determines that the feedback evaluation is inappropriate, it stores the appropriately corrected feedback evaluation in the storage device 60. If the response information R is determined to be inappropriate for the input information, it stores the appropriately corrected response information R in the storage device 60. If the feedback evaluation and response information R are appropriate, they are stored in the storage device 60 without modification.

[0130] Furthermore, the collection unit 512 may store in the storage device 60 whether the feedback evaluation is appropriate or not, and the feedback evaluation before correction (if the feedback evaluation is not appropriate). Similarly, the collection unit 512 may store in the storage device 60 whether the response information R is appropriate or not, and the response information R before correction (if the response information R is not appropriate).

[0131] According to the configuration of the third embodiment, if the feedback evaluation is deemed inappropriate, the appropriately corrected feedback evaluation is stored in the storage device 60, and if the response information R is deemed inappropriate for the request information, the appropriately corrected response information R is stored in the storage device 60. The information stored in the storage device 60 (database D1) can be used as training data for the generation model M2 of the first processing device 41, or can be referenced by the first processing device 41. Therefore, the first processing device 41 can generate the first output information W1 with higher accuracy. For example, a combination of request information and response information R (i.e., the first output information W1) is used as training data for the generation model M2. Alternatively, the generation model M2 may refer to database D1 to obtain the first output information W1 for the request information.

[0132] <Third Embodiment>

[0133] Figure 22 is a block diagram illustrating a function realized by the control device 51 executing a program stored in the storage device 52 of the information collection system 50 according to the third embodiment.

[0134] The control device 51 of the third embodiment provides multiple functions for collecting various information after presenting response information R to the user U, including an acquisition unit 511 and a collection unit 512, as well as a prompt generation unit 513 and an information generation unit 514. The functions of the information generation unit 514 may be mounted on a separate device that can communicate with the information collection system 50.

[0135] Firstly, the acquisition unit 511 of the third embodiment (an example of the "first acquisition unit") acquires the input information entered by the user U. The prompt generation unit 513 generates a prompt P4 (an example of the "fourth prompt") that includes the input information. Specifically, prompt P4 is information that requests the generation of related information related to the input information. In addition, prompt P4 may request the generation of multiple pieces of related information. The related information in the third embodiment is the same information as the related information G described in the first embodiment. That is, the related information, like the input information, is information for requesting the information desired by the user U.

[0136] The prompt P4 generated by the prompt generation unit 513 is input to the information generation unit 514, and related information related to the input information is generated. The information generation unit 514 (an example of the "second acquisition unit") generates (acquires) the related information using the trained generation model M5 (an example of the "fourth generation model").

[0137] Next, the prompt generation unit 513 generates a prompt P5 (an example of the "fifth prompt") containing related information. Specifically, prompt P5 is information requesting the generation of response information to the related information. The response information is information indicating the response to the related information (typically the information requested by the related information). If the related information is information indicating a query, the response information is information indicating the answer to that query. The response information has the same content as the first output information W1 (the answer to the query).

[0138] The prompt P5 generated by the prompt generation unit 513 is input to the information generation unit 514, and response information for the relevant information is generated. The information generation unit 514 (an example of the "third acquisition unit") generates (acquires) the response information using the trained generation model M6 (an example of the "fifth generation model").

[0139] In the third embodiment, the information generation unit 514 that generates related information functions as an element (second acquisition unit) that acquires related information generated by the generation model M5 in response to a prompt P4 containing input information, and related information associated with the input information. The information generation unit 514 that generates response information functions as an element (third acquisition unit) that acquires response information generated by the generation model M6 in response to a prompt P5 containing related information. However, if the functions of the information generation unit 514 are mounted on an external device, the elements that acquire the related information and response information generated by that device function as the second acquisition unit and the third acquisition unit, respectively.

[0140] The collection unit 512 stores the input information acquired by the acquisition unit 511, along with the related information and response information generated by the information generation unit 514, in the storage device 60. For example, the input information, related information, and response information are registered in association with each other in the database D2 stored in the storage device 60. However, storing the input information in the storage device 60 is not mandatory.

[0141] Before storing the input information, related information, and response information in the memory device 60, a check is performed to determine whether the response information is appropriate. For example, based on the content of the related information, it is checked whether the response information is appropriate (i.e., whether the response information is valid or not). This check of whether the response information is appropriate may be performed, for example, by the administrator of the information collection system 50, or by a processing unit composed of a computer system. The processing unit (not shown) checks whether the response information is appropriate using, for example, a generative model. Such a generative model may include, for example, a model in which the relationship between related information, response information, and the validity of the response information is mechanically represented, or a large-scale language model.

[0142] If the response information is deemed inappropriate, it will be appropriately corrected. The correction of the response information may be performed, for example, by the administrator of the information collection system 50, or by the processing unit. The processing unit (not shown in the figure) corrects the response information using, for example, a generative model. Such a generative model may include, for example, a machine learning model that has studied the relationship between related information, response information, and the corrected response information, or a large-scale language model.

[0143] In the third implementation configuration, if the data collection unit 512 determines that the response information is inappropriate for the related information, it stores the appropriately corrected response information in the storage device 60. If the response information is appropriate, it is stored in the storage device 60 without correction. The data collection unit 512 may also store in the storage device 60 whether the response information is appropriate or not, and the response information before correction (if the response information is inappropriate).

[0144] According to the configuration of the third embodiment, if the response information is deemed inappropriate, the appropriately corrected response information is stored in the storage device 60. The information stored in the storage device 60 (database D2) can be used as training data for the information generation system 30 (generation model M1) or can be referenced by the information generation system 30. Therefore, the information generation system 30 can generate related information G with higher accuracy. For example, a combination of input information and related information is used as training data for generation model M1. Also, since the response information has the same content as the first output information W1, a combination of related information (i.e., candidate request information) and response information (i.e., first output information W1) may be used as training data for generation model M2. Furthermore, generation model M1 may refer to database D2 to obtain related information for input information, or generation model M2 may refer to database D2 to obtain first output information W1 for request information.

[0145] Note that generation model M5 and generation model M6 may be a single common generation model. Also, generation model M5 may be common to generation model M1, and generation model M6 may be common to generation model M2. Furthermore, it may be determined whether the related information generated for the input information is appropriate, and if it is determined that the related information is inappropriate, the related information may be corrected and then stored in the storage device 60.

[0146] <Variation> Each of the forms exemplified above can be modified in various ways. Specific examples of modifications are given below. It is also possible to combine two or more forms arbitrarily selected from the following examples.

[0147] (1) In the forms described above, examples were given where the subject of the inquiry was a specific product, but the subject may also be a service. Services include, for example, water, gas, electricity, banking, medical care, accommodation, education, and various transportation services. The content of the reference information and identification information may be changed as appropriate depending on the type of subject. For example, if the subject of the inquiry is a service (e.g., water, gas, electricity), examples of identification information include information to identify the plan and subscriber for that service, and examples of reference information include information collected by the service provider (such as details of the plan the user has contracted or past usage history), or information that the service provider has made publicly available. Furthermore, the subject product is not limited to software, but can include various products such as home appliances, games, and food.

[0148] (2) In each of the above embodiments, some of the functions of the information generation system 30 or the information provision system 40 may be mounted on the terminal device 10. For example, some or all of the related information generation unit 312, processing unit 142, processing unit 242, index calculation unit 342, and response generation unit 343 may be mounted on the terminal device 10. For example, in a configuration in which the response generation unit 343 is mounted on the terminal device 10, the acquisition unit 113 that acquires response information R is an element that generates response information R (i.e., the response generation unit 343). That is, the acquisition unit 113 may be an element that acquires response information R by receiving it from an external device (first embodiment to third embodiment), or an element that acquires response information R by generating it (configuration in which the response generation unit 343 is mounted on the terminal device 10). Similarly, the acquisition unit 113 may be an element that acquires related information G by receiving it from an external device (first to third embodiments), or an element that acquires it by generating related information G (a configuration in which the related information generation unit 312 is mounted on the terminal device 10).

[0149] (3) In each of the above forms, it is not necessary to have a configuration that generates one or more related information G. That is, the input information may be used as requested information without presenting the related information G to the user U. The requested information corresponding to the input information includes the input information itself (including information that has been modified or processed from the input information) and related information G related to the input information (including information that has been modified or processed from the related information G).

[0150] (4) In each of the above-described configurations, it is also conceivable that some or all of the functions of the prompt generation unit 112 may be installed in a device other than the terminal device 10 (for example, the information generation system 30 or the information provision system 40). For example, if the function for generating prompts Pz2 and P3 is installed in the information provision system 40, then the terminal device 10 will transmit the information necessary for generating prompts Pz2 and P3 (for example, request information and identification information) to the information provision system 40.

[0151] Furthermore, in each of the above-described embodiments, the information acquisition device 44 generated prompt P2 from prompt Pz2, but the function of generating prompt P2 may be provided, for example, in the terminal device 10 or the first processing device 41.

[0152] (5) In each of the above-described forms, it is not essential that the prompt P2 contains identification information and reference information. However, the inclusion of identification information and reference information in the prompt P2 enables the generation model M2 to generate the first output information W1 with high accuracy (i.e., with high validity to the requested information). If the prompt P2 does not contain reference information, the information acquisition device 44 is omitted from the information provision system 40, and the prompt Pz2 (an example of the "first prompt") is transmitted directly from the terminal device 10 to the first processing unit 41. Similarly, it is not essential that the prompt P3 contains identification information. The prompt P3 may also contain reference information. Furthermore, the assignment of identification information to the prompts Pz2 and P3 is not limited to the terminal device 10, but may also be performed by the information management system 20.

[0153] (6) In each of the above configurations, it is not essential that the response generation unit 343 generates response information R according to the index K. In the above configurations, the index calculation unit 342 is omitted. For example, a configuration is adopted in which response information R is generated by combining all of the generated multiple output information W, or a configuration is adopted in which response information R is generated by combining one first output information W1 and one second output information W2. Alternatively, the similarity (matching rate) between the first output information W1 and the second output information W2 may be calculated, and response information R may be generated according to the similarity. For example, if the similarity exceeds a predetermined threshold, response information R including the second output information W2 may be generated, and if the similarity falls below the predetermined threshold, response information R including the first output information W1 may be generated.

[0154] (7) In the above-described configuration, the response generation unit 343 may generate response information R according to the output information W (or multiple output information Ws in descending order of index K up to a predetermined rank) in which index K is maximized, among the index K specified for each of the one or more first output information W1 and one or more second output information W2. In other words, it is not essential to generate response information R according to the result of comparing the index K of each output information W with a threshold.

[0155] (8) In each of the above-described forms, the user U may be made able to confirm the processing flow from input information to obtaining the final response information R. For example, the user U may be made able to confirm the input information entered by the user U, related information G generated from the input information, the request information selected by the user U from the input information and related information G, and the response information R for the request information. This information is stored in the storage device 12 of the terminal device 10 or in a storage device accessible from the terminal device 10 via the communication network 900.

[0156] (9) In each of the above-described embodiments, the information generation system 30, the information management system 20, the first processing unit 41, the second processing unit 42, the information providing device 43, the information acquisition device 44, and the information collection system 50 may be a single device or a computer system composed of multiple devices.

[0157] (10) The information processing system (terminal device, information generation system, information management system, information acquisition device, first processing unit, second processing unit, information provision device, information collection system) described above is realized through the cooperation of a computer (specifically, a control device) and a program. The program described above can be provided in a form stored on a computer-readable recording medium and installed on the computer. The recording medium is, for example, a non-transitory recording medium, such as an optical recording medium (optical disc) like a CD-ROM, but may include any known form of recording medium such as a semiconductor recording medium or a magnetic recording medium. Note that a non-transitory recording medium includes any recording medium except for transient propagation signals (transitory, propagating signals), and does not exclude volatile recording media. It is also possible to provide the program to the computer in the form of distribution via a communication network.

[0158] <Note> From the forms exemplified above, the following configuration can be understood, for example.

[0159] [1] A terminal device comprising a receiving unit that receives input information entered by a user, and an acquisition unit that acquires response information generated in response to request information corresponding to the input information, wherein the response information is information generated in response to one or more first output pieces generated by a first generative model trained for a first prompt including the request information, and one or more second output pieces generated by a second generative model trained for a second prompt including the request information.

[0160] With the above configuration, response information can be obtained in response to a first prompt containing request information corresponding to input information entered by the user, one or more first output pieces generated by a trained first generative model in response to the first prompt containing the request information, and one or more second output pieces generated by a trained second generative model in response to a second prompt containing the same request information. Therefore, compared to a configuration in which response information is obtained in response to output information generated by a single generative model in response to a prompt containing request information, for example, it is possible to present the user with more appropriate response information in response to the user's request information.

[0161] [2] The terminal device of [1], wherein the request information is information indicating an inquiry concerning a predetermined object, and the first generating model is a model trained using highly reliable information concerning the said object as training data.

[0162] In the above configuration, since the first model is a model trained using highly reliable information about a given object as training data, it can generate the first output information based on highly reliable information about the object.

[0163] [3] The response information is information generated according to an index specified for each of the one or more first output pieces and each of the one or more second output pieces, the index being an indicator of the validity of the request information [1] terminal device.

[0164] With the above configuration, response information is generated according to an indicator specified for each of the one or more first output pieces and each of the one or more second output pieces. Since this indicator indicates the validity of the requested information, the response information generated while taking the validity of the requested information into account can be presented to the user.

[0165] [4] The response information is generated in accordance with the result of comparing the index and the threshold [3].

[0166] In the above configuration, response information is generated according to the result of comparing the index and the threshold. For example, response information generated according to output information of 1 or more where the index exceeds the threshold (i.e., output information with high validity) can be presented to the user.

[0167] [5] A terminal device [4] in which the method for generating the response information differs depending on the number of output information from each of the one or more first output information and each of the one or more second output information in which the index exceeds the threshold.

[0168] In the above configuration, the user can be presented with response information that is generated in a different way depending on the number of output pieces of information among the one or more first output pieces and the one or more second output pieces, where the index exceeds a threshold.

[0169] [6] A terminal device in which the method for generating the response information differs depending on whether there is one output information among the one or more first output information and the one or more second output information in which the index exceeds the threshold, whether there are multiple output information among the one or more first output information and the one or more second output information in which the index exceeds the threshold, or whether there is no output information among the one or more first output information and the one or more second output information in which the index exceeds the threshold [5].

[0170] With the above configuration, the system can present the user with response information that is generated differently depending on whether there is one output information item among one or more first output information items and one or more second output information items in which the index exceeds the threshold, whether there are multiple output information items among one or more first output information items and one or more second output information items in which the index exceeds the threshold, or whether there is no output information item among one or more first output information items and one or more second output information items in which the index exceeds the threshold.

[0171] [7] The terminal device of [6] generates the response information according to the single output information if there is one output information that exceeds the threshold, and according to the multiple output information if there are multiple output information that exceeds the threshold.

[0172] In the above configuration, if there is one output piece that exceeds the threshold, response information is generated according to that single output piece. If there are multiple output pieces that exceed the threshold, response information is generated according to those multiple output pieces. Therefore, if there are multiple output pieces with high validity, the user can be presented with response information that takes all of those multiple output pieces into account.

[0173] [8] The terminal device of [7] wherein the first prompt includes reference information referenced in the generation of the first output information, and the response information is generated in accordance with the index specified for each of the one or more first output information newly generated by the first generation model for the first prompt in which the reference information has been changed to other reference information, and for each of the one or more second output information, if there is no output information that exceeds the threshold.

[0174] In the above configuration, if there is no output information that exceeds the threshold, response information is generated according to the specified index for each of the one or more first output information pieces newly generated by the first generative model in response to the first prompt in which the reference information has been changed to other reference information, and for each of the one or more second output information pieces. Therefore, if output information with high validity cannot be obtained, one or more new first output information pieces (i.e., updated one or more first output information pieces) can be used to generate response information.

[0175] [9] The acquisition unit is a terminal device [1] to [8] in which a trained third generative model generates a third prompt including the input information, acquires one or more pieces of related information relating to the input information, and the requested information is the information selected by the user from the input information and the one or more pieces of related information.

[0176] In the above configuration, the request information is the information selected by the user from among the input information and one or more related pieces of information. Therefore, the response information is generated with the information desired by the user from among multiple options (input information and one or more related pieces of information) as the request information. Consequently, compared to a configuration in which, for example, the input information entered by the user is used as the request information, it is possible to obtain response information that is in line with the user's intentions.

[0177]

[10] The request information is information indicating an inquiry relating to a predetermined object, and the first prompt includes identification information for identifying the object [1] terminal device

[0178] In the above configuration, the request information is information indicating a query regarding a predetermined object, and the first prompt includes identification information for identifying that object, so response information to the request information can be generated with high accuracy.

[0179]

[11] The first prompt is a terminal device [1] to [7] which includes reference information that is referenced in the generation of the response information.

[0180] In the above configuration, the first prompt includes reference information that is used in generating the response information, thus enabling the generation of response information to the request information with high accuracy.

[0181]

[12] A terminal device comprising a receiving unit that receives input information entered by a user, and an acquisition unit that uses a trained third generative model to generate information in response to a third prompt containing the input information, and acquires one or more pieces of related information related to the input information.

[0182] In the above configuration, the system includes a receiving unit that receives input information entered by the user, and an acquisition unit that uses a trained third generative model to generate a third prompt containing the input information and acquires one or more pieces of related information related to that input information. Therefore, it is possible to provide the user with one or more pieces of related information related to the input information. Consequently, for example, it is possible to generate a prompt that includes request information, which is the information desired by the user from among multiple options (input information and one or more pieces of related information). In turn, it is possible to present the user with more appropriate response information in response to the request information.

[0183]

[13] An information providing device comprising a response generation unit that generates response information to a request information in response to a first prompt containing request information corresponding to input information entered by a user, one or more first output pieces generated by a trained first generative model in response to a first prompt containing request information, and one or more second output pieces generated by a trained second generative model in response to a second prompt containing the request information.

[0184] In the above configuration, response information to a request is generated based on one or more first output pieces generated by a trained first generative model in response to a first prompt containing request information corresponding to the input information entered by the user, and one or more second output pieces generated by a trained second generative model in response to a second prompt containing request information. Therefore, compared to a configuration in which, for example, response information is obtained based on the output information generated by a single generative model in response to a prompt containing request information, more appropriate information can be presented to the user in response to the request information.

[0185]

[14] An information generation system comprising an association information generation unit that generates one or more association pieces of information related to input information by inputting a third prompt containing input information entered by a user into a trained third generative model.

[0186] In the above configuration, by inputting a third prompt containing the input information entered by the user into the third generative model, one or more related pieces of information are generated, thus providing the user with one or more pieces of related information related to the input information. Therefore, for example, it is possible to generate a prompt containing request information, which is the information desired by the user from among multiple options (input information and one or more pieces of related information). In turn, it is possible to present the user with more appropriate response information in response to the request information.

[0187]

[15] An information gathering system comprising: an evaluation of the validity of response information generated in response to request information corresponding to input information entered by a user; a first acquisition unit that acquires the request information and the response information; and a collection unit that stores the evaluation, the request information and the response information in a storage device.

[0188] In the above configuration, the validity evaluation of the response information generated in response to the request information entered by the user, along with the request information and the response information, are stored in the memory. Therefore, the evaluation, request information, and response information stored in the memory can be used, for example, as training data for the first generative model, or referenced when the first generative model generates the first output information. In other words, the first generative model can generate the first output information with high accuracy. Consequently, more appropriate response information can be presented to the user in response to the request information.

[0189]

[16] The information gathering system

[14] wherein if the collection unit determines that the evaluation is inappropriate, it stores the appropriately corrected evaluation in the storage device, and if the response information is determined to be inappropriate for the request information, it stores the appropriately corrected response information in the storage device.

[0190] In the above configuration, if an evaluation is deemed inappropriate, the appropriately corrected evaluation is stored in the memory device, and if the response information is deemed inappropriate for the requested information, the appropriately corrected response information is stored in the memory device. Therefore, the effect of enabling the first generation model to generate the first output information with high accuracy becomes significant.

[0191]

[17] An information gathering system comprising: a first acquisition unit that acquires related information related to a fourth prompt containing input information entered by a user, which is generated by a trained fourth generative model; a second acquisition unit that acquires response information generated by a trained fifth generative model in response to a fifth prompt containing the related information; and a collection unit that stores the related information and the response information in a storage device.

[0192] In the above configuration, a trained fourth generative model generates related information for a fourth prompt containing input information entered by the user, and this related information, along with the response information generated by a trained fifth generative model for a fifth prompt containing this related information, are stored in the memory. Therefore, the related information and response information stored in the memory can be used, for example, as training data for a third generative model, or referenced when the third generative model generates related information. Consequently, the third generative model can generate related information with high accuracy.

[0193]

[18] If the collection unit determines that the response information is inappropriate for the related information, the information collection system

[16] stores the appropriately corrected response information in the storage device.

[0194] In the above configuration, if the response information is deemed inappropriate for the related information, the appropriately corrected response information is stored in the memory device. This has the significant effect of enabling the third generation model to generate related information with high accuracy.

[0195]

[19] An information processing system comprising: a receiving unit that receives input information entered by a user; a response generating unit that generates response information to the request information in accordance with one or more first output pieces generated by a first generative model that has been trained for a first prompt containing request information corresponding to the input information, and one or more second output pieces generated by a second generative model that has been trained for a second prompt containing request information; and an acquisition unit that acquires the response information generated in response to the request information corresponding to the input information.

[0196] In the above configuration, response information can be obtained in response to a first prompt containing request information corresponding to the input information entered by the user, one or more first output pieces generated by a trained first generative model in response to the first prompt containing the request information, and one or more second output pieces generated by a trained second generative model in response to the second prompt containing the same request information. Therefore, compared to a configuration in which response information is obtained in response to output information generated by a single generative model in response to a prompt containing request information, for example, it is possible to present the user with more appropriate response information in response to the request information from the user.

[0197]

[20] An information acquisition method implemented by a computer system, which receives input information entered by a user, acquires response information generated in response to request information corresponding to the input information, wherein the response information is information generated in response to one or more first output pieces generated by a first generative model trained for a first prompt including the request information, and one or more second output pieces generated by a second generative model trained for a second prompt including the request information.

[0198] In the above configuration, response information can be obtained in response to a first prompt containing request information corresponding to the input information entered by the user, one or more first output pieces generated by a trained first generative model in response to the first prompt containing the request information, and one or more second output pieces generated by a trained second generative model in response to the second prompt containing the same request information. Therefore, compared to a configuration in which response information is obtained in response to output information generated by a single generative model in response to a prompt containing request information, for example, it is possible to present the user with more appropriate response information in response to the request information from the user.

[0199]

[21] A program that causes a computer system to function as a receiving unit that receives input information entered by a user, and an acquiring unit that acquires response information generated in response to request information corresponding to the input information, wherein the response information is information generated in response to one or more first output pieces of information generated by a first generative model trained for a first prompt including the request information, and one or more second output pieces of information generated by a second generative model trained for a second prompt including the request information.

[0200] In the above configuration, response information can be obtained in response to a first prompt containing request information corresponding to the input information entered by the user, one or more first output pieces generated by a trained first generative model in response to the first prompt containing the request information, and one or more second output pieces generated by a trained second generative model in response to the second prompt containing the same request information. Therefore, compared to a configuration in which response information is obtained in response to output information generated by a single generative model in response to a prompt containing request information, for example, it is possible to present the user with more appropriate response information in response to the request information from the user.

[0201] Furthermore, the invention is also specified as an invention of a method to be performed in the information providing device

[12] , the information providing device

[13] , the information generating system

[14] , and the information collecting system

[15] and

[17] , respectively. It is also specified as an invention of a program to make the information providing device

[12] , the information generating system

[13] , the information collecting system

[15] and

[17] , and the information processing system

[19] , respectively function. [Explanation of Symbols]

[0202] 10: Terminal device 11: Control device 12:Storage device 13: Communication device 14:Display device 15: Operating device 20: Information Management System 30: Information generation system 31: Control device 32: Storage device 33: Communication device 40: Information Provision System 41: First Processing Unit 42: Second processing unit 43: Information provision device 44: Information acquisition device 50: Information gathering system 51: Control device 52: Storage device 53: Communication device 60:Storage device 100: Information Processing Systems 111: Reception Department 112: Prompt generation unit 113: Acquisition Department 114: Presentation part 141: Acquisition Department 142: Processing Unit 241: Acquisition Department 242: Processing Unit 311: Acquisition Department 312: Related Information Generation Unit 341: Acquisition Department 342: Indicator calculation department 343: Response generation unit 411: Control device 412 :Storage device 413: Communication equipment 421: Control device 422 :Storage device 423: Communication device 431: Control device 432 :Storage device 433: Communication device 511: Acquisition Department 512: Collection Department 513: Prompt generation unit 514: Information generation section 900: Communication Network F: Feedback evaluation G: Related Information K: indicator M1-M6: Generative Models P1-P6: Prompt W1, W2: Output information

Claims

1. The system comprises a response generation unit that generates response information to the request information in accordance with one or more first output pieces generated by a trained first generative model in response to a first prompt containing request information corresponding to input information entered by the user, and one or more second output pieces generated by a trained second generative model in response to a second prompt containing the request information. The one or more first output information and the one or more second output information are candidates for the response information. Information provision device.

2. The computer system functions as a response generation unit that generates response information to the request information in response to a first prompt containing request information corresponding to input information entered by the user, one or more first output pieces generated by a trained first generative model in response to a first prompt containing request information corresponding to the input information entered by the user, and one or more second output pieces generated by a trained second generative model in response to a second prompt containing the request information. The one or more first output information and the one or more second output information are candidates for the response information. program.

3. Based on one or more first output pieces generated by a trained first generative model in response to a first prompt containing request information corresponding to input information entered by the user, and one or more second output pieces generated by a trained second generative model in response to a second prompt containing the request information, response information to the request information is generated. The one or more first output information and the one or more second output information are candidates for the response information. A method of providing information using a computer system.

Images