Response generation support program, response generation support method, and information processing device.

Abstract

This suppresses responses that include knowledge the target of imitation does not possess. [Solution] Based on target information 3 relating to the object to be imitated, the information processing device 10 determines a first knowledge domain in which the object to be imitated possesses knowledge, and a first knowledge level of the object to be imitated with respect to the first knowledge domain. When a first question 4 is input to the information processing device 10, it instructs the agent 12b, which responds to the question using the language model 1, to respond to the first question 4. In instructing the response, the information processing device 10 instructs the agent to respond by imitating the thinking of the object to be imitated, which is able to refer to information restricted based on the first knowledge domain and the first knowledge level.

Description

Technical Field

[0001] The present invention relates to a response generation support program, a response generation support method, and an information processing device.

Background Art

[0002] By using a large language model (LLM), a computer can automatically generate a highly accurate response equivalent to a human response. By using such an LLM, it is possible to replace the interlocutor in dialogues such as dialogue training that has been conventionally conducted face-to-face with a generation artificial intelligence (AI) system that mimics a specific persona (virtual human image). As a method of making the generation AI system mimic a persona, for example, there is a technique of fine-tuning the LLM according to the persona. Fine-tuning is a technique for improving performance by adapting a pre-trained model to a specific task.

[0003] As a technique using an LLM, for example, a technique for improving a generation model specialized for a task by analyzing search results has been proposed. An adaptive language model-based technique for solving interactive security and safety with data privacy has also been proposed. A communication system that can appropriately represent the personality and character of an individual user has also been proposed. A response generation method has also been proposed that can generate a response mimicking a target virtual character through only some utterances of the target virtual character in a dialogue model. Furthermore, a method for adjusting a target neural network using automatically generated test cases before deploying the target neural network has also been proposed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

[0005] Traditionally, systems that mimic personas may include knowledge that the persona does not possess in their responses. For example, if the LLM or the referenced database (DB) contains information that the persona does not have, that information may not be properly excluded and may be included in the response. The inclusion of such persona-independent knowledge in the response makes it unnatural for a response that mimics the persona.

[0006] In one respect, this aims to suppress the output of responses that contain knowledge that the target of imitation does not possess. [Means for solving the problem]

[0007] One proposal provides a response generation support program that instructs a computer to perform the following processes: Based on information about the object to be imitated, the computer determines a first knowledge domain in which the object possesses knowledge, and a first knowledge level of the object with respect to that first knowledge domain. When a first question is input, the computer instructs an agent, which responds to the question using a language model, to respond to the first question by imitating the thinking of the object to be imitated, which is limited by the information it can refer to based on the first knowledge domain and the first knowledge level. [Effects of the Invention]

[0008] According to one embodiment, the output of a response containing knowledge that the object of imitation does not possess is suppressed. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of a response generation support method according to the first embodiment. [Figure 2] This figure shows an example of a system configuration. [Figure 3] This figure shows an example of the hardware for an AI system. [Figure 4] This block shows the functions of the AI ​​system. [Figure 5] This figure shows an example of the information detail level setting process. [Figure 6] This is a sequence diagram illustrating an example of the procedure for estimating knowledge domains and knowledge levels. [Figure 7] This figure shows an example of the information used to estimate knowledge domains and knowledge levels. [Figure 8] This figure shows an example of persona information. [Figure 9] This figure shows an example of knowledge level definition information. [Figure 10] This figure shows an example of knowledge level format information. [Figure 11] This figure shows an example of prompts for estimating knowledge information and knowledge level. [Figure 12] This figure shows an example of a knowledge level list. [Figure 13] This is a sequence diagram illustrating an example of a response processing procedure in a dialogue format that mimics a persona. [Figure 14] This figure shows an example of the information used to generate a response to a question. [Figure 15] This figure shows an example of the process for estimating a persona's knowledge level in knowledge domains useful for answering questions. [Figure 16] This figure shows an example of a prompt generated by CoT prompting. [Figure 17]The figure shows an example of a response given by imitating a persona who has no interest in home appliances to questions about home appliances. [Figure 18] The figure shows an example of a response given by imitating a persona who is interested in home appliances to questions about home appliances. [Figure 19] The figure shows an example of a response given by imitating a persona who has no interest in programming languages to questions about programming languages. [Figure 20] The figure shows an example of a response given by imitating a persona who is interested in programming languages to questions about programming languages. [Figure 21] The figure shows an example of a response given by imitating a persona who has no knowledge of diseases to questions about diseases. [Figure 22] The figure shows an example of a response given by imitating a persona who has knowledge of diseases to questions about diseases.

Mode for Carrying Out the Invention

[0010] Hereinafter, this embodiment will be described with reference to the drawings. Note that multiple embodiments can be combined and implemented within a non - contradictory range. 〔First Embodiment〕 The first embodiment is a response generation support method for suppressing the generation of responses that include knowledge that the imitated target (hereinafter, may also be referred to as a "person") cannot have when imitating the thinking of a virtually set imitated target and responding to questions.

[0011] FIG. 1 is a diagram showing an example of the response generation support method according to the first embodiment. In FIG. 1, an information processing apparatus 10 for implementing the response generation support method is shown. The information processing apparatus 10 can implement the response generation support method by, for example, executing a response generation support program.

[0012] The information processing device 10 includes a storage unit 11 and a processing unit 12. The storage unit 11 is, for example, a memory or storage device of the information processing device 10. The processing unit 12 is, for example, a processor of the information processing device 10. The information processing device 10 may have multiple processors. Some of the multiple processes performed by the information processing device 10 may be executed on different processors.

[0013] The memory unit 11 stores the language model 1 and the reference information 2. The language model 1 is a trained machine learning model that performs response or data processing in response to the input of a question or request. The language model 1 is, for example, an LLM based on a neural network. The reference information 2 is information that can be used as a source of knowledge when generating a response using the language model 1. In the reference information 2, for example, knowledge about each knowledge domain is set up as reference unit information for each knowledge domain. The reference unit information is set up with an information detail level that indicates how detailed the reference unit information is.

[0014] The processing unit 12 includes a response generation support unit 12a and an agent 12b. The response generation support unit 12a generates prompts 6 and 8 such that an appropriate response 5 can be obtained in response to the first question 4, and inputs these prompts 6 and 8 to the agent 12b. The agent 12b responds to the question using the language model 1. The response generation support unit 12a can cause the agent 12b to produce a response that mimics the thinking of the target of imitation.

[0015] For example, the response generation support unit 12a determines a first knowledge domain in which the target of imitation possesses knowledge, and a first knowledge level of the target of imitation regarding the first knowledge domain, based on the target information 3 concerning the target of imitation. The target information 3 represents, for example, the persona of the person being imitated. The target information 3 can also be called persona information. For example, the response generation support unit 12a instructs agent 12b via prompt 6 to estimate the first knowledge domain and the first knowledge level based on the target information 3. Upon receiving prompt 6, agent 12b uses language model 1 to estimate the first knowledge domain and the first knowledge level. Agent 12b then transmits a knowledge level list 7, showing the estimated first knowledge domain and first knowledge level, to the response generation support unit 12a.

[0016] Subsequently, when the first question 4 is input to the information processing device 10, the response generation support unit 12a instructs agent 12b to generate a response 5 to the first question 4 by mimicking the thinking of an imitation target that can refer to information restricted based on the first knowledge domain and the first knowledge level. For example, the response generation support unit 12a inputs a prompt 8 to agent 12b that indicates the response generation procedure. For example, prompt 8 includes target information 3, the first question 4, and a knowledge level list 7. Agent 12b, having received prompt 8, generates a response 5 according to the instructions shown in prompt 8.

[0017] In this way, agent 12b can be made to imitate the thinking of the object to be imitated, which is limited based on the knowledge domain possessed by the object and the level of knowledge it has in that knowledge domain. As a result, information that the object to be imitated does not know is not referenced, and the generation of responses containing such information is suppressed.

[0018] Prompt 8 includes instructions for estimating a second knowledge domain useful for responding to the first question 4, for example, based on the first question 4. Prompt 8 also includes instructions for estimating the second knowledge level of the imitation target with respect to the second knowledge domain, based on the first knowledge domain and the first knowledge level. And Prompt 8 includes instructions for responding to the first question 4 while imitating the imitation target.

[0019] Such a prompt 8 allows for the accurate estimation of a second knowledge level that represents the subject's knowledge of a second knowledge domain useful for responding to the first question 4, based on the information shown in the knowledge level list 7. This ensures that the information available for reference when generating response 5 is properly determined, and prevents the inclusion of information unavailable to the subject in response 5.

[0020] The response generation support unit 12a instructs, for example, in prompt 8, to estimate the second knowledge level based on the first knowledge level of the first knowledge domain which is similar to the second knowledge domain. As a result, the second knowledge level is estimated without being affected by the knowledge level of the imitation target regarding the first knowledge domain which is dissimilar to the second knowledge domain. Consequently, the accuracy of the second knowledge level estimation is improved, the information that can be referenced when generating response 5 is appropriately determined, and the inclusion of information that cannot be used by the imitation target in response 5 is suppressed.

[0021] Furthermore, the response generation support unit 12a can be instructed at prompt 8 to estimate a first level of information detail, which indicates how detailed the information available to the imitation target is regarding the second knowledge domain, based on the second knowledge level. Information available to the imitation target is information that the imitation target can understand to the extent that it can explain the content of that information to a third party when it encounters it. In this case, the response generation support unit 12a instructs to generate a response to the first question 4 by limiting the information that the imitation target can refer to to the information of the first level of information detail among the information related to the second knowledge domain.

[0022] This prevents the response from including information from the second knowledge domain that the object of imitation cannot understand or use. The processing unit 12 can also have agent 12b set the information detail level for each reference information unit in reference information 2. For example, the processing unit 12 can instruct agent 12b to estimate the information detail level for each reference unit information in reference information 2, indicating how detailed the information is. This makes it easier to set the information detail level and increases the amount of information in the reference information for which an information detail level has been set. As a result, the amount of information within the limited scope increases, improving the accuracy of responses to questions.

[0023] [Second Embodiment] The second embodiment is an AI system that restricts the information that can be referenced according to persona information and can suppress the generation of responses that include information that deviates from the persona's knowledge.

[0024] Figure 2 shows an example of a system configuration. The AI ​​system 100 is a computer system composed of one or more computers. For example, the AI ​​system 100 is a cloud computing system. The AI ​​system 100 responds to questions by mimicking a designated persona.

[0025] A terminal device 30 is connected to the AI ​​system 100 via a network 20. The terminal device 30 is a computer used by a user to interact with the AI ​​system 100, which is mimicking a virtual person identified by a persona.

[0026] Figure 3 shows an example of the hardware of an AI system. The entire AI system 100 is controlled by a processor 101. The processor 101 is connected to a memory 102 and several peripheral devices via a bus 109.

[0027] The AI ​​system 100 may be a multiprocessor system having multiple processors. A collection of multiple processors in a multiprocessor system can be called a processor 101. A processor 101 may also be called a processor circuitry. Each of the multiple processors can execute some or all of the processes that are executed in the AI ​​system 100. When there are multiple related processes, two or more of those processes may be executed by different processors.

[0028] The processor 101 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor). At least some of the functions that the processor 101 implements by executing a program may be implemented by electronic circuits such as an ASIC (Application Specific Integrated Circuit) or a PLD (Programmable Logic Device).

[0029] Memory 102 is used as the main memory of the AI ​​system 100. Memory 102 temporarily stores at least a portion of the OS (Operating System) program and application programs that are to be executed by the processor 101. Memory 102 also stores various data used for processing by the processor 101. As memory 102, a volatile semiconductor memory device such as RAM (Random Access Memory) is used.

[0030] Peripheral devices connected to bus 109 include a storage device 103, a graphics controller 104, an input interface 105, an optical drive device 106, a device connection interface 107, and a network interface 108.

[0031] The storage device 103 electrically or magnetically writes and reads data from its built-in recording medium. The storage device 103 is used as an auxiliary storage device for the AI ​​system 100. The storage device 103 stores the OS program, application programs, and various data. For example, the storage device 103 can be an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0032] The graphics controller 104 is an arithmetic unit that performs image processing. The graphics controller 104 is, for example, a GPU (Graphics Processing Unit). A monitor 21 is connected to the graphics controller 104. The graphics controller 104 displays images on the screen of the monitor 21 according to instructions from the processor 101. The monitor 21 can be an OLED (Electroluminescence) display device or a liquid crystal display device. If a GPU is used as the graphics controller 104, the graphics controller 104 can also perform complex numerical calculations such as matrix calculations.

[0033] The input interface 105 is connected to a keyboard 22 and a mouse 23. The input interface 105 transmits signals from the keyboard 22 and mouse 23 to the processor 101. Note that the mouse 23 is just one example of a pointing device; other pointing devices can also be used. Other pointing devices include touch panels, tablets, touchpads, and trackballs.

[0034] The optical drive device 106 uses laser light or the like to read data recorded on the optical disc 24 or write data to the optical disc 24. The optical disc 24 is a portable recording medium on which data is recorded in a way that makes it readable by the reflection of light. Examples of optical discs 24 include DVD (Digital Versatile Disc), DVD-RAM, CD-ROM (Compact Disc Read Only Memory), and CD-R (Recordable) / RW (ReWritable).

[0035] The device connection interface 107 is a communication interface for connecting peripheral devices to the AI ​​system 100. For example, a memory device 25 and a memory reader / writer 26 can be connected to the device connection interface 107. The memory device 25 is a recording medium equipped with a communication function with the device connection interface 107. The memory reader / writer 26 is a device that writes data to or reads data from the memory card 27. The memory card 27 is a card-type recording medium.

[0036] The network interface 108 is connected to the network 20. The network interface 108 transmits and receives data to and from other computers or communication devices via the network 20. The network interface 108 is a wired communication interface, for example, connected by cable to a wired communication device such as a switch or router. Alternatively, the network interface 108 may be a wireless communication interface, connected by radio waves to a wireless communication device such as a base station or access point.

[0037] The AI ​​system 100 can realize the processing functions of the second embodiment using the hardware described above. The information processing device 10 shown in the first embodiment can also be realized using the same hardware as the AI ​​system 100 shown in Figure 3.

[0038] The AI ​​system 100 realizes the processing functions of the second embodiment by executing a program recorded on a computer-readable recording medium, for example. The program describing the processing content to be executed by the AI ​​system 100 can be recorded on various recording media. For example, the program to be executed by the AI ​​system 100 can be stored in the storage device 103. The processor 101 loads at least a portion of the program in the storage device 103 into the memory 102 and executes the program. Alternatively, the program to be executed by the AI ​​system 100 can be recorded on a portable recording medium such as an optical disc 24, a memory device 25, or a memory card 27. The program stored on the portable recording medium becomes executable after being installed in the storage device 103, for example, under control from the processor 101. The processor 101 can also directly read and execute the program from the portable recording medium.

[0039] In such a system, the user uses terminal device 30 to send persona information representing the target person to AI system 100 and requests an interactive simulation based on the persona information. The AI ​​system 100 then responds to the questions sent from terminal device 30 by mimicking the persona.

[0040] The AI ​​system 100 has a database (DB) that stores information that forms the basis of its responses. The DB may be organized by subject area, for example. When responding to a question, the AI ​​system 100 refers to this DB to generate the response. Referring to the DB helps to suppress hallucination (errors in responses). On the other hand, the DB may contain information that the persona is unlikely to know. Therefore, a technology is needed to exclude information that the persona could not possibly know from the response.

[0041] For example, a technology called Character-LLM can generate LLMs that mimic historical figures. In the case of historical figures, information about them after their death is information they could not have known. Therefore, fine-tuning is performed on the LLM to erase this unknown information.

[0042] However, it is difficult to prepare finely tuned LLMs for all the unspecified personas that users wish to train in dialogue. Furthermore, in the intended application, accurately imitating the target using a finely tuned LLM requires restricting reference information not only on the time axis but also on finer axes (such as expertise and interests). It is difficult to generate comprehensively trained LLMs for all conceivable personas.

[0043] For example, let's consider a scenario where sales training is conducted using AI system 100. The purpose of the sales training is to enable sales representatives (users) to improve their sales skills through question-and-answer sessions with hypothetical customers (agents provided with persona information).

[0044] The AI ​​system 100, which mimics a customer, is given persona information such as "not knowledgeable about home appliances." In this case, it is important that the AI ​​system 100 does not start talking about detailed information regarding air conditioners.

[0045] For example, suppose the question "What do you know about the Gokudan Air Conditioner?" is entered into AI system 100, which is mimicking a persona that "is not knowledgeable about home appliances." If the database contains technical information about home appliances and the specifications for the Gokudan Air Conditioner, AI system 100 may answer using specialized terminology about air conditioners and providing information that could only be obtained by thoroughly examining the specifications. In this case, the behavior of AI system 100 would be inappropriate for mimicking a persona that "is not knowledgeable about home appliances."

[0046] The AI ​​system 100 then estimates the availability of databases for each knowledge domain and the scope of access to the available databases, based on inputs such as persona information. The AI ​​system 100 then generates a response to the question based on the information that can be accessed within the available databases.

[0047] For example, the reason why responses containing information that deviates from the persona may be generated is likely because there are no criteria for determining whether the information the persona deals with requires expertise (usability of the information from a difficulty perspective) or how much they know about it (scope of information reference). Therefore, AI system 100 performs Chain-of-Thought (CoT) prompting, which is equivalent to multi-stage filtering. That is, AI system 100 estimates which domain of knowledge is required to answer the question and to what extent the persona is likely to be able to handle information in that domain, from two perspectives: "knowledge domain" and "knowledge level". Then, based on the information that can be referenced through estimation, AI system 100 generates a response that mimics the persona.

[0048] Furthermore, AI system 100 introduces a separate "information detail level" to determine how much reference information it can know, and uses this level to limit the information it can access. In the example of an air conditioner brochure, AI system 100 controls how well it can understand air conditioner-related terminology using "knowledge domains" and "knowledge levels." Moreover, AI system 100 controls whether it knows the information contained in the brochure using the "information detail level." This enables diverse knowledge restrictions based on persona information.

[0049] In summary, AI system 100 performs "knowledge domain estimation," "knowledge level estimation," and "information detail level estimation" to extract usable information from LLM's internal knowledge and database. These estimation processes are used for the following purposes:

[0050] Knowledge domain estimation is the process of estimating the knowledge domains (e.g., botany, automotive engineering, etc.) necessary to understand the input persona information and queries (questions). The results of the knowledge domain estimation are used, for example, to limit the terminology that AI system 100 can understand and use when mimicking the persona.

[0051] Knowledge level estimation is the process of estimating an indicator (knowledge level) that shows how deep a persona's knowledge is in the previously estimated knowledge domain. The persona's knowledge level for each knowledge domain is used, for example, to limit the available knowledge within the LLM's internal knowledge when generating responses using LLM, or to limit the reference information in the database.

[0052] Estimating the information detail level is the process of estimating an index (information detail level) that represents how detailed the information within a database is that can be understood. The accessible information detail level is used, for example, to limit the level of detail of information that can be accessed when generating a response.

[0053] In this way, the AI ​​system 100 performs "knowledge domain estimation," "knowledge level estimation," and "information detail level estimation," and then uses the estimation results to restrict the information available for response generation from the information in the DB and the LLM internal information. This suppresses the generation of responses that deviate from the persona's capabilities.

[0054] Figure 4 is a block diagram showing the functions of the AI ​​system. The AI ​​system 100 includes a memory unit 110, an information detail level setting unit 120, a knowledge level management unit 130, and a dialogue processing unit 140.

[0055] The memory unit 110 stores multiple DBs 111a, 111b, ... and LLM 112 for each knowledge domain. DBs 111a, 111b, ... are referenced when generating responses to questions. LLM 112 is a pre-trained model that has been trained using general training data, not limited to a specific persona.

[0056] The information detail level setting unit 120 sets the information detail level for each predetermined reference unit (e.g., a statement) contained in DB 111a, 111b, ... For example, the information detail level setting unit 120 causes agent 150 to perform an information detail level estimation process using LLM 112. The information detail level is an indicator of how specific or specialized the information shown in that reference unit is. For example, the information detail level for information that shows an overview of a certain matter is set low. Conversely, the information detail level for information that shows detailed content using specialized knowledge about a certain matter is set high.

[0057] The knowledge level management unit 130 determines the persona's knowledge level for each knowledge domain based on persona information that identifies a specific persona. For example, the knowledge level management unit 130 instructs agent 150 to estimate the knowledge level using LLM 112 based on the persona information. The knowledge level management unit 130 transmits the knowledge level list, which shows the persona's knowledge level determined for each knowledge domain, along with the persona information, to the dialogue processing unit 140.

[0058] When the dialogue processing unit 140 receives a query from the terminal device 30 indicating a question from the user, it generates a prompt that shows the procedure for generating a persona-like response using the LLM 112. The dialogue processing unit 140 sends the generated prompt to the agent 150. When the dialogue processing unit 140 receives a response from the agent 150, it sends that response to the terminal device 30.

[0059] Agent 150 performs various inference processes using LLM 112. For example, Agent 150 estimates the information detail level for each reference unit in DB 111a, 111b, ... according to instructions from the information detail level setting unit 120. Agent 150 also estimates the knowledge domains in which the persona possesses knowledge and the persona's knowledge level for those knowledge domains, according to instructions from the knowledge level management unit 130. Furthermore, when Agent 150 receives a prompt corresponding to a question from the dialogue processing unit 140, it interprets the referenced information in DB 111a, 111b, ... according to the prompt, mimicking the persona, and generates a response to the question.

[0060] The functions of each element shown in Figure 4 can be realized, for example, by having the processor 101 execute the program module corresponding to that element. With the AI ​​system 100 possessing such functions, dialogue that appropriately mimics a persona is realized using the general-purpose LLM 112. Furthermore, the information in DB 111a, 111b, ... which is referenced when generating responses to questions, has its information detail level pre-set according to instructions from the information detail level setting unit 120.

[0061] Figure 5 shows an example of the information detail level setting process. The information detail levels for DB111a, 111b, ... are set, for example, when data is registered. In this case, when DB registration data 41 is input to the AI ​​system 100, the information detail level setting unit 120 instructs the agent 150 to set the information detail level for each row in the DB registration data 41. At that time, the information detail level setting unit 120 sends information detail level definition information 121 to the agent 150.

[0062] The information detail level definition information 121 shows specific examples of information corresponding to each information detail level. In the example in Figure 5, the information detail level definition information 121 has four levels of information detail set.

[0063] "Information Detail Level 1 (dl Lv.1)" is a summary level. Specific examples of summary level content include the cover of a brochure, the headline of a flyer, and the title slide of a PowerPoint presentation.

[0064] "Information Detail Level 2 (dl Lv.2)" is the basic information level. Specific examples of basic information levels include section headings in brochures, main bullet points in flyers, and bulleted slides in PowerPoint presentations.

[0065] "Information Detail Level 3 (dl Lv.3)" is a level of detailed information. Examples of detailed information levels include book chapters, documentary text, and detailed PowerPoint slides. "Information Detail Level 4 (dl Lv.4)" is a level of specialized information. Specific examples of specialized information include data sections in academic papers, appendices in books, and supplementary materials in PowerPoint presentations.

[0066] Agent 150 uses the information detail level definition information 121 as a criterion to determine the information detail level of each row in the DB registration data 41 by LLM 112. For example, Agent 150 extracts rows one by one from the DB registration data 41, and uses the information detail level determination instruction including the extracted rows as input to LLM 112 to calculate the output of LLM 112. Agent 150 assigns a tag indicating the information detail level to each row in the DB registration data 41 and registers it in DB 111a, which is the destination for the DB registration data 41.

[0067] When Agent 150 responds by mimicking the persona, the level of detail of the information that can be referenced in DB 111a, 111b, ... is determined according to the persona's knowledge level for each knowledge domain. For example, for knowledge domains where the persona has a high knowledge level, up to the rows with the highest level of detail in the corresponding DB can be referenced. Conversely, for knowledge domains where the persona has a low knowledge level, only rows with the lowest level of detail in the corresponding DB can be referenced.

[0068] In this way, the combination of the persona's knowledge level and information detail level restricts the information that can be referenced. By restricting information referencing according to the information detail level, responses are generated that are limited to information that the persona can grasp according to their interests and concerns from external materials (information within DB111a, 111b, ...). In other words, if there is a real person corresponding to the persona, even if that person refers to DB111a, 111b, ..., they will not understand information that they are not interested in and do not have basic knowledge of. When that person responds to questions from others, even if they are able to respond by referring to many materials, they will not respond using information that they do not understand. By using information detail levels, when agent 150, which mimics the persona, responds to questions, it is suppressed from responding based on information that the persona does not understand.

[0069] Furthermore, the determination of the information detail level for each row of DB111a, 111b, ... can be performed by agent 150 when responding to a question, rather than being done in advance. Dynamic determination of the information detail level when responding to a question can be achieved, for example, using LLM112.

[0070] After setting the level of detail for DB111a, 111b, ..., a user who wants the AI ​​system 100 to imitate a persona uses terminal device 30 to send persona information indicating the persona to be imitated to the AI ​​system 100. Upon receiving the persona information, the AI ​​system 100, in cooperation with the knowledge level management unit 130 and agent 150, estimates the persona's knowledge domain and the persona's knowledge level for each knowledge domain.

[0071] Figure 6 is a sequence diagram showing an example of the procedure for estimating knowledge domains and knowledge levels. The terminal device 30 generates persona information based on user input and transmits the generated persona information to the AI ​​system 100 (step S101).

[0072] Upon receiving the persona information, the AI ​​system 100 sends an instruction to the agent 150 to estimate the persona's knowledge domain and knowledge level (step S102). The instruction to estimate the knowledge domain and knowledge level includes the persona information.

[0073] Agent 150 uses LLM 112 to estimate the persona's knowledge domain and knowledge level within that domain based on the persona information (step S103). Agent 150 transmits a knowledge level list showing the estimated knowledge domain and knowledge level to the knowledge level management unit 130 (step S104).

[0074] The knowledge level management unit 130 transmits the persona information and the knowledge level list to the dialogue processing unit 140 (step S105). The dialogue processing unit 140 stores the persona information and the knowledge level list in the memory 102 (step S106).

[0075] In this way, the knowledge domain possessed by the persona being imitated, and the level of knowledge within that domain, can be estimated. Figure 7 shows an example of information used to estimate knowledge domains and knowledge levels. The knowledge level management unit 130 generates a prompt 45 that instructs the estimation of knowledge domains and knowledge levels based on persona information 42, knowledge level definition information 43, and knowledge level format information 44. Persona information 42 is information that indicates the persona to be imitated. Knowledge level definition information 43 is information that indicates the criteria for determining knowledge levels. Knowledge level format information 44 is information that indicates the data format of the knowledge level list 46.

[0076] The knowledge level management unit 130 sends the generated prompt 45 to the agent 150. The agent 150 uses the LLM 112 to generate a knowledge level list 46 corresponding to the prompt 45. The agent 150 sends the generated knowledge level list 46 to the knowledge level management unit 130.

[0077] Figure 8 shows an example of persona information. Persona information includes, for example, a list of attributes of the person being imitated, and information describing the characteristics of the person being imitated in natural language. Attributes of the person being imitated include name, gender, age, occupation, and address. Information describing the characteristics of the person being imitated includes personality, hobbies, and views and feelings about privacy. In the example in Figure 8, interest in home appliances is included as information describing the characteristics of the person being imitated.

[0078] Figure 9 shows an example of knowledge level definition information. Knowledge level definition information 43 includes instructions to agent 150: "Higher levels indicate more advanced knowledge, but intermediate levels such as Lv.24 can be applied, so allow for some variation." Knowledge level definition information 43 also shows the difficulty level of knowledge that can be mentioned for each knowledge level in the knowledge domain being estimated.

[0079] "Knowledge Level 1 (Lv.1: Ignorance)" is the knowledge level when you have no knowledge of the subject matter. "Knowledge Level 10 (Lv.10: Beginner 1)" is the knowledge level when you have the knowledge acquired by graduating from elementary school. "Knowledge Level 20 (Lv.20: Beginner 2)" is the knowledge level when you have the knowledge acquired by graduating from junior high school. "Knowledge Level 30 (Lv.30: Novice)" is the knowledge level when you have the knowledge acquired by graduating from high school. "Knowledge Level 40 (Lv.40: Intermediate)" is the knowledge level when you have the knowledge acquired by graduating from university. "Knowledge Level 50 (Lv.50: Advanced)" is the knowledge level when you have the knowledge acquired through graduate school research or professional work experience. Knowledge Level 60 (Lv.60: Expert) represents the knowledge level of an expert who holds a doctoral degree and is engaged in research or practice in that field. Knowledge Level 70 (Lv.70: Authority) represents the knowledge level of a prominent researcher in that field.

[0080] By defining knowledge levels in this way, the difficulty level of knowledge that can be mentioned at each knowledge level becomes clear, and the expertise of the terminology used in responses can be appropriately controlled. In other words, even if a simple prompting specifies that an agent is knowledgeable about "AA" but lacks knowledge of "BB," the criteria are ambiguous, making it difficult to restrict specialized terminology as intended according to the persona. Failure to properly restrict specialized terminology can lead to hallucination. By clarifying the difficulty level of knowledge at each knowledge level through the knowledge level definition information 43, the specialized terminology that agent 150 can use when imitating the persona becomes accurate, and the occurrence of hallucination is suppressed.

[0081] Figure 10 shows an example of knowledge level format information. Knowledge level format information 44 shows the format for listing knowledge domains and the knowledge levels for those domains in a knowledge level list. For example, knowledge level format information 44 shows that the knowledge domain name is listed, and the knowledge level and its description are listed on the line below the knowledge domain name.

[0082] Based on the information shown in Figures 8 to 10, a prompt 45 is generated that instructs the system to estimate the persona's knowledge information and the persona's knowledge level regarding that information. Figure 11 shows an example of prompts for estimating knowledge information and knowledge levels. Prompt 45 shows, for example, the instruction to agent 150: "Generate the knowledge domains and knowledge levels included in the persona information below in the specified format." The line following "#Persona Information" in prompt 45 shows the persona information 42 shown in Figure 8, followed by "$". The line following "#Knowledge Level" in prompt 45 shows the knowledge level definition information 43 shown in Figure 9, followed by "$". The line following "#Format" in prompt 45 shows the knowledge level format information 44 shown in Figure 10, followed by "$".

[0083] Upon receiving prompt 45, agent 150 performs inference using LLM112. As a result of the inference, knowledge level list 46 is generated. Figure 12 shows an example of a knowledge level list. The knowledge level list 46 describes the knowledge level for each knowledge domain in the data format shown in the knowledge level format information 44.

[0084] The generated knowledge level list 46 is held in the dialogue processing unit 140. When the dialogue processing unit 140 obtains the knowledge level list 46, it starts accepting queries from the terminal device 30. When a query is input to the AI ​​system 100 from the terminal device 30, a response is processed in a dialogue format that mimics the persona.

[0085] Figure 13 is a sequence diagram showing an example of the procedure for response processing in a dialogue format that mimics a persona. When the terminal device 30 receives a string indicating a question from the user, it sends a query containing that question to the AI ​​system 100 (step S201). In the AI ​​system 100, the dialogue processing unit 140 receives the query. The dialogue processing unit 140 retrieves persona information from memory 102 (step S202). The dialogue processing unit 140 also retrieves a knowledge level list from memory 102 (step S203). Then, the dialogue processing unit 140 generates a prompt containing the query, persona information, and knowledge level list using CoT prompting, and sends the generated prompt to the agent 150 (step S204).

[0086] Agent 150 generates responses to questions by mimicking the persona, following the steps indicated by the prompt. Specifically, Agent 150 first uses LLM112 to estimate the knowledge domains useful for responding to the questions presented in the query (step S205). Next, Agent 150 uses LLM112 to estimate the persona's knowledge level for the knowledge domains estimated in step S205 (step S206). Next, Agent 150 uses LLM112 to estimate the level of detail at which information can be referenced in the database of the estimated knowledge domains (step S207). Next, Agent 150 extracts the referenced information from the database of the estimated knowledge domains (step S208). Next, Agent 150 uses LLM112 to verify the information shown in the database of the estimated knowledge domains at the persona's knowledge level (step S209). In verifying the information, Agent 150 excludes information that cannot be handled at the persona's knowledge level (e.g., information that is too specialized).

[0087] Agent 150 then uses LLM 112 to generate a response appropriate to the persona's knowledge level, utilizing the information confirmed in step S209. Agent 150 sends the generated response to the dialogue processing unit 140 (step S210). The dialogue processing unit 140 sends the response received from Agent 150 to the terminal device 30 (step S211).

[0088] In this way, an appropriate response is made that refers only to the knowledge the persona possesses and consists only of information that the persona can understand. Figure 14 shows an example of information used to generate a response to a question. The dialogue processing unit 140 obtains a query 51 indicating a question from the terminal device 30. The dialogue processing unit 140 generates a prompt 52 that includes the question indicated in query 51, persona information 42, and knowledge level list 46, and sends the prompt 52 to the agent 150. The agent 150 generates a response 53 based on the DB 111a and LLM 112 of knowledge fields useful for answering the question, and sends the response 53 to the dialogue processing unit 140. The dialogue processing unit 140 sends a response 54 with the same content as the obtained response 53 to the terminal device 30.

[0089] The prompt 52 generated by the dialogue processing unit 140 indicates the processing steps S205 to S210 (see Figure 13), and the agent 150 generates the content of the response according to these steps. Of the processing steps S205 to S210, the knowledge level estimation process in step S206 is useful when the knowledge level of a knowledge domain useful for responding to the question is not indicated in the knowledge level list 46.

[0090] Figure 15 shows an example of the process for estimating a persona's knowledge level in knowledge domains useful for answering a question. For example, suppose that when the knowledge level of a persona in knowledge domains is estimated based on persona information 42, the knowledge level in psychology is "40", the knowledge level in economics is "30", and the knowledge level in electronics is "20". In this case, when a question about home appliances is entered as query 51, knowledge in the home appliance field is required as knowledge useful for answering that question.

[0091] The home appliance field is not included in the knowledge domains estimated based on the persona information 42. In this case, agent 150, following prompt 52, estimates the similarity between each of the persona's knowledge domains and the knowledge domains useful for the response. Then, agent 150 estimates the persona's knowledge level in the knowledge domains useful for the response based on the similarity between each of the persona's knowledge domains and the knowledge domains useful for the response.

[0092] In the example in Figure 15, knowledge in the home appliance field includes knowledge of home appliance prices and sales share (popularity), thus showing similarity to knowledge in economics. Furthermore, knowledge in the home appliance field includes knowledge of the technical characteristics of home appliances, thus showing similarity to knowledge in electronics. On the other hand, no similarity is found between knowledge in the home appliance field and knowledge in psychology. Therefore, the knowledge domains that are similar to the home appliance field, which are useful for responses, are economics and electronics. Based on the persona's economics knowledge level of "30" and electronics knowledge level of "20," for example, the persona's knowledge level in the home appliance field is estimated to be "25."

[0093] In this way, by estimating the persona's knowledge level in knowledge domains useful for responses, the influence of knowledge levels in unrelated knowledge domains on the estimation of the knowledge level in the knowledge domains useful for responses is suppressed.

[0094] The sequence of operations to be performed by agent 150 is represented by prompt 52, which is generated by CoT prompting. Figure 16 shows an example of a prompt generated by CoT prompting. Prompt 52 indicates the role that agent 150 will play: "Your role is: You are an actor playing the role of a customer of the given persona." Prompt 52 also specifies the task that agent 150 will perform: "Your task is: As a customer of the persona, generate a response according to the response generation procedure, response examples, and response rules. Providing a persona-like response is of utmost importance."

[0095] The line following "#Question" in prompt 52 shows the question string indicated in query 51, followed by "$". The line following "#Persona Information" in prompt 52 shows the content of persona information 42, followed by "$". The line following "#Persona's Knowledge Domain and Knowledge Level Information" in prompt 52 shows the content of knowledge level list 46, followed by "$". The line following "#Response Rule" in prompt 52 shows the rule to be followed when generating a response (response rule), followed by "$". The line following "#Knowledge Level Criteria" in prompt 52 shows knowledge level definition information 43, followed by "$".

[0096] The processing procedure for steps S205 to S210 shown in Figure 13 is shown starting from the line following "#Response generation procedure" in prompt 52. Specifically, the following is described: "Please generate a response to the entered statement by following the steps below. Strictly adhere to the level of detail for information that can be referenced, and do not mention any information that cannot be referenced." 1. Estimating Knowledge Domains: Estimate the necessary knowledge domains. If you lack specialized knowledge, use general knowledge. 2. Knowledge Level Estimation: Based on information about the persona's knowledge domains and knowledge levels, the knowledge level is estimated by considering the similarity with related knowledge domains. 3. Estimation of Information Detail Level: Based on the persona's interests, the information detail level is estimated on a scale from 1 to 4. 4. Information Extraction: Gather information at the permitted level of detail. Changing or exceeding the level is prohibited. 5. Creating the response: Create the final response, excluding any information the persona cannot handle. In prompt 52, the line following "#DB information" shows the database information to be referenced, followed by "$". Then, in prompt 52, the line following "#example response" shows an example response to the question, followed by "$".

[0097] By sending the prompt 52 described above to agent 150, agent 150 generates a response that correctly mimics a specific persona. In this process, agent 150 can generate the response using the general-purpose LLM112. Therefore, fine-tuning of LLM112 to suit the persona is unnecessary.

[0098] Next, we will refer to Figures 17 to 22 to explain specific examples of responses to questions. Figures 17 and 18 show examples of using AI system 100 in sales training simulations.

[0099] Figure 17 shows an example of a case where a persona with no interest in home appliances is used to respond to questions about home appliances. For example, persona information 42a, which clearly states that the persona has no particular interest in home appliances and is not knowledgeable about them, is sent from terminal device 30 to AI system 100. Subsequently, a query 51a, including the question, "Please tell me what you know about the Gokudan air conditioner," is sent from terminal device 30 to AI system 100.

[0100] In this case, the AI ​​system 100 sends a response 54a to the terminal device 30 that does not contain detailed information about the specified home appliance, "Gokudan Air Conditioner." In the example in Figure 17, the response 54a sent is: "I'm sorry, but I don't have detailed information about the 'Gokudan Air Conditioner.' You might find good information about the functions and features of the air conditioner on the manufacturer's official website or a reliable review site. Please let me know if there is anything else you would like to know."

[0101] Figure 18 shows an example of a response made by mimicking a persona interested in home appliances to a question about home appliances. For example, persona information 42b, which clearly indicates an interest in home appliances, is sent from terminal device 30 to AI system 100. Then, for example, suppose a query 51a similar to that in Figure 17 is sent from terminal device 30 to AI system 100. In this case, AI system 100 sends a response 54b to terminal device 30 that contains detailed information about the specified home appliance, "Gokudan Air Conditioner". In the example in Figure 18, the response 54b is sent which says, "You're asking about the Gokudan Air Conditioner. Its basic functions include the following features: - The indoor and outdoor units are compact and do not take up much space. - The outdoor unit can withstand extremely high and low temperatures and can be used in harsh environments. - It can be linked with smartphones and smart speakers and can be operated remotely. - It employs technologies such as a high-density multipath heat exchanger and an interleaved PAM inverter, and has excellent energy-saving performance."

[0102] Figures 19 and 20 show examples of how AI system 100 can be used in programming education simulations. Figure 19 shows an example of a case where a persona with no interest in programming languages ​​is used to respond to a question about programming languages. For example, persona information 42c, which clearly indicates that the person has no knowledge of programming languages, is sent from terminal device 30 to AI system 100. Subsequently, a query 51b, which includes the question, "Do you know how to use the print function in programming language A?", is sent from terminal device 30 to AI system 100.

[0103] In this case, the AI ​​system 100 sends a response 54c to the terminal device 30 that does not contain specific information about the specified "programming language A". In the example in Figure 19, the response 54c sent is: "I don't have any specific knowledge about the print function of programming language A, but I'm interested in programming and would like to learn it someday. I would appreciate it if you could tell me if there are any books or online courses that would be helpful."

[0104] Figure 20 shows an example of a response to a question about programming languages, mimicking a persona interested in programming languages. For example, persona information 42d, which clearly indicates that the AI ​​system 100 has already mastered a programming language, is sent from terminal device 30. For example, persona information 42d may include the description, "Programming language: Mastered the basics of programming language A and programming language B." Subsequently, for example, a query 51b similar to that in Figure 19 is sent from terminal device 30 to the AI ​​system 100.

[0105] In this case, the AI ​​system 100 sends a response 54d to the terminal device 30 that contains specific information about the specified "programming language A". In the example in Figure 20, the response 54d sent is: "Yes, I know the basic usage of the print function in programming language A. The print function is used to output a specified string or the value of a variable to the console. For example, if you write `print("Hello,World!")`, "Hello,World!" will be displayed on the console. This is often one of the first things you learn when you start learning programming language A."

[0106] Figures 21 and 22 show examples of using AI system 100 in medical consultation simulations. In medical consultation simulations, for example, a question-and-answer simulation is conducted after intentionally creating differences in the patients' perceptions of their symptoms (knowledge and experience regarding the disease). Note that the examples in Figures 21 and 22 are given common reference information related to the disease, and the only difference is in the persona information.

[0107] Figure 21 shows an example of a case where a persona with no knowledge of illness is used to respond to questions about illness. For example, persona information 42e, which clearly indicates that the persona has little knowledge about the illness they have contracted, is sent from terminal device 30 to AI system 100. For example, persona information 42e may include the statement, "I came to the hospital because of a headache, but I don't have any knowledge about illnesses as I haven't had any before." Then, a query 51c, which includes the question, "Hello. I am the one who will be seeing you today. You mentioned that you have been experiencing persistent headaches, but could you tell me if you have any idea what might be causing it?" is sent from terminal device 30 to AI system 100.

[0108] The AI ​​system 100 has a database 111c containing medical information. DB111c contains examples of diseases that cause headaches, for example. However, since the persona information 42e indicates a lack of knowledge about diseases, the AI ​​system 100 generates response 54e without referring to most of the information contained in DB111c.

[0109] For example, the AI ​​system 100 sends a response 54e to the terminal device 30 that only describes the symptoms and does not include any further specialized information. In the example in Figure 21, the response 54e sent is: "Hello. I've been having a headache for a while and it's bothering me a bit. I can't think of any particular reason, but I may have been under a lot of stress lately. I usually try to relax by meditating or taking walks, but that hasn't helped this time... I don't really know what the specifics are, so please help me." The response 54e explains that the user has no idea what might be causing the headache and lacks any specific knowledge about it.

[0110] Figure 22 shows an example of a case where a persona with knowledge of the disease is used to respond to questions about the disease. For example, persona information 42f, which clearly indicates that the persona has gathered information about the disease they have contracted and possesses basic knowledge, is sent from terminal device 30 to AI system 100. For example, persona information 42f may include a statement such as, "I went to the hospital because of a headache. I have already looked up information on the internet and have a rough idea of ​​what it might be." Subsequently, for example, a query 51c similar to that in Figure 21 is sent from terminal device 30 to AI system 100.

[0111] In this case, the AI ​​system 100 refers to the information in DB 111c and sends a response 54f to the terminal device 30 that includes a description of the symptoms as well as information that may be related to the cause of the illness. In the example in Figure 22, the response 54f sent is: "Hello. Thank you for examining me. Regarding my headache, I've been feeling particularly stressed lately, and I think I'm experiencing symptoms similar to tension headaches. My job at the library is busy, and I also feel stiffness in my shoulders and neck. I've been trying to relax by meditating and taking walks, but it's not getting any better." Response 54f explains that stress is a possible cause of the headache and that the symptoms are similar to those of a tension headache.

[0112] Comparing the examples in Figure 21 and Figure 22, a response 54f is generated using the information shown in the referenced DB111c (that stress is a cause of tension headaches) only when persona information 42f with an idea of ​​the illness is entered.

[0113] As shown in Figures 19 to 22, the AI ​​system 100 prevents responses that deviate from its knowledge if the persona information does not contain database information or information that can respond to the question, and it can handle detailed information if the persona contains information related to the question.

[0114] In the second embodiment, in addition to restricting information according to the persona's knowledge level, reference restrictions are also applied using the information detail level for each reference unit information within DB111a, 111b, .... This is to ensure that information unavailable to the persona is excluded from the response content when generating responses using LLM112. In other words, if agent 150 is instructed to restrict information based on the persona's knowledge level, agent 150 will apply restrictions according to the knowledge level to the information learned by LLM112. Even in this case, if information within DB111a, 111b, ... that exceeds the knowledge level remains accessible, that information may be referenced and reflected in the response. By applying reference restrictions to the information within DB111a, 111b, ... according to the persona's knowledge level, the generation of responses containing information of a difficulty level exceeding the persona's knowledge level is reliably suppressed.

[0115] [Other embodiments] The LLM112 in AI system 100 was not trained to mimic a specific persona. Therefore, LLM112 and agent 150 may be functions provided by a general-purpose AI system available from AI system 100 via network 20. In that case, AI system 100 sends prompts to a system such as a generative AI provided as an external service.

[0116] Although embodiments have been illustrated above, the configurations of each part shown in the embodiments can be replaced with others having similar functions. Furthermore, other arbitrary components or processes may be added. Moreover, any two or more configurations (features) from the embodiments described above may be combined. [Explanation of symbols]

[0117] 1. Language Model 2. Reference Information 3. Target Information 4. Question 1 5. Response 6,8 Prompt 7. Knowledge Level List 10 Information Processing Devices 11 Storage section 12 Processing Units 12a Response generation support unit 12b Agent

Claims

1. Based on the target information relating to the object to be imitated, a first knowledge domain in which the object to be imitated possesses knowledge, and a first knowledge level of the object to be imitated with respect to the first knowledge domain are determined. When a first question is input, an agent that responds to the question using a language model is instructed to respond to the first question by mimicking the thinking of the object to be mimicked, which is able to refer to information restricted based on the first knowledge domain and the first knowledge level. A response generation support program that directs a computer to perform processing.

2. The process for instructing a response to the first question involves estimating a second knowledge domain useful for responding to the first question, and estimating the second knowledge level of the object to be imitated with respect to the second knowledge domain based on the first knowledge domain and the first knowledge level, thereby instructing the generation of a response to the first question. The response generation support program according to claim 1.

3. The process that instructs the response to the first question instructs the estimation of the second knowledge level based on the first knowledge level of the first knowledge domain which is similar to the second knowledge domain. The response generation support program according to claim 2.

4. The process for instructing the response to the first question involves estimating a first level of detail, based on the second knowledge level, which indicates how detailed the information available to the imitation object is regarding the second knowledge domain, and instructing the imitation object to generate a response to the first question by limiting the information it can access to the information of the second knowledge domain that is at the first level of detail. The response generation support program according to claim 2.

5. In the process of determining the first knowledge domain and the first knowledge level, the agent is instructed to estimate the first knowledge domain and the first knowledge level based on the target information. The response generation support program according to claim 1.

6. The agent is instructed to estimate the level of detail of each reference unit in the reference information representing knowledge in each of the multiple knowledge domains, indicating how detailed the information is. The response generation support program according to claim 1, which causes a computer to perform further processing.

7. Based on the target information relating to the object to be imitated, a first knowledge domain in which the object to be imitated possesses knowledge, and a first knowledge level of the object to be imitated with respect to the first knowledge domain are determined. When a first question is input, an agent that responds to the question using a language model is instructed to respond to the first question by mimicking the thinking of the object to be mimicked, which is able to refer to information restricted based on the first knowledge domain and the first knowledge level. A method for supporting response generation, in which a computer performs the processing.

8. A processing unit determines, based on target information relating to the object to be imitated, a first knowledge domain in which the object to be imitated possesses knowledge, and a first knowledge level of the object to be imitated with respect to the first knowledge domain, and when a first question is input, instructs an agent that responds to the question using a language model to respond to the first question by imitating the thinking of the object to be imitated, which is able to refer to information restricted based on the first knowledge domain and the first knowledge level. An information processing device having

Images